Apertus: a fully open, transparent, multilingual language model
ethz.ch
EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus 2 September, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

"Apertus: a fully open, transparent, multilingual language model

EPFL, ETH Zurich and the Swiss National Supercomputing Centre (CSCS) released Apertus 2 September, Switzerland’s first large-scale, open, multilingual language model — a milestone in generative AI for transparency and diversity.

Researchers from EPFL, ETH Zurich and CSCS have developed the large language model Apertus – it is one of the largest open LLMs and a basic technology on which others can build.

In brief Researchers at EPFL, ETH Zurich and CSCS have developed Apertus, a fully open Large Language Model (LLM) – one of the largest of its kind. As a foundational technology, Apertus enables innovation and strengthens AI expertise across research, society and industry by allowing others to build upon it. Apertus is currently available through strategic partner Swisscom, the AI platform Hugging Face, and the Public AI network. …

The model is named Apertus – Latin for “open” – highlighting its distinctive feature: the entire development process, including its architecture, model weights, and training data and recipes, is openly accessible and fully documented.

AI researchers, professionals, and experienced enthusiasts can either access the model through the strategic partner Swisscom or download it from Hugging Face – a platform for AI models and applications – and deploy it for their own projects. Apertus is freely available in two sizes – featuring 8 billion and 70 billion parameters, the smaller model being more appropriate for individual usage. Both models are released under a permissive open-source license, allowing use in education and research as well as broad societal and commercial applications. …

Trained on 15 trillion tokens across more than 1,000 languages – 40% of the data is non-English – Apertus includes many languages that have so far been underrepresented in LLMs, such as Swiss German, Romansh, and many others. …

Furthermore, for people outside of Switzerland, the external pagePublic AI Inference Utility will make Apertus accessible as part of a global movement for public AI. “Currently, Apertus is the leading public AI model: a model built by public institutions, for the public interest. It is our best proof yet that AI can be a form of public infrastructure like highways, water, or electricity,” says Joshua Tan, Lead Maintainer of the Public AI Inference Utility."

  • hendrik@palaver.p3x.deEnglish
    3·
    8 hours ago

    They’re fairly transparent with everything. You could call it open-source. And it’s supposed to be the first large model which complies with the EU AI regulations. They try to make an effort not to include too much material from people who objected to AI use, and there is a way to opt out. They did not deliberately pirate books like Meta did. But with that said, it’s still AI. Training needs a lot of water and energy. Though I think this Alps supercomputing center tries to be carbon-neutral and use Swiss hydropower. Whatever that means in practice. Opt-out is probably the best thing we can do but it’s not exactly consent from the authors of the training material. And I don’t think there is a way to compensate them. And AI can of course be problematic once used, so that depends on what people do with it.

    I’d call it more ethical (than other models). But I don’t see how it’d be strictly ethical in absolute terms. Looks to me like an effort to improve, maybe substantially on what other people did. But there’s still a lot of problematic aspects of AI which scientists and society hasn’t addressed yet.