The new open-source LLM Aya 23 will appear in two versions: one with 8 billion parameters and one with 35 billion parameters.
Both versions focus on model depth by combining a high-performant trained model with the Aya dataset collections. “The result is a powerful multilingual large language research model serving 23 languages, expanding state-of-the-art language modeling capabilities to nearly half of the world’s population,” Cohere said. High-performing language models typically support a handful of languages
Aya 23, however, can handle 23 languages, including Dutch. In addition, it supports Arabic, Chinese (simplified and traditional), Czech, English, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian and Vietnamese.
Cohere’s new model builds on Aya 101, which it previously developed to cover 101 languages. With that model, however, the focus was primarily on breadth, bringing AI to many companies. Aya 23 thus goes more for depth and should be able to answer many companies’ questions accurately.
The two versions
The version with 8 billion parameters focuses on serving the “everyday developer.” Because of its smaller size, it requires fewer resources than the larger model. Cohere sees in this version of the LLM a means of lowering the threshold of AI development. In its own words, it is democratizing access to advanced technology. The benchmarks below show how that advanced technology performs.
As you can see, Aya 23 scores better in the tests with 35 billion parameters. For this purpose, Aya 23 improves on the decoder-only Transformer architecture. Models with this design evaluate every word to determine intent and context. In this way, the LLM can provide more accurate responses than models relying on older techniques. However, this requires more computing power. The model is, therefore, intended for developers with decent requirements.
Cohere is one of the generative AI startups that has caught the attention of major tech companies in the past year and a half. It has raised hundreds of millions, and Nvidia and Oracle, among others, have invested in it.
Tip: Cohere aims to realize market value of 4.6 billion euros with investment round