Open-source AI specialist Mistral has recently introduced its new LLM Mixtral 8x22B. This LLM is the successor to the Mixtral 8x7B LLM and should be an equal competitor to LLMs such as OpenAI’s GPT 3.5 and Meta’s Llama 2.
Mistral AI is stepping up its competition with the major LLM providers and presents its latest open-source LLM, Mixtral 8x22B. The context window consists of 65,000 tokens, which refers to the amount of text it can process and reference simultaneously.
In addition, the new LLM has 176 billion parameters, or variables, with which it can make decisions and possible predictions.
Better performance
According to the French open-source AI specialist, the new model outperforms the previous Mixtral 8x7B top model. The model features an advanced mixture-of-experts architecture for efficient computations and high performance for different tasks.
This architecture uses a combination of different models, each specifically ‘working’ for a distinct category of tasks. This should improve performance and reduce (computational) costs.
As a result, despite its huge size of 176 billion parameters, the new LLM only needs to use 44 billion active parameters for each step in the computational process or ‘forward pass’. According to Mistral AI, this makes it faster and also more cost-effective than comparable LLMs such as GPT 3.5 and Llama 2.
‘Silent’ introduction via torrent
The new Mixtral 8x22B model was more or less quietly launched recently. A remarkable torrent link appeared on the French AI specialist’s X account on April 10, allowing users to download the new model. Mistral AI’s new LLM later became available on AI platforms HuggingFace and Together AI.
At the time of writing, Mistral AI has not announced the arrival of the Mixtral 8x22B LLM.
More new LLM launches
Mistral AI is not the only AI giant introducing new versions of LLMs this week. OpenAI recently presented the latest iteration of GPT 4-Turbo, GPT-4 Turbo with Vision. Google did not stay behind and made the Gemini Pro 1.5 LLM generally available at Google Next ’24. Meta further announced that it will still present Llama 3 this month.
Also read: Llama 3 available within a few weeks