Meta has announced the first models in the Llama 4 series. The new multimodal AI models, Llama 4 Scout and Llama 4 Maverick, offer strong contextual capacities and are the first open-weight models built with a mixture-of-experts (MoE) architecture. Meta claims these models perform better than competing solutions and are more efficient.
Llama 4 has two variants: Scout and Maverick. Scout has 17 billion active parameters with 16 experts and fits on a single NvidiaH100 GPU. This model supports a context window of 10 million tokens, which allows it to process enormous amounts of information at once. This allows it to outperform models such as Gemma 3 and Mistral 3.1 on various benchmarks.
The more powerful Llama 4 Maverick has the same 17 billion active parameters but with 128 experts and 400 billion total parameters. This model outperforms GPT-4o and Gemini 2.0 Flash on multiple benchmarks while achieving results comparable to the new DeepSeek v3 regarding reasoning and coding with less than half the active parameters.
Technological innovation
The mixture-of-experts architecture is an important innovation in these models. With this approach, only a fraction of the total parameters are activated per token, leading to significant efficiency improvements in training and inference. For example, Llama 4 Maverick contains 400 billion total parameters, but uses only a portion of them for each calculation.
Another important innovation is the native multimodality with early fusion, which allows text and image tokens to be seamlessly integrated into the model. Meta has also developed an improved vision encoder based on MetaCLIP, but it is trained separately in combination with a frozen Llama model.
In addition to the two available models, Meta also gave a preview of Llama 4 Behemoth, a 288 billion active parameter model with 16 experts and almost two trillion total parameters. According to Meta, this model, which is still being trained, outperforms GPT-4.5, Claude Sonnet 3.7 and Gemini 2.0 Pro on various STEM benchmarks. Behemoth served as a ‘teacher’ for the smaller Llama 4 models during training.
Meta emphasizes that it continues to believe in openness as a driver for innovation. That is why Llama 4 Scout and Llama 4 Maverick are now available for download at llama.com and Hugging Face. Developers can use these models to build new applications.