Claude Opus 4.5 (Thinking) is the new leader on our Vals Index, surpassing its cheaper variant, Claude Sonnet 4.5 (Thinking).

Like other Anthropic models, Claude Opus 4.5 (Thinking) demonstrates strong performance on coding tasks - the thinking and nonthinking variants take the top 2 spots on SWE-Bench. However, the model performs worse than Claude Sonnet 4.5 (Thinking) on our new VibeCodeBench.

Unlike other Anthropic models, Claude Opus 4.5 (Thinking) showcases strong multimodal capabilities, placing first on our Multimodal Vals Index. It also places first on SAGE, surpassing the previous leader, Gemini 3 Pro (11/25). It’s also cheaper than before - it’s a third of the price of Claude Opus 4.1 (Nonthinking) and on some benchmarks, more token efficient than Claude Sonnet 4.5 (Thinking)!

Along with Claude Sonnet 4.5 (Thinking) and Claude Opus 4.5 (Thinking), the top 3 models on our Vals Index are now all from Anthropic. We look forward to other providers catching up!

We just released Vibe Code Bench : the first benchmark that tests whether AI models can actually build complete web applications from scratch.

• GPT 5.1 takes first place with 24% accuracy, improving by 4% over its third-place predecessor GPT 5 (with 20% accuracy).
• Claude Sonnet 4.5 (Thinking) takes second place, but costs more than twice as much than the OpenAI models.
• Despite the much-vaunted release, Gemini 3 Pro (11/25) lags behind in fourth place, with only 14.3% accuracy.

Overall, we see these results as evidence that vibe code has plenty of room to improve. We look forward to seeing this improvement!

We just evaluated Grok 4.1 Fast (Reasoning) and Grok 4.1 Fast Non-Reasoning on all benchmarks!

We found that the reasoning variant consistently outperforms the non-reasoning variant - see our comparison page. Grok 4.1 Fast Non-Reasoning struggles - its best performance is tenth place on our IOI benchmark, while Grok 4.1 Fast (Reasoning) achieves strong performance across the board, placing third on our CorpFin benchmark.

Both models perform similarly to their predecessors - see our comparison pages for both the reasoning and non-reasoning variants. Like their predecessors, both models are also extremely cost-efficient on a per-token basis.

Congratulations to the xAI team for the launch of Grok 4.1!

We just evaluated Gemini 3 Pro (11/25) on our Vals Index! Key takeaways:

Gemini 3 Pro (11/25) demonstrates strong performance across the board, placing first on SAGE, GPQA and MortgageTax and second on our Multimodal Vals Index.

Gemini 3 Pro (11/25) is three times faster than GPT 5.1, and is also cheaper than Claude Sonnet 4.5 (Thinking), the leader on the Vals Multimodal Index.

Overall, Gemini 3 Pro (11/25) excels in multimodal use cases and provides meaningful improvement over its predecessor, Gemini 2.5 Pro Exp.