We just evaluated Grok 4.1 Fast (Reasoning) and Grok 4.1 Fast Non-Reasoning on all benchmarks!

We found that the reasoning variant consistently outperforms the non-reasoning variant - see our comparison page. Grok 4.1 Fast Non-Reasoning struggles - its best performance is tenth place on our IOI benchmark, while Grok 4.1 Fast (Reasoning) achieves strong performance across the board, placing third on our CorpFin benchmark.

Both models perform similarly to their predecessors - see our comparison pages for both the reasoning and non-reasoning variants. Like their predecessors, both models are also extremely cost-efficient on a per-token basis.

Congratulations to the xAI team for the launch of Grok 4.1!

We just evaluated Gemini 3 Pro (11/25) on our Vals Index! Key takeaways:

Gemini 3 Pro (11/25) demonstrates strong performance across the board, placing first on SAGE, GPQA and MortgageTax and second on our Multimodal Vals Index.

Gemini 3 Pro (11/25) is three times faster than GPT 5.1, and is also cheaper than Claude Sonnet 4.5 (Thinking), the leader on the Vals Multimodal Index.

Overall, Gemini 3 Pro (11/25) excels in multimodal use cases and provides meaningful improvement over its predecessor, Gemini 2.5 Pro Exp.

We just evaluated GPT 5.1 on all benchmarks. Our findings are below:

• GPT 5.1 takes second place on the Vals Index, behind the reigning champion, Claude Sonnet 4.5 (Thinking).
• GPT 5.1 shines on agent benchmarks, placing first on our Finance Agent Benchmark.
• GPT 5.1 also excels at competitive coding, placing second on both LiveCodeBench and our IOI Benchmark.
• GPT 5.1 performs comparably to GPT 5, but much more cheaply and quickly - we consider it a clear leader on Pareto frontier between performance and cost.

We evaluated Kimi K2 Thinking on all benchmarks, finding strong performance on coding benchmarks like SWE-bench. We otherwise found middle-of-the-pack performance on both public and private benchmarks.

In comparison to Kimi K2 Instruct, we found a modest improvement across most industries, with the exception of legal and finance. For instance, the Instruct model outperforms the thinking model by nearly 5% on our TaxEval benchmark.

Overall, we found Kimi K2 Thinking to be a strong successor to Kimi K2 Instruct, but doesn’t live up to the twitter hype of “best model on the market”.