The newly-released Claude Opus 4.6 (Thinking) has taken first place on both our Vals Index and Vals Multimodal Index, both by substantial margins. Here are our key takeaways:

• Claude Opus 4.6 (Thinking) is head and shoulders above the competition on coding tasks, placing first on by several percent on SWE-bench and tying Claude Opus 4.5 (Thinking) for first on Terminal Bench 2.
• Claude Opus 4.6 (Thinking) also excels across other domains, placing first on our Finance Agent Benchmark and our new formal math benchmark, ProofBench.
• Claude Opus 4.6 (Thinking) is performant across the board, placing among the top 10 across all the benchmarks on which we’ve evaluated it so far, with the sole exception of CaseLaw.
• Claude Opus 4.6 (Thinking) is fast - it’s faster than comparable GPT models (though slower on our index than Gemini 3 Pro (11/25)) and the fastest model on our LegalBench leaderboard.

Congrats to Anthropic on the release!

We found that Qwen 3 Max Thinking shines on financial tasks, but struggles on coding evaluations when compared to its predecessor, Qwen 3 Max. We attribute the difference to use of new “reasoning” mode like those featured by closed-source providers.

Its most impressive finish is second on CorpFin, behind Kimi K2.5. It also improves by 10% over its predecessor on our Finance Agent Benchmark.

By contrast, it performs worse than Qwen 3 Max on both SWE-bench and Terminal-Bench 2, placing 20th and 25th, respectively.

We also found that the model can get expensive on long-context agentic tasks, with the highest pricing tier equivalent to that of the Claude Sonnet models. Additionally, the model is worse than its predecessor at context caching, which increases prices per token further.

Congrats to the team at Alibaba on the release!

We’re releasing ProofBench, a new benchmark evaluating whether models can write formally verified mathematical proofs. Unlike natural language math benchmarks where subtle errors can go unnoticed, ProofBench requires proofs that are machine-checkable; either a solution compiles or it doesn’t.

• Aristotle, a system from Harmonic, achieved the best performance at 71%, roughly 2x the score of the best foundation model.
• Foundation models struggle significantly withs formal proof generation. Claude Opus 4.5 (Thinking) leads foundation models at 36%, followed by Gemini 3 Pro (11/25) at 20% and Claude Sonnet 4.5 (Thinking) at 19%.
• Problems are drawn from advanced undergraduate and graduate-level mathematics, spanning probability theory, measure theory, analysis, algebra, algebraic geometry, and number theory.
• While specialized tools like Aristotle show promise for assisting with formal mathematics, the large gap in foundation model performance highlights significant room for improvement in reliable formal reasoning.

We’ve finished our evaluation of Kimi K2.5 and continue to find excellent performance across the board. Here are some highlights:

• The model places first on CorpFin by a significant margin (2%), though it takes substantially longer to respond than other models in the top 10.
• The model is remarkably consistent - its worst placement is 16th, on CaseLaw, on which it remains within 10% of first place.
• The model is head and shoulders above its open-weight competition, placing first among open-weight models on 13 of the 17 benchmarks on which it was evaluated.

We ran all benchmarks on the native provider with temperature 1 as recommended by Kimi, with 128K max tokens on all but the coding benchmarks, for which we used 256K.

Stay tuned for our evaluation on Vibe Code Bench and congrats again to Moonshot on the release!