We’ve done a full review of the data, harness, and evaluation for Finance Agent, and are releasing it today as “FAB v1.1”!

• On the data, we ran quality control on the benchmark data using finance experts from leading global investment banks, private equity firms, and hedge funds, including Goldman Sachs, Silver Lake, and Citadel.
• We made several notable improvements to the harness, such as upgrading the search provider to Tavily, switching submission to a tool call, and improving instructions on rounding.
• Claude Opus 4.6 (Thinking) remains on top, although many models like Gemini 3 Pro (11/25) and Grok 4.1 Fast (Reasoning) saw major improvements.

We will continue to release refreshes on our benchmarks to ensure quality remains at the same high standard.

The newly-released Claude Opus 4.6 (Thinking) has taken first place on both our Vals Index and Vals Multimodal Index, both by substantial margins. Here are our key takeaways:

• Claude Opus 4.6 (Thinking) is head and shoulders above the competition on coding tasks, placing first on by several percent on SWE-bench and tying Claude Opus 4.5 (Thinking) for first on Terminal Bench 2.
• Claude Opus 4.6 (Thinking) also excels across other domains, placing first on our Finance Agent Benchmark and our new formal math benchmark, ProofBench.
• Claude Opus 4.6 (Thinking) is performant across the board, placing among the top 10 across all the benchmarks on which we’ve evaluated it so far, with the sole exception of CaseLaw.
• Claude Opus 4.6 (Thinking) is fast - it’s faster than comparable GPT models (though slower on our index than Gemini 3 Pro (11/25)) and the fastest model on our LegalBench leaderboard.

Congrats to Anthropic on the release!

We found that Qwen 3 Max Thinking shines on financial tasks, but struggles on coding evaluations when compared to its predecessor, Qwen 3 Max. We attribute the difference to use of new “reasoning” mode like those featured by closed-source providers.

Its most impressive finish is second on CorpFin, behind Kimi K2.5. It also improves by 10% over its predecessor on our Finance Agent Benchmark.

By contrast, it performs worse than Qwen 3 Max on both SWE-bench and Terminal-Bench 2, placing 20th and 25th, respectively.

We also found that the model can get expensive on long-context agentic tasks, with the highest pricing tier equivalent to that of the Claude Sonnet models. Additionally, the model is worse than its predecessor at context caching, which increases prices per token further.

Congrats to the team at Alibaba on the release!

We’re releasing ProofBench, a new benchmark evaluating whether models can write formally verified mathematical proofs. Unlike natural language math benchmarks where subtle errors can go unnoticed, ProofBench requires proofs that are machine-checkable; either a solution compiles or it doesn’t.

• Aristotle, a system from Harmonic, achieved the best performance at 71%, roughly 2x the score of the best foundation model.
• Foundation models struggle significantly withs formal proof generation. Claude Opus 4.5 (Thinking) leads foundation models at 36%, followed by Gemini 3 Pro (11/25) at 20% and Claude Sonnet 4.5 (Thinking) at 19%.
• Problems are drawn from advanced undergraduate and graduate-level mathematics, spanning probability theory, measure theory, analysis, algebra, algebraic geometry, and number theory.
• While specialized tools like Aristotle show promise for assisting with formal mathematics, the large gap in foundation model performance highlights significant room for improvement in reliable formal reasoning.