We’ve released v1.1 of the Vals Index and Vals Multimodal Index.

The main change is the finance component: both indices now use the Finance Agent v2 index subset, scored as the mean of three runs per model. The rest of each index is unchanged.

GPT 5.5 is the new leader on both updated indices, scoring 67.62% on Vals Index v1.1 and 67.77% on Vals Multimodal Index v1.1.

Claude Opus 4.7 is close behind on both leaderboards, with Claude Sonnet 4.6 rounding out the top three.

We’ve released Finance Agent v2, the next generation of our benchmark for evaluating AI agents on realistic financial analyst tasks.

• Nine analytical categories spanning retrieval, market analysis, comparables, precedents, adjustments, earnings, disclosure, and financial modeling.
• Severity-weighted partial credit with dealbreaker gating: a question receives no credit when a load-bearing fact is wrong, regardless of how much of the surrounding answer is correct.
• Every model is evaluated three times on the full benchmark; reported scores are the mean across runs.

Compared to v1.1, scores fall ~14 points on average across models evaluated on both versions, with the steepest drop in Financial Modeling — where multi-step arithmetic and finance-specific conventions are least forgiving under the new rubric.

GPT 5.5, Claude Opus 4.7, and Claude Sonnet 4.6 are the top three performers, separated by less than a point.

We have updated the Vals Index and the Vals Multimodal Index to better reflect the capabilities that matter most when deploying frontier models to real economic work.

Vibe Code Bench now joins the coding bucket alongside SWE-bench Verified and Terminal-Bench 2.0.

The CaseLaw benchmark had become saturated, and was no longer providing useful differentiation between models. Consequently, it was removed from both indexes.

We evaluated Mistral Medium 3.5 on our suite of benchmarks. Here are the key takeaways:

• Mistral’s new reasoning-mode Medium 3.5 lands at #32 of 46 on the Vals Index (52.77%), and #10 of 18 among open-weight models.
• It is a sizable jump over the prior Mistral Large 3 across most of the suite: +28 points on Finance Agent (46.1% vs 18.1%), +29 points on the SWE-bench Verified subset of the Vals Index (64.7% vs 35.3%), +21 points on Terminal-Bench 2.0 (30.3% vs 9.0%), and +13 points on SAGE (37.6% vs 24.6%).
• One regression worth flagging: Case Law (v2) drops 17 points (44.2% vs 61.4%).

All evals were run via the Mistral API at temperature 0.7, top-p 0.95, with reasoning_effort: high and an 80k max output budget.