We have updated the Vals Index and the Vals Multimodal Index to better reflect the capabilities that matter most when deploying frontier models to real economic work.

Vibe Code Bench now joins the coding bucket alongside SWE-Bench and Terminal Bench 2.

The CaseLaw benchmark had become saturated, and was no longer providing useful differentiation between models. Consequently, it was removed from both indexes.

We evaluated Mistral Medium 3.5 on our suite of benchmarks. Here are the key takeaways:

• Mistral’s new reasoning-mode Medium 3.5 lands at #32 of 46 on the Vals Index (52.77%), and #10 of 18 among open-weight models.
• It is a sizable jump over the prior Mistral Large 3 across most of the suite: +28 points on Finance Agent (46.1% vs 18.1%), +29 points on the SWE-bench subset of the Vals Index (64.7% vs 35.3%), +21 points on Terminal-Bench 2.0 (30.3% vs 9.0%), and +13 points on SAGE (37.6% vs 24.6%).
• One regression worth flagging: Case Law (v2) drops 17 points (44.2% vs 61.4%).

All evals were run via the Mistral API at temperature 0.7, top-p 0.95, with reasoning_effort: high and an 80k max output budget.

Results for xAI’s Grok 4.3 are now available:

• Grok 4.3 places #13 of 46 on the Vals Index at 62.6%.
• It takes #1 on CorpFin (v2) (68.5%) and #1 on Case Law (v2) (79.3%).
• On Vibe Code Bench, it scores 19.4% — a +15-point improvement over its predecessor.

We evaluated Grok 4.3 with a temperature 0.7, top_p 0.95, and default top_k. The model supports a 1M-token context window, and had an average latency of 584.24 seconds per test at $0.38 per test on the index.

We evaluated Qwen 3.6 27B on our suite of benchmarks. Here are the key takeaways:

• Alibaba’s Qwen 3.6 27B is on the Vals leaderboard: #22 of 46 on the Vals Index and #8 of 18 among open-weight models.
• Head-to-head with the much larger Qwen 3.6 Plus, scores are within error margins of each other on the Vals Index, Vals Multimodal Index, as well as Case Law (v2), Mortgage Tax, SAGE, and Terminal-Bench 2.0.

All evals were run via the Alibaba API. We used temperature 1.0 by default, and 0.6 on coding benchmarks, which are the recommended settings on the Hugging Face model card. For agentic benchmarks we enabled preserve_thinking, following Qwen’s guidance.