We evaluated Grok 4.20 Beta (Reasoning) across our full benchmark suite.

• Grok 4.20 Beta (Reasoning) ranks #13/33 on Vals Index (57.70% accuracy) and #14/23 on Vals Multimodal Index (53.96%).
• It achieves strong results on academic and coding-heavy benchmarks, including #5 on AIME (96.46%), #6 on GPQA (88.64%), #9 on MMMU (83.47%), and #9 on SWE-bench (74.20%).
• On our finance and tax benchmarks, it delivers solid performance with #15 on Corp Fin (v2) (63.68%), #14 on Finance Agent (v1.1) (52.29%), and #16 on Tax Eval (v2) (74.12%).
• On healthcare tasks, performance is mixed: #16 on MedQA (94.55%), alongside #36 on MedCode (32.16%) and #43 on MedScribe (63.41%).
• On legal benchmarks, it currently ranks #30 on Case Law (v2) (54.45%) and #62 on LegalBench (77.74%).

Overall, the model generally is an improvement over previous xAI models, with room for improvement in certain domains.

Evaluations were run with a temperature of 0.7 and a top_p of 0.95 via the xAI API. This model is still in beta, and we will update results as and when updates are released by xAI.

We just evaluated Grok 4.20 Beta (Reasoning) on Vals Index.

• Grok 4.20 Beta (Reasoning) lands at #13 overall on Vals Index.
• On the Corp Fin (v2) split of our index, it posts 67.1% accuracy, and on the SWE-bench split, it reaches 72.55.6%.

Evaluations were run with a temperature of 0.7 and a top_p of 0.95 via the xAI API. This model is still in Beta, results will be updated after the official release.

We evaluated GPT 5.4 across our full benchmark suite.

• GPT 5.4 ranks #4 on both our Vals Index and #3 on the Vals Multimodal Index.
• It takes #1 on Vibe Code Bench, Proof Bench, and IOI.
• On Proof Bench, GPT 5.4 reaches 56% accuracy (up from sub 20% for prior OpenAI models, and beating Claude Opus 4.6 (Thinking) by 6%).
• Relative to GPT 5.2, we see especially large gains on coding/math-heavy benchmarks, including Vibe Code Bench (+13.9%), IOI (+13.0%), and Terminal Bench 2 (+6.7%), along with a smaller margin of improvement on SWE-bench (+1.8%).
• We also observe a few areas where GPT 5.4 is roughly flat or slightly behind GPT 5.2 in our current data (for example, GPQA is tied, while LCB and Finance Agent have GPT 5.2 slightly higher).

Evaluations were run via the official OpenAI API using “xhigh” reasoning, except on Terminal Bench 2, where we used “high”.

We evaluated Gemini 3.1 Flash Lite Preview across our full benchmark suite. The model is Google’s fast and cost-efficient offering.

• The model does well on two of our multimodal benchmarks, SAGE (ranked 5th, with 49.5% accuracy) and Mortgage Tax (ranked 7th, 67.8% accuracy).
• It ranks 14th on MMLU Pro with 86.2% accuracy.
• On coding tasks the model still has room for improvement—it ranks 29th on both Live Code Bench and Terminal Bench 2, and 28th on SWE-bench. It scores 0% on Vibe Code Bench.
• Overall, it places 15th/20 on the Vals Multimodal Index and 22nd/31 on the Vals Index.
• The cost savings compared to other models in the Gemini 3 series or Gemini 2.5 are dramatic: roughly 5–20x cheaper per test across benchmarks, while maintaining respectable accuracy. For example, on Finance Agent it costs $0.072 per test vs $0.370 for Gemini 3 Flash, while performing comparably.

Our results show that the model does not perform as well as other models in the Gemini 3 series. However, it is fast and quite cost-efficient relative to those models, making it a good choice for applications that demand scale, speed, or cost-efficiency.

Evaluations were run with a temperature of 1.0 and a “high” thinking level, via the official Google API.