🐳 We just evaluated DeepSeek’s R1 model on three of our private datasets! 🐳

• The model demonstrates its strong reasoning ability, rivaling Open AI’s o1 model on our Tax dataset.
• However, R1 performs extremely poorly on ContractLaw and with middling performance on CaseLaw. The model’s performance is not uniform, suggest task-specific evaluation must be done before adoption
• Overall, this large Chinese model shows impressive ability and further closes the gap between closed and open-source models.