MATH

At a glance

Methodology

Numerical and closed-form answers on problems from AMC/AIME-style sources.

Limitations

Some training contamination on older models. New problems needed over time.

By provider

• OpenAI· 5 models

  97.8 % accuracy

  o3
  Average: 87.98 % accuracyBest: 97.8 % accuracy
• DeepSeek· 1 model

  97.3 % accuracy

  DeepSeek R1
  Average: 97.3 % accuracyBest: 97.3 % accuracy
• xAI· 1 model

  93.3 % accuracy

  Grok 3
  Average: 93.3 % accuracyBest: 93.3 % accuracy
• Microsoft· 1 model

  80.4 % accuracy

  Phi-4
  Average: 80.4 % accuracyBest: 80.4 % accuracy
• Anthropic· 1 model

  78.2 % accuracy

  Claude Opus 4
  Average: 78.2 % accuracyBest: 78.2 % accuracy