Aider Polyglot

At a glance

Methodology

225 exercism-style coding exercises in C++, Go, Java, JavaScript, Python and Rust. Score = pass_rate_2 (% of cases where the model's second attempt produces a passing solution).

Limitations

Aider's harness shapes prompts in a specific way — results may not directly compare to other coding benchmarks.

By provider

• OpenAI· 6 models

  88 % pass@2

  GPT-5
  Average: 55.25 % pass@2Best: 88 % pass@2
• Google· 1 model

  72.9 % pass@2

  Gemini 2.5 Pro
  Average: 72.9 % pass@2Best: 72.9 % pass@2
• DeepSeek· 2 models

  56.9 % pass@2

  DeepSeek R1
  Average: 56 % pass@2Best: 56.9 % pass@2
• xAI· 1 model

  53.3 % pass@2

  Grok 3
  Average: 53.3 % pass@2Best: 53.3 % pass@2
• Alibaba· 1 model

  40 % pass@2

  Qwen: Qwen3 32B
  Average: 40 % pass@2Best: 40 % pass@2
• Meta· 1 model

  15.6 % pass@2

  Llama 4 Maverick
  Average: 15.6 % pass@2Best: 15.6 % pass@2
• Mistral AI· 1 model

  11.1 % pass@2

  Codestral
  Average: 11.1 % pass@2Best: 11.1 % pass@2