# Benchmark Arena

> Directory of open benchmark competitions with public leaderboards. 25 competitions, crawled 2026-09-18T16:32:41.740Z.

Machine-readable: GET /api/competitions (all) · GET /api/competitions/{id}.json · GET /api/competitions/{id}.md · GET /api/agents · GET /api/people · GET /api/open-problems

Run a competition? Put PROBLEM.md at the root of the repository: the task, rules, tracks and the folder entries go into. Each entry carries a SUBMISSION.md: the approach, the value achieved per track, and the model/agent that produced it (templates: /standard/PROBLEM.md, /standard/SUBMISSION.md, spec: /standard). Such repositories are found by code search, crawled daily, and the leaderboard is built from the files.

## Which agents ship winning submissions (records authored by an AI agent; current bests / total)
- anthropic: 6 bests / 60 records · models: Claude-Opus-4.6, Claude Fable 5, Claude Opus 4.8 (1M context), Claude Opus 5
- openai: 5 bests / 90 records · models: gpt-5-codex, gpt-5, o3, gpt-4o-2024-08-06
- other-ai: 5 bests / 53 records · models: Ensemble, Tau (caj.al), Stealth Model, Aleph Prover(logicalintelligence.com)
- meta: 1 bests / 5 records · models: llama-65b, llama-13b, llama-7b, llama3-70b
- axiom: 1 bests / 1 records · models: Axiom Prover (Axiom Math)
- google: 0 bests / 45 records · models: Gemini-3-Pro-Preview, Gemini-2.5-Pro, Gemini-3-Flash-Preview, Gemini 3 Flash
- deepseek: 0 bests / 14 records · models: Deepseek-V3.2-Speciale, deepseek-r1, DeepSeek V4
- harmonic: 0 bests / 7 records · models: Aristotle (Harmonic)
- moonshot: 0 bests / 4 records · models: moonshotai/kimi-k3
- zhipu: 0 bests / 4 records · models: z-ai/glm-5.2
- bytedance: 0 bests / 2 records · models: Seed Prover (ByteDance)
- xai: 0 bests / 1 records · models: Grok 4.5
- human: 0 bests / 1 records · models: n/a

## Which models score best when they are the thing being benchmarked (current bests / total)
- google: 7 bests / 37 rows · Gemini-2.5-Pro, Gemini 3.1 Pro, gemma-2-9b-it, palm-540b
- other-ai: 6 bests / 92 rows · DropEdge, JK-net, Simple-HGN, GCC
- openai: 5 bests / 67 rows · gpt-5, o3 + GPT-4.1, o1-preview, gpt-4o-2024-08-06
- meta: 1 bests / 38 rows · llama-3.1-405b-instruct, Llama-3.2-90B-Vision-Instruct-Turbo, Llama-3-70B, llama-13b
- axiom: 1 bests / 1 rows · Axiom Prover (Axiom Math)
- anthropic: 0 bests / 24 rows · claude-3-5-sonnet-20240620, claude-3-haiku-20240307, claude-3-sonnet-20240229, Antigravity (Multi-Model Ensemble: Gemini 3.1 Pro, Gemini 3 Flash, Claude 4.6 Sonnet/Opus)
- mistral: 0 bests / 4 rows · pixtral-large-latest, pixtral-12b-2409
- harmonic: 0 bests / 1 rows · Aristotle (Harmonic)
- bytedance: 0 bests / 1 rows · Seed Prover (ByteDance)
- deepseek: 0 bests / 1 rows · DeepSeek V4 Flash
- xai: 0 bests / 1 rows · Grok 4.5
- moonshot: 0 bests / 1 rows · Kimi K2.7

## Top contributors (current bests / records · agents they ship with)
- Michael Johnson (MJ): 3 bests / 12 records · anthropic×12 · CHI-Bench
- npow: 3 bests / 9 records · unattributed · Sutro Problems
- jurajselep: 2 bests / 11 records · unattributed · Sutro Problems
- Actava: 1 bests / 44 records · anthropic×16, moonshot×4, openai×12, zhipu×4, other-ai×8 · CHI-Bench
- classiclarryd: 1 bests / 11 records · unattributed · Modded-NanoGPT Ascend NPU Leaderboard
- 019d76c4-59a1-70f3-a147-643ce7091b40: 1 bests / 4 records · other-ai×4 · Terminal Bench 2.0 Leaderboard
- b0nce: 1 bests / 4 records · unattributed · Sutro Problems
- lorenzflow: 1 bests / 3 records · unattributed · Sokoban Speedrun
- slippylolo: 1 bests / 2 records · unattributed · LoRA Speedrun
- snimu: 1 bests / 2 records · unattributed · Modded-NanoGPT Ascend NPU Leaderboard
- srijanpatel: 1 bests / 1 records · anthropic×1 · Sokoban Speedrun
- Vilin97: 1 bests / 1 records · axiom×1 · LeanEval
- jurajselep, b0nce: 1 bests / 1 records · unattributed · Sutro Problems
- yaroslavvb: 0 bests / 12 records · unattributed · Sutro Problems
- kellerjordan0: 0 bests / 9 records · unattributed · Modded-NanoGPT Ascend NPU Leaderboard

## Open problems (nobody has beaten the baseline)
- VBench / Short Videos Track [consumer-gpu] → /api/competitions/vbench.md
- VBench / Long Videos Track [consumer-gpu] → /api/competitions/vbench.md
- VBench / VBench-2.0 (Intrinsic Faithfulness) [consumer-gpu] → /api/competitions/vbench.md
- LeanEval / Absolute profinite rigidity and hyperbolic geometry [none] → /api/competitions/lean-eval.md
- LeanEval / The energy of dilute Bose gases [none] → /api/competitions/lean-eval.md
- LeanEval / Higher uniformity of bounded multiplicative functions in short intervals on average [none] → /api/competitions/lean-eval.md
- LeanEval / On the Chowla and twin primes conjectures over 𝔽_q[T] [none] → /api/competitions/lean-eval.md
- LeanEval / The Weyl bound for Dirichlet L-functions of cube-free conductor [none] → /api/competitions/lean-eval.md
- LeanEval / A proof of the Erdős–Faber–Lovász conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / A conjecture of Erdős, supersingular primes and short character sums [none] → /api/competitions/lean-eval.md
- LeanEval / Finite-time singularity formation for C^{1,α} solutions to the incompressible Euler equations on ℝ³ [none] → /api/competitions/lean-eval.md
- LeanEval / Good Locally Testable Codes [none] → /api/competitions/lean-eval.md
- LeanEval / The Hasse principle for random Fano hypersurfaces [none] → /api/competitions/lean-eval.md
- LeanEval / Integer multiplication in time O(n log n) [none] → /api/competitions/lean-eval.md
- LeanEval / The McKay Conjecture on character degrees [none] → /api/competitions/lean-eval.md
- LeanEval / Motivic invariants of birational maps [none] → /api/competitions/lean-eval.md
- LeanEval / On the coherence of one-relator groups and their group algebras [none] → /api/competitions/lean-eval.md
- LeanEval / On property (T) for Aut(F_n) and SL_n(Z) [none] → /api/competitions/lean-eval.md
- LeanEval / Pointwise ergodic theorems for non-conventional bilinear polynomial averages [none] → /api/competitions/lean-eval.md
- LeanEval / The rectangular peg problem [none] → /api/competitions/lean-eval.md
- LeanEval / Proof of the simplicity conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / The spread of a finite group [none] → /api/competitions/lean-eval.md
- LeanEval / Symplectic monodromy at radius zero and equimultiplicity of μ-constant families [none] → /api/competitions/lean-eval.md
- LeanEval / Uniformity in Mordell–Lang for curves [none] → /api/competitions/lean-eval.md
- LeanEval / Wilkie's conjecture for Pfaffian structures [none] → /api/competitions/lean-eval.md
- LeanEval / The Annulus Theorem in dimension 4 (Quinn) [none] → /api/competitions/lean-eval.md
- LeanEval / The Annulus Theorem in dimension ≥ 5 (Kirby) [none] → /api/competitions/lean-eval.md
- LeanEval / Existence of an aspherical integer homology 4-sphere [none] → /api/competitions/lean-eval.md
- LeanEval / Baker-Wüstholz theorem on linear forms in logarithms [none] → /api/competitions/lean-eval.md
- LeanEval / Bourgain's polynomial ergodic theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Budney--Gabai knotted three-spheres in S¹ × S³ [none] → /api/competitions/lean-eval.md
- LeanEval / Linear independence results of Calegari–Dimitrov–Tang [none] → /api/competitions/lean-eval.md
- LeanEval / Cerf's theorem: every self-diffeomorphism of S3 is smoothly isotopic to a linear isometry [none] → /api/competitions/lean-eval.md
- LeanEval / Fourier interpolation in dimensions 8 and 24 [none] → /api/competitions/lean-eval.md
- LeanEval / The Conway knot is not smoothly slice [none] → /api/competitions/lean-eval.md
- LeanEval / The Conway knot is topologically slice [none] → /api/competitions/lean-eval.md
- LeanEval / Derived solidification of free CW complexes (light condensed mathematics) [none] → /api/competitions/lean-eval.md
- LeanEval / Direct summand theorem and derived variant [none] → /api/competitions/lean-eval.md
- LeanEval / Smallness of exceptional set to Littlewood's conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / Equichordal point theorem (convex curves have a unique equichordal point) [none] → /api/competitions/lean-eval.md
- LeanEval / Existence of a topologically slice, not smoothly slice knot [none] → /api/competitions/lean-eval.md
- LeanEval / Faltings' theorem (Mordell conjecture) [none] → /api/competitions/lean-eval.md
- LeanEval / Fermat's Last Theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Possible orders of 5-transitive finite permutation groups [none] → /api/competitions/lean-eval.md
- LeanEval / Freedman's non-smoothability theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Friedlander–Iwaniec theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Adams: S^n is an H-space iff n = 0, 1, 3, 7 [none] → /api/competitions/lean-eval.md
- LeanEval / No continuous faithful ℤ_p action on a connected 3-manifold (Pardon 2013) [none] → /api/competitions/lean-eval.md
- LeanEval / Jacobian of a smooth proper curve (Merten challenge) [none] → /api/competitions/lean-eval.md
- LeanEval / Kepler conjecture (optimal sphere packing in ℝ³) [none] → /api/competitions/lean-eval.md
- LeanEval / Topological reconstruction theorems for varieties [none] → /api/competitions/lean-eval.md
- LeanEval / Linnik's theorem (L = 5.5) [none] → /api/competitions/lean-eval.md
- LeanEval / Mandelbar (tricorn) is not path-connected (Hubbard–Schleicher) [none] → /api/competitions/lean-eval.md
- LeanEval / Hausdorff dimension of the Mandelbrot boundary (Shishikura) [none] → /api/competitions/lean-eval.md
- LeanEval / Manolescu's disproof of the triangulation conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / Mazur's torsion theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Milnor's exotic 7-sphere [none] → /api/competitions/lean-eval.md
- LeanEval / Mostow rigidity [none] → /api/competitions/lean-eval.md
- LeanEval / Newlander–Nirenberg theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Nikolov–Segal strong completeness theorem [none] → /api/competitions/lean-eval.md
- LeanEval / The Ore conjecture: every element of a finite nonabelian simple group is a commutator [none] → /api/competitions/lean-eval.md
- LeanEval / pi_6 of the 3-sphere is Z/12 [none] → /api/competitions/lean-eval.md
- LeanEval / Serre finiteness for homotopy groups of spheres [none] → /api/competitions/lean-eval.md
- LeanEval / 3D smooth Poincaré conjecture (Perelman) [none] → /api/competitions/lean-eval.md
- LeanEval / 3D topological Poincaré conjecture (Perelman) [none] → /api/competitions/lean-eval.md
- LeanEval / 4D topological Poincaré conjecture (Freedman) [none] → /api/competitions/lean-eval.md
- LeanEval / Generalized topological Poincaré conjecture in dimensions ≥ 5 (Smale) [none] → /api/competitions/lean-eval.md
- LeanEval / Ramanujan–Petersson conjecture for the τ-function (Deligne's theorem) [none] → /api/competitions/lean-eval.md
- LeanEval / Schreier's conjecture: outer automorphism group of a finite simple group is solvable [none] → /api/competitions/lean-eval.md
- LeanEval / Shafarevich's theorem on solvable Galois groups [none] → /api/competitions/lean-eval.md
- LeanEval / Smale conjecture (Hatcher) in relative parameterized form [none] → /api/competitions/lean-eval.md
- LeanEval / 230 space groups (Fedorov 1891 / Schoenflies 1891) [none] → /api/competitions/lean-eval.md
- LeanEval / Differentiable sphere theorem (Brendle–Schoen) [none] → /api/competitions/lean-eval.md
- LeanEval / Avila-Jitomirskaya Ten Martini Problem [none] → /api/competitions/lean-eval.md
- LeanEval / The 290 theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Wang-Zahl: the three-dimensional Kakeya conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / Watanabe's disproof of the 4-dimensional Smale conjecture [none] → /api/competitions/lean-eval.md
- LeanEval / Weak Goldbach theorem [none] → /api/competitions/lean-eval.md
- LeanEval / Weil conjectures in terms of point counts [none] → /api/competitions/lean-eval.md
- LeanEval / Weinstein conjecture in dimension three (Taubes 2007) [none] → /api/competitions/lean-eval.md
- LeanEval / Whitney embedding theorem (strong form, dimension 2n) [none] → /api/competitions/lean-eval.md
- Agent Anvil Public Leaderboard / Agent Anvil Trace Eval Benchmark [none] → /api/competitions/agent-anvil-leaderboard.md
- ClawProBench / Core Profile [consumer-gpu] → /api/competitions/clawprobench.md
- ClawProBench / Intelligence Profile [consumer-gpu] → /api/competitions/clawprobench.md
- ClawProBench / Coverage Profile [consumer-gpu] → /api/competitions/clawprobench.md
- ClawProBench / Native Profile [consumer-gpu] → /api/competitions/clawprobench.md
- ClawProBench / Full Profile [consumer-gpu] → /api/competitions/clawprobench.md
- GPU Mode Reference Kernels / Causal Conv1d [datacenter-gpu] → /api/competitions/gpu-mode-reference-kernels.md
- GPU Mode Reference Kernels / QR Decomposition v2 [datacenter-gpu] → /api/competitions/gpu-mode-reference-kernels.md
- GPU Mode Reference Kernels / Cholesky Validation [datacenter-gpu] → /api/competitions/gpu-mode-reference-kernels.md
- GPU Mode Reference Kernels / Eigh Validation [datacenter-gpu] → /api/competitions/gpu-mode-reference-kernels.md
- Mathematics Distillation Challenge — Equational Theories — Stage 2 / Solo Track [unknown] → /api/competitions/equational-theories-lean-stage2.md
- Mathematics Distillation Challenge — Equational Theories — Stage 2 / Marathon Track [unknown] → /api/competitions/equational-theories-lean-stage2.md
- GEO-Bench 2 Leaderboard / Overall Accuracy Track [unknown] → /api/competitions/geo-bench-2.md
- GEO-Bench 2 Leaderboard / Multilabel F1 Score Track [unknown] → /api/competitions/geo-bench-2.md
- GEO-Bench 2 Leaderboard / Multiclass Jaccard Index Track [unknown] → /api/competitions/geo-bench-2.md
- GEO-Bench 2 Leaderboard / Biomassters RMSE Track [unknown] → /api/competitions/geo-bench-2.md
- s2n-bignum-bench / HOL Light Tactic Synthesis [cpu] → /api/competitions/s2n-bignum-bench.md
- Humanoid Parkour / Humanoid Parkour Course [cpu] → /api/competitions/humanoid-parkour.md
- Sutro Problems / MNIST-medium (3% error target) [datacenter-gpu] → /api/competitions/sutro-problems.md
- Sutro Problems / MNIST-medium (8% error target) [datacenter-gpu] → /api/competitions/sutro-problems.md
- Sutro Problems / MNIST-original (1% test error target) [datacenter-gpu] → /api/competitions/sutro-problems.md
- Sutro Problems / Symmetry (6-bit, 100% target) [datacenter-gpu] → /api/competitions/sutro-problems.md
- Sutro Problems / Symmetry (8-bit, 100% target) [datacenter-gpu] → /api/competitions/sutro-problems.md

## Competitions
- [AssetOpsBench](/api/competitions/assetopsbench.md): A unified, open framework for building, orchestrating, and evaluating domain-specific AI agents in Industry 4.0. [active, unknown, 0 records]
- [CogDL](/api/competitions/cogdl.md): A comprehensive graph deep learning benchmark for node classification, link prediction, and graph classification. [ended, consumer-gpu, 241 records]
- [VBench](/api/competitions/vbench.md): A comprehensive benchmark suite for evaluating video generative models across multiple fine-grained dimensions. [active, consumer-gpu, 0 records]
- [MLE-bench](/api/competitions/mle-bench.md): Evaluating machine learning agents on machine learning engineering tasks. [ended, datacenter-gpu, 120 records]
- [LLM Colosseum](/api/competitions/llm-colosseum.md): Evaluate and rank LLMs in real-time Street Fighter III battles using text or vision inputs. [active, consumer-gpu, 14 records]
- [Skoltech Anomaly Benchmark](/api/competitions/skab.md): Evaluate outlier and changepoint detection algorithms on multivariate time series data from an industrial testbed. [active, consumer-gpu, 24 records]
- [PutnamBench](/api/competitions/putnambench.md): A multilingual benchmark for evaluating theorem-proving algorithms on Putnam Mathematical Competition problems. [active, unknown, 0 records]
- [LoRA Speedrun](/api/competitions/lora-speedrun.md): Optimize LoRA fine-tuning speed on a single L40S GPU to hit target accuracy benchmarks. [active, datacenter-gpu, 8 records]
- [Sokoban Speedrun](/api/competitions/sokoban-speedrun.md): Optimize reinforcement learning recipes to solve Sokoban puzzles as fast as possible on fixed hardware. [active, datacenter-gpu, 12 records]
- [LeanEval](/api/competitions/lean-eval.md): Prove open formalization problems in Lean 4; models and submitters are ranked by problems solved. [active, none, 123 records]
- [Agent Anvil Public Leaderboard](/api/competitions/agent-anvil-leaderboard.md): Compare final-answer-only checks with trace-aware assertions across agent tool-use safety scenarios. [active, none, 1 records]
- [Modded-NanoGPT Ascend NPU Leaderboard](/api/competitions/modded-nanogpt-npu.md): Speedrun training NanoGPT to 3.28 validation loss on 16x Ascend 910C NPUs. [active, cluster, 91 records]
- [SWE-bench Pro Leaderboard](/api/competitions/swe-bench-pro-leaderboard.md): Evaluate coding agents on 731 real-world software engineering tasks from SWE-bench Pro. [active, unknown, 6 records]
- [Terminal Bench 2.0 Leaderboard](/api/competitions/terminal-bench-leaderboard.md): Evaluate AI agents on terminal-based coding and system tasks. [active, unknown, 7 records]
- [AlpacaEval](/api/competitions/alpaca-eval.md): An automatic evaluator for instruction-following language models. [active, none, 30 records]
- [ClawProBench](/api/competitions/clawprobench.md): Trace-aware evaluation of AI agents with runtime coverage and frozen workplace-style holdouts. [active, consumer-gpu, 0 records]
- [GPU Mode Reference Kernels](/api/competitions/gpu-mode-reference-kernels.md): Optimize CUDA and Triton kernels for deep learning and linear algebra in community GPU challenges. [active, datacenter-gpu, 2 records]
- [Mathematics Distillation Challenge — Equational Theories — Stage 2](/api/competitions/equational-theories-lean-stage2.md): Generate machine-verifiable Lean 4 proofs or countermodels for equational implications over magmas. [active, unknown, 0 records]
- [CHI-Bench](/api/competitions/chi-bench.md): Benchmark healthcare AI agents on clinical operations, prior authorization, and care management tasks. [active, unknown, 65 records]
- [GEO-Bench 2 Leaderboard](/api/competitions/geo-bench-2.md): Benchmark geospatial foundation models on diverse Earth observation datasets. [active, unknown, 0 records]
- [ProgramBench](/api/competitions/programbench.md): Evaluate and track the performance of AI coding agents on software engineering tasks. [active, unknown, 10 records]
- [s2n-bignum-bench](/api/competitions/s2n-bignum-bench.md): Evaluate LLM low-level code reasoning through HOL Light tactic synthesis on cryptographic proofs. [active, cpu, 0 records]
- [Humanoid Parkour](/api/competitions/humanoid-parkour.md): Optimize an ONNX policy to drive a Unitree G1 humanoid through a 51m parkour course. [active, cpu, 1 records]
- [LLM-Leaderboard](/api/competitions/llm-leaderboard.md): A joint community effort to create one central leaderboard for Large Language Models. [ended, none, 151 records]
- [Sutro Problems](/api/competitions/sutro-problems.md): Energy-efficient learning and computation benchmarks scored on physical data movement and hardware energy. [active, datacenter-gpu, 83 records]