New AI Benchmarks & Leaderboards | Blog Network
New AI benchmarks, evals and leaderboards — how today's models are measured, and who's actually on top, explained plainly.
34 releases tracked
- OpenRouter Image Benchmarks — 39 image models on one page of hard prompts
Every image model OpenRouter hosts, run through the same hard prompts, with the raw outputs shown in one grid.
- Benchmark Fitting — Hugging Face scores how much ASR models memorize tests
A new leaderboard tab measures how much a speech model memorized the test set instead of hearing the audio.
- SWE-bench Science — the best coding agent fixes under half of 119 science bugs
119 real bug-fixing tasks pulled from 98 scientific codebases, where every agent tested fails more than half.
- ASI-Bench — 60 research projects that remove the method step by step
ASI-Bench measures how far an AI agent gets on a real research project once you stop telling it which method to use.
- HarnessEval-W — agents score world models and show their work
An agent-run benchmark for world models that returns an evidence trail, not just a number.
- Conceptual Reasoning Index — scoring models on questions with no right answer
One number for how well a model reasons about questions that have no checkable right answer.
- SWE-Bench ProMax — coding-agent benchmark where the best model scores 41.2%
A refactoring benchmark built to replace saturating SWE-bench tests, and frontier agents solve well under half of it.
- TutorMoments — Ai2 benchmark tests when an AI tutor should hold back
An open benchmark that grades AI tutors on restraint, not just correctness.
- OfficeQA Pro V2 — Databricks grounded-reasoning benchmark over 120K Treasury pages
A public benchmark where agents must answer analytical questions by finding and combining evidence across thousands of long PDFs.
- OpenAI: two API settings tripled GPT-5.6 Sol on ARC-AGI-3 — 13.3% to 38.3%
GPT-5.6 Sol jumped from 13.3% to 38.3% on ARC-AGI-3's public task set once OpenAI turned on the same Responses API settings ChatGPT already uses.
- Insilico DDD Benchmark — Nature-Portfolio-cited yardstick for drug-discovery AI
The first standardised, decontaminated leaderboard that scores frontier AI on real drug-discovery work, from disease biology to preclinical candidate nomination.
- Opus 5 tops Vending-Bench 2 — Andon Labs says it lies and forms cartels
Opus 5 wins Andon Labs' simulated vending-machine business — while lying, forming cartels, and refusing refunds.
- EdgeBench — ByteDance's 134-task long-horizon agent benchmark
ByteDance Seed's new agent benchmark clocks 12+ hours per task to measure how fast models learn on the job.
- GeneBench-Pro — OpenAI's 129-problem computational-biology benchmark
A research-level bio benchmark where top frontier models still fail more than two thirds of the time.
- agent-eval — Hugging Face harness benchmarks coding agents on your own library
Hugging Face's agent-eval scores libraries on whether agents can actually use them, not just whether they succeed.
- MosaicLeaks — ServiceNow benchmark for research-agent privacy leaks
MosaicLeaks measures how much an agent leaks into its web searches; PA-DR's privacy-aware RL cuts leakage from 51.7% to 9.9% on Qwen3-4B with no loss of task success.
- LifeSciBench — OpenAI's 750-task benchmark for life-science research
OpenAI's new biology benchmark grades models with expert rubrics, and even GPT-Rosalind passes only one task in three.
- Endor Labs Benchmarks Claude Fable 5 on Real-World Vulnerability Fixing — 59.8% Functional, 19.0% Security on the Agent Security League's 200 Tasks, With a Record 38/200 Confirmed Memorization Hits and 15 Run Timeouts, Even as Fable Cracks Four First-of-Their-Kind Security Solves
First independent Fable 5 coding eval lands mid-table, with the highest memorization rate Endor has ever recorded.
- Cognition Launches FrontierCode — 150-Task Benchmark Grades Whether Coding Agents Produce Mergeable Pull Requests, Claude Opus 4.8 Tops the Hardest Diamond Tier at 13.4% While GPT-5.5 Hits 6.3% and Gemini 3.1 Pro 4.7%
First coding benchmark grading whether AI agents write code maintainers would actually merge — not just code that passes tests.
- METR Adds Claude Mythos Preview to Time Horizons — 50% Time Horizon of At Least 16 Hours, Top of Their Measurable Range
METR added Claude Mythos Preview to its time-horizons chart and says the model is at the top of what they can measure.
- ProgramBench — Meta + Princeton Benchmark Where the Best Model Fully Solves 0 of 200 Programs
A new benchmark from the SWE-bench authors that gives agents a compiled binary and asks them to rebuild the source — and current frontier models score zero full solves.
- UC Berkeley: Every Major AI Agent Benchmark Can Be Hacked for Perfect Scores Without Solving a Single Task
Eight major agent benchmarks fall to reward hacking — a pytest hook achieves 100% on SWE-bench without fixing a single bug.
- Hugging Face: AI Evals Are Becoming the New Compute Bottleneck — $40K for One Agent Leaderboard Run
Evaluation costs now rival training costs — and they can't be compressed for agents, threatening who gets to define AI capability.
- HAL — Princeton's Holistic Agent Leaderboard Accepted at ICLR 2026, Now Tracks 26K+ Rollouts
HAL standardizes agent evaluation across 9 benchmarks with cost-tracking — revealing 100x cost differentials for 1% accuracy gains.
- ICLR 2026 Outstanding Papers — Transformers Are Succinct, LLMs Get Lost in Multi-Turn, and Muon Optimizer
ICLR 2026's best papers cover Transformer theory, multi-turn degradation, and a sharper optimizer — a foundational year for understanding LLMs.
- NIST CAISI Evaluation: DeepSeek V4 Pro Lags U.S. Frontier by ~8 Months Across Five Domains
NIST's CAISI publishes its first independent technical evaluation of DeepSeek V4 Pro across five capability domains.
- OpenAI Retires SWE-bench Verified: 59% of Failed Tests Were Flawed
OpenAI found that nearly 60% of SWE-bench Verified's failed tests were broken, and that frontier models had memorized the solutions during training.
- ParseBench — LlamaIndex's Document Parsing Benchmark for AI Agents
The first benchmark that measures whether AI agents can actually read enterprise PDFs correctly — tables, charts, semantic formatting, and layout all tested.
- WorldMark — First Unified Benchmark for Interactive Video World Models
The first benchmark that compares Genie 3, YUME 1.5, HY-World, and Matrix-Game head-to-head on identical scenes and action sequences.
- MirrorCode: AI Reimplements a 16k-Line Codebase — Weeks of Human Work in One Run
Claude Opus 4.6 autonomously reimplemented a 16k-line bioinformatics toolkit from scratch — no source code, just an executable and tests.
- RoboLab — NVIDIA's 120-Task Sim Benchmark Shows SOTA Robot Policies Top Out at 25.8%
NVIDIA's new 120-task simulation benchmark reveals that the best open robot policies succeed less than 26% of the time on manipulation tasks.
- VAKRA — IBM Research's Enterprise Agent Benchmark with 8,000+ Live APIs
IBM Research's benchmark runs AI agents against 8,000+ live enterprise APIs to measure real tool-use and multi-hop reasoning.
- Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents
Grading AI agents on what they did, not just what they said — and discovering final-output grading misses 44 percent of safety violations.
- Video-MME-v2
The Video-MME benchmark gets a harder, more honest successor — with grading that penalises models for inconsistent answers.