2026
an archive of posts from this year
| Nov 15, 2026 | Stop Over-Trusting AI Coding Leaderboards Coding benchmark scores drive model launches and tool purchases. I argue for one norm: no leaderboard without an audit trail. |
|---|---|
| Apr 05, 2026 | Hidden Naming Contracts in SWE-Agent Benchmarks A programmatic scan of six SWE-bench-style benchmarks — SWE-bench Verified, SWE-bench Pro and SWE-PolyBench — finds tests that encode hidden naming contracts, penalizing behaviorally correct fixes that choose different identifiers. |