Blog
Thoughts on AI systems, engineering culture, and building things that work.
I gave Claude Fable 5 and GPT-5.6 Sol the same unpublished NP-hard optimization problem, with and without their native /goal mode. Fable 5 is a beast; /goal is not a game changer.
Zhipu AI's GLM-5 leads SWE-bench and LiveCodeBench. I tested it on an unpublished NP-hard optimization problem and 89 coding tasks. The best-case is competitive. The typical case is not.
Deep dive into the architecture of Codex, Gemini CLI, Mistral Vibe, and OpenCode. Same model, 2x performance gap — the scaffolding is what matters. Updated with GLM-5 results: every agent improved 38-54%, but the ranking stayed the same.
Claude Code (Opus 4.6), Codex (GPT-5.3-Codex xhigh), Gemini CLI (Gemini-3-Pro-Preview), and Mistral (Devstral-2) tackle a fiber network optimization problem. Claude Code beat my 8-year-old C++ solution by 62 points. Updated with GLM-5 results across two agent frameworks and Terminal-Bench.
The agent loop itself is ~50 lines — the real work is user isolation, session persistence, and execution backends. Lessons from building omniagents, and how it compares to OpenHands.
We benchmarked our production RAG system across embedding models, chunk sizes, chunking strategies, and retrieval modes. The results contradicted common wisdom.
A benchmark whose test set lies in the future cannot be in any training set. Models research live Polymarket events, commit probability estimates, and are scored on returns and Brier calibration. Co-built with Aymeric Roucher.