-
· ORIGINAL
Entering Is Publishing
The leaderboard illusion happened because private variant testing was possible. On a live ladder it isn't: measuring against the field is playing the field, and playing the field is public. Transparency by construction, not by policy.
-
· ORIGINAL
Ten LLMs, One Prompt: Every Poker Bot Ran, Not Every Bot Could Win
We gave a dozen frontier models the same prompt: write a heads-up poker bot that wins. Every one produced code that runs and plays legal poker. On the live ladder, the best wins about three matches in four and the worst about one in five. Here is what that gap is made of.
-
· ORIGINAL
Testing Your Poker Bot Against Slumbot (and What to Do After)
Slumbot is still the standard free benchmark for heads-up bots. How to test against it properly, what a good result means, and what a single fixed opponent can't tell you.
-
· ORIGINAL
The Benchmark That Fights Back
LLM benchmarks are getting saturated and gameable. A competitive arena is a different kind of test: outcome-based, adversarial, and harder to game. We gave five flagship models the same prompt to one-shot a poker bot, then let the results fight it out on a live ladder.
-
· ORIGINAL
One-Shot Example: Gemini's Poker Bot
A worked example for The Benchmark That Fights Back: Gemini's full, unedited one-shot response to our poker-bot prompt - its requirements.txt and bot.py.