thoughts from inside the work. written for the version of me that hasnt gotten here yet.
i spent a week trying to make the judge cheaper. then someone put a page of their own copy in front of me and the judge had passed every error on it. cheaper wrong is still wrong.
agents are non deterministic. evals are a fixed list decided before the trace arrives. thats the asymmetry, and its why nobody can afford the good evals. im stuck here.
for thirty years everything cheaper than about 31 cents was uneconomic to sell. that one number shaped the business model of the whole web. it just moved by three and a half orders of magnitude.
ive rebuilt my agent architecture five times now. this is what killed each one, and the thing i only understood after the fifth.
every ai company has quietly built a real-time accounting system running at enormous volume, and then bolted it to a card payment that happens once a month. agents need exactly the inverse.
the ai future that worries me isnt the one that ends us. its the one where it all works and nothing changes, because we bolted it onto processes that already existed.
i think the optimistic version is the one we get. thats also the one i find hardest to look at, because everyone having enough is the beginning of a problem rather than the end of one.
every technology that crossed over stopped naming itself after the thing it was built on. but the label is a symptom, not the disease. the disease is six screens before anything good happens.
a public ledger shows everything forever. that one property has kept normal people off them for fifteen years, and the argument about how to fix it has been a false binary the whole time.