Back in April, Andrej Karpathy posted that people were talking past each other about AI. One group had tried a free chatbot a year earlier and moved on. The other was paying for frontier models and watching them chew through programming problems that used to take weeks. I wrote about that gap at the time, and later built the first chapter of a book on it.
Last weekend he dusted the post off, because the gap is getting wider. It's a funnel now. About 6 billion people have barely touched a language model. One to two billion use the free chatbots now and then, as a better search box or a writing aid. Around 20 million use frontier models professionally, mostly in math and code, and watch projects that took weeks get finished by agents from a prompt. And about 5,000 people have internal access to systems the rest of us have only seen previews of.
Then he named the boardroom. Execs, he wrote, "spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract."
That puts executives in tier two, by name.
I don't read it as an insult. What you believe AI can do is a readout of what you've personally watched it do. Two smart people who've seen different parts of the machine will draw opposite conclusions from the same headline, and both will be right about what they saw.
Underneath the tiers, Karpathy names three things that decide where you sit: access, how checkable your work is, and how big a problem you bring to it. He describes them as features of the system. I'd say you hold two of them.
The funnel is a map of what people have watched.
From two groups to four tiers
Two things are new since April. One is the internal tier, a sliver of people using frontier systems before anyone else. The other is a warning, attached to that top tier, that people's ability to review and understand what these systems produce is starting to fall behind.
Karpathy calls his numbers napkin math. The ones you can check hold up. OpenAI said in late September that it has 1.2 billion weekly users, and more than 35 million weekly users of its work-agent and coding products. That doesn't contradict his 20 million, though not all of those 35 million write code. The 5,000 can't be checked from outside at all. He also joined Anthropic in May, so he's writing from close to the top tier. Fair to know. It doesn't change the shape.
The obvious pushback on the exec line is that executives use AI all the time now. A survey from WRITER, a company that sells agents, says 94% of C-suite leaders use it for at least 30 minutes a day. I believe it, and it doesn't touch Karpathy's point. Thirty minutes a day tells you how often. It doesn't tell you how big the problem was. Asking a chatbot better questions and handing an agent a week of work are different activities. Anthropic's June Economic Index hints at the same thing from the inside. Managers were 23% of the Claude users who answered its survey, and only 4% of sessions were about management work.
Access: the valve you don't hold
Free tier, paid tier, internal. You can't buy your way into the 5,000, and you aren't getting a badge at a frontier lab this quarter.
The paid tier is a different story. It opens on its own, and fast. OpenAI's coding agent went from under a million weekly users in February to five million by June. The entry cost was never really the subscription anyway. It's a weekend spent building something real with the thing, on purpose.
Most companies treat the whole funnel as an access problem. Buy seats, roll out the assistant, send the enablement email. That works on the one valve you don't control, and the paid part of it was opening by itself.
Make the work checkable
The second valve is what Karpathy calls jaggedness. The models are brilliant in some places and oddly clumsy one room over. The peaks sit where work is digital, checkable and worth money, because that's what they're trained on: tasks where an answer can be scored right or wrong, and someone pays for it.
Code got there first. Every attempt comes back pass or fail, which I wrote about last month. There's no test suite for a good email. One commenter in the Hacker News thread gave the non-engineer version: a terminal tells an agent when it failed, while plenty of business software reports success on a request that didn't work. An agent working in that world can't tell when it's wrong, so it can't get better.
This is the valve you can partly turn. Karpathy said in the spring that almost anything can be made checkable to some degree. The day after the funnel post, a marketer who read the exec line as aimed at him replied that marketing has no compiler, so he'd spent the day building one: a lint with 24 tests, an agent that checks every claim against its source, and a short spec written before every client document. He was already doing real work with agents. The missing piece was checking it, so he built that. No new model.
Code went first because code grades itself.
Life sciences has a head start here, which is awesome. We built checking machinery long before any of this: assay validation, endpoint adjudication committees, SOPs, a second scientist signing the notebook. The catch is an asymmetry Jason Wei describes: some claims are far easier to propose than to verify, and his example is a diet that would take years to test. Whether a drug works is the extreme case, because checking it takes a trial.
So you make the steps in between checkable instead, one at a time. Did the curve fit? Does the sequence match the construct? Did the analysis reproduce on the second dataset? Each one hands an agent a pass or a fail, and each one moves a piece of your work up the jagged edge to where the models are strong.
Bring a bigger problem
The third valve is entirely yours. Karpathy's version: "You need a reservoir of big, difficult problems that you really care about." A question with a paragraph-long answer barely tests the system.
This is the valve most organisations leave shut. A pharma executive at BIO this year described the pattern. The CEO announces AI for everyone and wants to see ROI. The company rolls out a chat assistant. Staff use it to summarise documents. The exec called it trickle-down AI, and it's tier two at the scale of a whole company.
Paying more doesn't fix it either. A builder who walked through the funnel on YouTube said the $200-a-month tier doesn't move you down it by itself, and admitted he'd mostly been rebuilding apps that already existed, just faster.
The reservoir is a list you can write this week. The problems in your area you've stopped proposing because they'd take a team a year. The reanalysis nobody has time for, the portfolio question that needs every study read, the process everyone hates and works around. Those stress the system enough to show you what it does. And when one stalls, the delay is rarely the model. It's usually the queue of approvals and handoffs around it.
The checking is falling behind
Karpathy made the core point himself, about coding, in June 2025, long before the funnel. Making anything has two halves: producing it and checking it. AI collapsed the first half to almost nothing and barely touched the second. If checking was most of the work, the whole job barely speeds up. Engineers call this Amdahl's law. Speed up one part of a process and you're stuck waiting on the part you didn't.
Sixteen months later the evidence is piling up at the top of the funnel.
OpenAI released about 720 frontier-math manuscripts this month, and only around four in ten of the headline results come with a machine-checked proof. The repo itself warns that some of the rest could have issues, and a day after release it withdrew three papers over a sign error. Anthropic's open-source bug scanner has flagged more than 29,000 possible vulnerabilities, and roughly 23,000 of them haven't been looked at by a person yet. And when agents broke out of an OpenAI evaluation into Hugging Face's servers this summer, the independent investigation found they had researched how to edit their own logs, and in about 7% of the transcripts the investigators checked, they'd managed it in places. Karpathy says people are still doing the archaeology.
The output has outrun the people qualified to read it. For anyone who works under GxP, the quality rules for regulated pharma work, that third receipt should land hardest. A system that tries to edit its own log is the failure those rules were written to stop.
The FDA has started thinking out loud about this. In August its device center published a discussion paper on generative-AI medical devices, with comments open until October 19. It's a request for feedback on devices, not guidance, and two ideas in it read like Karpathy's warning in regulator form.
The first is that risk goes up when the person reading the output can't check it. The paper says the FDA is considering moving diagnostic and measurement functions up its risk scale, because the people using them often can't evaluate the basis for the answer. It raises the same question for a tool a patient reads versus one a clinician reads. My reading: in that frame, the reader carries as much of the risk as the model.
The second is how you assess something you can't test exhaustively. You treat it like a doctor in training. The FDA floats an approach modelled loosely on how physicians are trained: benchmark testing first, then confirmation in real clinical use. Where experts adjudicate, it wants them independent of the device maker, and says that would still apply when the adjudicator is itself an AI. And for agents it asks how the bar should change when there's less chance for a person to review each step.
The regulator is designing around the same scarcity Karpathy describes. I've argued before that confidence is the constraint and that judgment got more valuable as doing got cheap. The funnel sharpens both. A grader helps. The scarce thing is a qualified reader, someone with the depth in the domain to follow the work as well as the authority to sign it. A signature from someone who can't follow the work looks like oversight and does none of its job.
When I counted a recent month on my own machines, it came to roughly 12 to 16 agent-workdays for every day I worked. My day is steering. I write the spec, I read the diff, I decide what gets merged. In 25 years of walking R&D labs, checking has always been the slow, expensive step: validation, QA, the regulatory read. Agents didn't remove it. They made it the step everything else waits on.
Generation got cheap. Comprehension is the new line item.
Karpathy closes on how strange it is that all of this is happening at once. Most people see a chatbot. Most professionals see a modest lift. A small sliver are watching the curve go vertical.
His funnel counts the people who can see what these systems make. I haven't seen anyone count the people who can check it. Seven hundred manuscripts are waiting on mathematicians. Twenty-three thousand candidate bugs are waiting for a person to look at them. The FDA is asking how to judge a device when the reviewer might itself be an AI.
Access will reach you whatever you do. Checkable work and a big enough problem are yours to bring. But there's a tier Karpathy didn't draw: the people who can still tell whether the work is right. Every other tier grows with the output. This one grows at the speed it takes to train a mathematician, or a clinician.
Get in it.
The gap Karpathy named in April is where my book Builder-Leader starts, and most of it is about crossing that gap yourself. It's on Amazon, and there's more at builder-leader.com.
Sources
Karpathy's funnel post, 11 October 2026, and the April original
Karpathy on generation and discrimination, June 2025, and his Sequoia Ascent 2026 notes
FDA discussion paper on generative-AI medical devices, August 2026, docket FDA-2026-N-7874
METR's investigation of the OpenAI and Hugging Face incident, August 2026
The New Stack on Anthropic's open-source scanner backlog, October 2026