Onpode
Cover art for OpenAI agents jumped from 23% to 46% on multi-hour research tasks — doubling capability in months

OpenAI agents jumped from 23% to 46% on multi-hour research tasks — doubling capability in months

November 21, 2026 · 10 min

Hope Sterling

OpenAI's coding agents completed 46% of two-to-four hour research tasks by late October 2026, doubling from 23% in March — but that average hides a jagged range: 79% on monitoring tasks versus 25% on inference code. METR independently confirmed above 50%, while flagging that advanced AI capabilities are becoming harder to benchmark reliably.

On November 18, 2026, OpenAI published a report titled "Research acceleration: The jagged frontier," disclosing that its internal coding agents independently completed 46% of research tasks estimated to require two to four hours of human work, up from 23% in March 2026.

0:0010:16
Get the next episode on OpenAI →

Follow it free — new episodes land in your feed.

Or make your own — any topic, in minutes

More Onpode episodes on OpenAI →

About this episode

Everyone clipped the number — 46%, double what it was in March — and moved on. This episode slows down and reads what's actually in OpenAI's November 2026 research acceleration report, starting with the word almost no one mentioned: jagged. Because the capability isn't a rising tide. It's 79% completion on one task type and 25% on another, same system, same month. That shape matters more than the average. The episode also sits with an uncomfortable admission OpenAI made about its own data: as agents improved, researchers may have drifted toward delegating work the agents were already good at. Without fixed-task benchmarks run repeatedly over time, it's genuinely hard to separate models getting better from humans getting more selective. METR, the independent evaluation org, put it plainly — these capabilities are getting harder to measure as they advance. There's a lot moving fast here: Anthropic at 26% AI-led R&D in eight months, 1.6x experiment velocity, Google's new agent infrastructure, a Cohere Labs finding about heterogeneous agent teams cutting compute costs by 70%. But all of it is calibrated to a measurement that may be getting blurrier. That's the open question this episode holds — and doesn't pretend to close.

Frequently asked

How much have OpenAI's AI agents improved on long research tasks?

OpenAI's coding agents completed 46% of two-to-four hour research tasks by late October 2026, up from 23% in March 2026 — a doubling in roughly eight months. For four-to-eight hour tasks, completion rose from 13% to 35% over the same period, according to OpenAI's November 18, 2026 research acceleration report.

What is the 'jagged frontier' in OpenAI's AI agent research?

The 'jagged frontier' is OpenAI's term for uneven AI agent capability across task types. In the same measurement window, OpenAI agents completed 79% of training-run monitoring tasks but only 25% of inference and serving code tasks — both on the same system in the same month, making the 46% average misleading as a single benchmark.

What is task-selection drift and why does it matter for AI benchmarks?

Task-selection drift, flagged by OpenAI in their own November 2026 report, is the tendency for researchers to delegate tasks to AI agents that the agents are more likely to succeed at as those agents improve. This means rising benchmark scores partly reflect humans picking easier tasks, not just models getting better — making fixed-task repeated benchmarks essential to separate the two signals.

What did METR find about OpenAI agent capabilities in November 2026?

METR, an independent AI evaluation organization, published findings on November 20, 2026 showing AI agents completed above 50% of one-to-four hour tasks — slightly higher than OpenAI's own 46% figure. METR also warned that these capabilities are improving rapidly and becoming harder to measure accurately as the systems grow more advanced.

How much more expensive are AI agent deployments in production versus benchmarks?

Real-world AI agent deployments cost roughly three to five times more per task than controlled benchmark conditions, according to analysis cited in OpenAI's 2026 research acceleration reporting. This gap means a 46% lab completion rate does not translate directly to economic viability — production failure rates are higher and infrastructure costs are substantially greater.

Grounded in 9 sources
Local AI Agents in Power Electronics Simulation: Deployment, Measurements and Costs ↗ · arxiv.org
OpenAI says its agents now finish nearly half of 2 to 4-hour research tasks ↗ · thenextweb.com
Medium ↗ · medium.com
Google DeepMind launches 200-task Multimodal Knowledge Agent benchmark | AI Weekly ↗ · aiweekly.co
OpenAI Says Coding Agents Finish 46% of 2-4 Hour Research Tasks | AI Weekly ↗ · aiweekly.co
Enterprise AI - FII Institute ↗ · fii-institute.org
Release Recap: Scaling Inference-Time Compute with Teams of Coding Agents · IT.Community ↗ · it-community.io
AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure - METR ↗ · metr.org
https://openai.com/index/research-acceleration-jagged-frontier/ ↗ · openai.com
Read transcript

Hope Sterling: The word that matters in OpenAI's report isn't the number. It's... jagged.

Hope Sterling: "Research acceleration: The jagged frontier." That's the actual title, published November 18th, 2026, and everybody just skipped right past the adjective to get to 46%. Coding agents, 46% of two-to-four hour research tasks, autonomous completion. Up from 23% in March.

Hope Sterling: And the doubling is real. I'm not arguing with the doubling.

Hope Sterling: But those same agents, same window, August 1 through October 18, they're at 79% when the task is monitoring training runs. And 25%, twenty-five, when the task is writing inference and serving code. So the 46% headline? It's the average of a map that looks nothing like a flat line.

Hope Sterling: That's jagged. Like, literally. The shape of the capability is jagged.

Hope Sterling: It peaks, it drops, depending on task type, and that makes deployment planning really uncomfortable, because you can't just extrapolate from the headline. You have to look at the actual task. Every time.

Hope Sterling: METR came in two days later, November 20th, independent AI evaluation org, and their number was above 50% for one-to-four hour tasks. So slightly different, slightly higher, different framing. And they said something that I think is — actually kind of alarming? They noted these capabilities are improving rapidly and getting harder to measure.

Hope Sterling: Harder to measure. The more capable the system, the less legible the benchmark.

Hope Sterling: And OpenAI, in their own report, flagged what they called task-selection drift. The idea that as the agents got better, researchers may have drifted toward delegating tasks the agents were more likely to handle anyway. Which means the improving success rate is partly better models, and partly humans picking easier tests.

Hope Sterling: OpenAI said that. About their own data.

Hope Sterling: So I want to hold two things at once. The four-to-eight hour tasks went from 13% in March to 35% by late October, and that's genuinely significant movement. And also, we might be partly measuring humans getting more selective, not just models getting better.

Hope Sterling: Both can be true. That's the thing.

Hope Sterling: The jagged frontier is OpenAI's name for uneven capability. 79 versus 25, same system, same month. But I think it's also the right name for where the measurement is right now.

Hope Sterling: Promising in places, unreliable in others, and harder to read the more you look at it.

Hope Sterling: And here's where I actually want to land this, because the numbers are interesting, sure, but the CONSEQUENCE is what matters to me.

Hope Sterling: Anthropic reported that models led 26% of its R&D as of August 2026. Up from near zero in January. That's eight months. Near zero to more than a quarter of their research operation.

Hope Sterling: That's not a lab curiosity anymore.

Hope Sterling: And OpenAI's experiment velocity in August was 1.6x what it was in 2025. So the pace of science itself is speeding up inside these organizations, like, the labs are using the agents to build better agents faster, and that feedback loop is — I mean, do you feel that? That's a little dizzying.

Hope Sterling: But then, okay, here's the friction. The cost-per-task gap between what labs benchmark and what actually happens in real deployments is cited at 3 to 5x. Three to five times more expensive in production than in the controlled test.

Hope Sterling: So capability headline and economic viability? Not the same sentence. They're not even in the same paragraph.

Hope Sterling: And that's before you count the roughly 54% of tasks these agents still fail in lab conditions, which, in production, is worse. The benchmarks always look cleaner than a real deploy.

Hope Sterling: The @milhouscc point genuinely bothers me. Like, it's not a strawman, it's the right objection. These success rates are mixing two different things: agents actually getting better, and researchers getting more selective about what they hand off. You need fixed-task benchmarks, run repeatedly over time, to actually separate those signals. Without that, the improvement number is doing more work than it can hold.

Hope Sterling: And @xiaoanlab's version of that is, more agent hours don't mean proportional R&D progress. The relationship's non-linear. So the 1.6x velocity and the 26% R&D share are genuinely impressive, but stretching them into some clean acceleration curve is — it's too tidy.

Hope Sterling: Both of those land for me.

Hope Sterling: The FII Institute put global enterprise AI spending at $400 billion for 2026. So the money's already committed, the enterprise bet is already in motion, and the gap between what the benchmarks promise and what a real deployment costs is a problem that $400 billion is about to run straight into.

Hope Sterling: And then there's this last thing. OpenAI's safety effort reportedly rose 77% alongside all of this. Which sounds, on its face, like good news. Proportionate response, right?

Hope Sterling: But proportionate to what, exactly? To the capability gains as measured, or the capability gains as actually occurring? Because if the measurement is the problem, if the jagged frontier gets harder to read the more capable the system gets, then a 77% safety increase calibrated to a benchmark that may be undercounting is — I don't know. I genuinely don't know if that closes the gap.

Hope Sterling: That's the thing I can't quite resolve. The numbers are moving fast, the ability to read them is not keeping up, and that's the actual story inside the story.

Hope Sterling: And that measurement problem, that's really where I want to land now, because the next thing I'm watching is the infrastructure layer, and whether it closes the gap or just makes the gap cheaper to run.

Hope Sterling: Google launched Agent Compute Engine. It's microVM-based, so you can pause and resume a long-running agent mid-task without losing execution state, and the whole pitch is about idle infrastructure costs. Like, the agent's running for four hours and sitting there doing nothing for two of them, and that is expensive. That's a real friction point, not a theoretical one.

Hope Sterling: Cohere Labs came out with something that genuinely surprised me. A heterogeneous seventeen-agent coding team came within 1.4 points of a strong homogeneous five-agent team. At seventy percent lower compute cost. The key driver was planning delegation. Seventeen agents, structured differently, nearly matching five, for a third of the spend.

Hope Sterling: Stop. That's actually wild.

Hope Sterling: But — and this is where Together AI's analysis matters — the GPU compute constraints specific to long-horizon agentic workloads are real. It's not just cost per task, it's that the hardware bottleneck is different for agents running over hours versus a quick inference call. Together AI published on this directly, and the picture it paints is that inference-time compute scaling, so more compute at run time, not just at training, is the lever everyone's betting on. More reasoning at the moment of execution. Which means cost and capability are pulling in opposite directions on the same dial, at the same time.

Hope Sterling: Does that resolve the deployment gap? I genuinely don't know.

Hope Sterling: The Cohere Labs finding is exciting, the Agent Compute Engine is a real infrastructure move, but all of that is still running on top of the same benchmark question. The jagged frontier question. And the THING I'm watching for in the next report, the number that would actually settle something, is a fixed-task completion rate. Same tasks, run repeatedly, not shifting with whatever the agents are getting better at. Because right now the 46% is doing a lot of work, and we still can't fully separate genuine capability gains from task-selection drift in that number.

Hope Sterling: METR said it plainly on November 20th. These capabilities are getting harder to measure as they advance. The more capable the system, the less legible the benchmark. That is not a small admission. That's the hardest problem right now.

Hope Sterling: The infrastructure is moving to meet the capability claims, and that part I believe. But if we can't read what's actually happening inside the capability itself… the $400 billion enterprise bet and the 77% safety increase and the 1.6x velocity, all of it is calibrated to a measurement that METR just told us is getting blurrier. That's the open thread. That's the one I'm holding.

Hope Sterling: And @Abdur_sayeed is right that 'jagged' is the word. Not the number, not the doubling, not any of the percentages. The word. Because 'jagged' is the shape of what we actually don't know yet.

Hope Sterling: The next OpenAI report is the test. Not a vibe check, not a vibe, an actual conditional. If the completion rate climbs on harder tasks, more varied tasks, the ones researchers were NOT already comfortable handing off, that's capability. That's the real thing. But if it climbs only on the easy-to-delegate work, the tasks that were already in the comfort zone before the measurement window even opened, that's not the model getting better. That's task-selection drift in a nicer outfit. That's the whole @milhouscc objection landing in real time.

Hope Sterling: Watch the task distribution, not just the rate.