Eleanor Crane: Ben, hey — I have to ask, did you sleep alright? Because I did not, and I'm blaming the thing we're talking about today.
Ben Okonkwo: Ha — what kept you up, specifically?
Eleanor Crane: Well, I kept turning over this one comparison. Two models. GPT-3, one hundred and seventy-five billion parameters — and it cannot do modular arithmetic. Like, near-random. Essentially guessing. Then PaLM, five hundred and forty billion — near-perfect. Same task. Nothing in between those two numbers. No curve, no gradient. Just... a cliff.
Ben Okonkwo: Right — and nobody engineered that in.
Eleanor Crane: That's the part I can't put down. Jason Wei and his colleagues published this in 2022, formally — they called it emergent abilities, built it into a whole research category. And the definition is almost unsettling when you say it plainly: capabilities that are absent at smaller scale and appear suddenly once you cross certain thresholds. Not trained for. Not expected. Just... there.
Ben Okonkwo: And it's not one task — symbolic reasoning, complex arithmetic, question answering. The same discontinuous jump shows up across all of them.
Eleanor Crane: Which raises the question that I think is actually what this whole conversation is about — is this a real phase transition happening inside the model, or is the cliff an illusion built into how we measured?
Ben Okonkwo: And the answer matters more than it might seem, because the entire scaling strategy — the trillion-dollar bet — rests on which one is true.
Eleanor Crane: That's exactly the part that nagged at me at midnight, yeah.
Ben Okonkwo: Honestly? Fair.
Eleanor Crane: And that bet — I want to understand the actual mechanism underneath it before we get to whether the cliff is real. Like, what does it even feel like from the inside of the thing?
Ben Okonkwo: Think of it like a radio coming into range. The signal was always there — you're just driving through the hills, and below a certain point all you hear is static. Then you crest, and suddenly: music. Full music. The radio didn't learn to receive. You just crossed a threshold.
Eleanor Crane: No explicit moment of teaching it.
Ben Okonkwo: None. And that's the one clean sentence I'd want someone to walk away with: scale crosses a threshold, and capabilities appear that nobody wrote a rule for. Now — in-context learning is probably the clearest case. You hand GPT-3 three examples in the prompt. No weight updates, nothing retrained. And it solves a task it has never seen. The model isn't being fine-tuned in that moment — it's reading the examples and... generalizing. Instantly.
Eleanor Crane: Wait — nobody trained it to do that? Like, there's no step in the pipeline that says 'now learn from examples in the prompt'?
Ben Okonkwo: Correct. It emerged from scale. And chain-of-thought is the same story — actually, this one is weirder, if anything. Smaller models, you ask them a multi-step arithmetic problem, they jump straight to an answer. Wrong, usually. Larger models, past a certain threshold, they start breaking it into steps. Intermediate reasoning. Nobody told them to show their work.
Eleanor Crane: But wait — in-context learning, chain-of-thought — these are real skills. A researcher could sit down and try to teach those explicitly. And yet—
Ben Okonkwo: And yet scale just... produced them. The symbolic abstraction piece is actually where I find the evidence most interesting — GPT-2 variants, absent. You look for the mechanisms that let a model identify an abstract rule and apply it, and they're not there. Above certain scale thresholds, they appear. Across model families, not just one architecture.
Eleanor Crane: Across unrelated domains too, though — it's not like this happens on one benchmark.
Ben Okonkwo: Right — and that's what makes Jason Wei et al.'s 2022 work so hard to dismiss. It's not isolated. The same discontinuous jump, symbolic reasoning, complex arithmetic, question answering — shows up across tasks and architectures. That's the pattern that earns the term emergent abilities as a research category, not just a single fluke on one benchmark.
Eleanor Crane: So — and I want to make sure I'm tracking this right — PaLM at five-forty billion does the thing, GPT-3 at a hundred and seventy-five billion essentially guesses. And the space between those two numbers contains no gradual improvement. Is that a cliff, or is that slope we just can't see yet?
Ben Okonkwo: That is exactly — hm, okay, that's the question that the next piece of evidence either answers or complicates. Because it depends entirely on how you're measuring.
Eleanor Crane: And here's what genuinely unsettles me — a software engineer, let's say it's a Saturday afternoon, she's benchmarking two model sizes, runs the same reasoning task, and she watches the accuracy graph go from twelve percent to ninety-one. Cliff. She thinks: phase transition. Then she swaps in a graded scoring rubric, partial credit for partial reasoning, and suddenly there's... a slope. A smooth climb that was there the whole time.
Ben Okonkwo: Right — and she didn't change the model. She changed the ruler.
Eleanor Crane: So what did she actually see the first time?
Ben Okonkwo: She saw the metric's non-linearity. Accuracy is binary — right or wrong. So a model that's getting sixty percent of the reasoning correct but consistently failing the final step scores identically to a model that's guessing randomly. Both score near zero. Then you add one more layer of scale, the model clears that last step, and suddenly it scores one hundred. The jump was always in the threshold of the measurement, not a phase transition inside the network. It's — like asking 'is it light yet?' every hour versus actually tracking luminosity. The sunrise was always gradual. The binary question manufactured the drama.
Eleanor Crane: The mirage framing.
Ben Okonkwo: Exactly that. And if the measurement artifact critique is right, what you're documenting when you see abrupt emergence is the non-linearity of accuracy as an evaluation choice — not genuine internal reorganization in the model.
Eleanor Crane: But does that actually resolve anything? Because — I mean, that engineer on her Saturday afternoon, she still watched it go from twelve to ninety-one. The model still couldn't do the task and then it could. Whether the slope was hidden or whether it was a cliff, the capability is there now and it wasn't before.
Ben Okonkwo: That's the load-bearing question, and honestly — the evidence doesn't resolve it. Not definitively. The measurement artifact critique is real, the graded metrics do flatten the curve, but the phase-transition interpretation isn't dead. The PaLM modular arithmetic data, the symbolic abstraction findings, those don't fully dissolve when you switch metrics. Some of the discontinuity survives. Which means we're in a genuinely unresolved debate, not a settled one.
Eleanor Crane: Wait — so both can be partially true?
Ben Okonkwo: Some jumps are metric artifacts. Some might be real internal reorganization. We don't yet have a clean way to tell them apart from the outside, which — yeah. That's the problem.
Eleanor Crane: And that distinction isn't academic, because what hinges on it is whether future capabilities are predictable — and governance frameworks being built right now are treating this as more settled than it is.
Ben Okonkwo: Which is where the stakes get uncomfortable — and the part that actually makes this worse is what's happening to the definition of scaling itself, which we haven't touched yet.
Eleanor Crane: Well, and that question — whether the cliff is in the model or in the ruler — that's the thing I keep returning to, because the answer changes everything about how much we should trust what we think we know.
Ben Okonkwo: And that's exactly where the scaling definition starts to crack — because OpenAI, Google DeepMind, Anthropic, Meta, all four of them have organized their entire R&D pipelines around scaling laws. Expecting qualitatively new capabilities to emerge. That's not a side bet, that's the architecture of the strategy.
Eleanor Crane: Meaning if the mechanism is wrong, the whole roadmap is wrong.
Ben Okonkwo: Right — but now look at what o3 actually did. On ARC-AGI, GPT-4o scores 5%. o1 gets 13.33%. Then o3 hits 88%. That's the same cliff shape we've been describing. Except — and this is the part that changes things — that jump didn't come from piling on more pre-training parameters. It came from scaled post-training reinforcement learning and search-based inference at inference time.
Eleanor Crane: Hold on. 5% to 88%?
Ben Okonkwo: 5 to 88. And DeepSeek-R1 is the same signal — significant reasoning gains through post-training RL and inference-time compute, not through growing the parameter count. So the thing we've been calling 'scaling' — the thing all four labs built their strategies around — that definition is quietly moving underneath everyone.
Eleanor Crane: So it's not just 'add parameters, watch capabilities appear.' The lever is changing.
Ben Okonkwo: Which means — okay, imagine a policy analyst, let's say she's in Brussels, it's a Tuesday evening, she's drafting a risk framework for frontier models. She's building it on the premise that unpredictable capability jumps come from pre-training scale. That's the foundation. And now the mechanism is post-training compute, inference-time search. The jump still happened. But the thing she was regulating — she was, I mean, she was regulating the wrong variable.
Eleanor Crane: And that matters because — well, the governance case rests on a specific claim. That emergence is unpredictable. That you cannot see the capability jump coming. Is that still true if the jump is coming from how you run the model at inference, not just how big it is?
Ben Okonkwo: That's the uncomfortable part. Policymakers and safety researchers have been treating emergent abilities as the core driver of why frontier AI is hard to regulate — because unpredictable capability jumps make risk assessment fundamentally harder. And that logic still holds even if the source of the jump shifts. The unpredictability doesn't go away, it just moves.
Eleanor Crane: There's an asymmetry here, though. If emergence reflects real phase transitions — something genuinely reorganizing inside the model — then precautionary oversight is the only rational response. We cannot predict the next jump. But if it's largely a measurement artifact, then the urgency tied to 'unknown capabilities appearing unpredictably' might be miscalibrated. Those two regulatory postures are not the same.
Ben Okonkwo: And neither one is comfortable.
Eleanor Crane: No. Because if you're wrong toward caution, you've over-regulated something you didn't understand. And if you're wrong the other direction—
Ben Okonkwo: You missed the jump.
Eleanor Crane: Governance frameworks being designed right now — under conditions of genuinely unresolved foundational uncertainty about the phenomenon they're meant to govern. That's the thing I don't think gets said plainly enough.
Ben Okonkwo: And the o3 and DeepSeek-R1 data makes it more unresolved, not less — because now we don't even have consensus on what dimension of scaling produces the next capability jump. The empirical ground is shifting at the same moment the regulatory ground is being poured.
Eleanor Crane: What actually settles for me is that grokking gives us a story. Memorization first, then abruptly, generalization. And it's a good story. It maps onto what we're seeing. But it was built to explain a different phenomenon and then extended to emergence by analogy, not by confirmation. We're borrowing a narrative.
Ben Okonkwo: Yeah — and the phase transition framing is the same category of thing. Internal circuit reorganization, representations suddenly restructuring at scale. A research direction. Real scientists working on it seriously. But not settled. We do not have a confirmed mechanism for why PaLM clears that modular arithmetic threshold and GPT-3 doesn't.
Eleanor Crane: Which is — I mean, that's the honest state of it, isn't it. Mechanistic ignorance. Not as an indictment of the researchers, just as a plain description of where the field actually is.
Ben Okonkwo: That's the one thing I'd want anyone walking away from this to actually hold. Not 'emergence is real' or 'emergence is a mirage' — we genuinely don't know. The question isn't resolving that debate before we scale further. It's whether we can build research and governance frameworks that function under either assumption.
Eleanor Crane: Both assumptions have to be live at the same time.
Ben Okonkwo: Both assumptions have to be live. That's it.
Eleanor Crane: I think that's actually a harder ask than it sounds. Holding genuine uncertainty without collapsing it into 'we'll figure it out eventually' on one side, or 'therefore stop everything' on the other. Just — sitting in it. Designing for it.
Ben Okonkwo: Honest uncertainty, named clearly, is at least the right starting point. Not comfortable — but right. Thanks for thinking through it with me.
