Walt Garner: Nina, you look like someone who has been arguing with a spreadsheet all week.
Nina Park: Two facts — I keep putting them next to each other and they refuse to coexist. You know that feeling?
Walt Garner: Mm, quite frequently. Name them.
Nina Park: Fact one: at Dell Technologies World 2026 in Las Vegas, Jensen Huang declares that memory — not GPU compute — is the single biggest bottleneck in the global AI supply chain. Michael Dell walks right over to a Bloomberg camera and says the same thing. Fact two: six weeks before that, on December 24th, Nvidia paid twenty billion dollars for a license to Groq's LPU architecture and hired Jonathan Ross, Groq's CEO. A company whose entire value proposition is that GPUs are architecturally wrong for inference decode.
Walt Garner: You see the load-bearing contradiction there — if the problem is memory bandwidth, you optimize the GPU's memory subsystem. You do not pay twenty billion dollars for a chip that was specifically designed to route around GPUs entirely.
Nina Park: That's it, yes — those are two different diagnoses of what's broken. And they both came from the same company within six weeks of each other.
Walt Garner: So the question isn't whether Huang is right. The question is whether we're looking at a genuine philosophical pivot — or a very expensive rebranding of a problem he's known about for years.
Nina Park: Did he change his mind, or did he change his story — and honestly, I'm not sure which answer is more unsettling.
Walt Garner: Well, before we get to which one it is — changed mind or changed story — there's a mechanical reason why the story had to change, and I think it's worth just stating plainly. During LLM inference, specifically in the decode phase, the model has to haul its entire weight matrix out of memory for every single token it generates. Every. Single. Token. It doesn't matter how many FLOPS you have sitting there. The cores are idle, waiting for data that the memory bus is still delivering.
Nina Park: Hold on — every token? Not batched somehow?
Walt Garner: Every token. Think of it like a chef with a massive recipe book. The knives are fast — absurdly fast. But for every single step in the recipe, the chef has to flip back to the full book. The bottleneck isn't the knife speed. It's the page-turning. The compute sits there, sharp and ready, while the memory bus is still turning pages.
Nina Park: Okay, that actually — yeah. So the TFLOPS number, which is the number everyone's been citing for years, that's measuring knife speed when the problem was always page-turning.
Walt Garner: Precisely, and the Cerebras numbers make that gap viscerally concrete. Their WSE-3 — wafer-scale architecture, memory physically adjacent to compute — demonstrated roughly two thousand one hundred to two thousand six hundred tokens per second on Llama 70B-class models. H100-class GPUs, in the same inference workload, were doing fifty to one hundred fifty tokens per second. That's not a ten percent gap. That's a ten-to-fifty times gap. And it's not because the H100 lacks compute — it's the memory wall.
Nina Park: Two thousand six hundred versus fifty. That's — I mean, that's not even close to the same ballpark.
Walt Garner: And this is exactly why Groq's Language Processing Unit — the thing Nvidia just paid twenty billion dollars to license — was built around that reality from the start. The LPU architecture didn't try to win on FLOPS. It was purpose-built for a memory-bound workload. Which, now, actually — that raises a question I keep returning to. If Cerebras and Groq have had these throughput numbers available for some time, why did it take until Blackwell's launch for this to become the headline narrative at Nvidia?
Nina Park: Because nobody cared about page-turning speed until the chef's kitchen got expensive enough that idle knives became a boardroom problem.
Walt Garner: Right — but the timeline of when Huang let that into the room publicly, that's the crucial part. Because through Volta, Ampere, Hopper — the A100, the H100 — the marketing was pure TFLOPS. That was the headline number. That was the press release.
Nina Park: Imagine you're a procurement analyst watching Huang's GTC 2024 keynote on a flight — I mean, you're ready for the FLOPS number, that's what you've budgeted around — and the headline metric being cheered isn't petaflops. It's eight terabytes per second of memory bandwidth. Blackwell. You have to open your laptop on the tray table and explain to your board, mid-air, why the procurement model just changed.
Walt Garner: And that's GTC 2024 — genuinely the first time bandwidth lands as co-equal with compute in Nvidia's own framing.
Nina Park: But that's after Blackwell is already built. Which — wait, that's the timing problem I keep snagging on.
Walt Garner: Yes, and it only deepens. GTC 2025, Huang puts a formula on stage — FLOPS explicitly linked to bandwidth, written out, public. Then CES 2026, HBM and DRAM bottlenecks again. The messaging keeps arriving after the hardware ships, not before.
Nina Park: Which is — I mean, I want to be honest that the sourcing here is spread across venues and we're pattern-matching, not citing one clean speech. But the pattern is hard to ignore.
Walt Garner: Now, Intel's Srini Krishna publishing in July 2026 that memory bandwidth — not compute TOPS — is the decisive metric, pointing to DDR5 at eight thousand megatransfers per second on Xeon 6 — that looks like independent corroboration until you notice Intel has every structural incentive to make that argument. It rehabilitates their entire product line.
Nina Park: Convergent insight or convergent self-interest — yeah, I genuinely cannot tell.
Walt Garner: And the Morgan Stanley closed-door meeting in San Francisco, March 2026 — Huang personally attending, Nvidia stock dropping roughly four percent after — that's not a technical briefing. That's damage control on whether the bandwidth thesis is actually delivering returns. The part that makes all of this worse is what the Groq acquisition says quietly underneath all of it, and we should get to that.
Nina Park: What Groq says quietly is that the non-exclusive part of that license — I mean, Nvidia paid twenty billion dollars and Groq can still license the same LPU architecture to anyone else. That's not an acquisition. That's — what is that?
Walt Garner: It's insurance with a competitor still holding the policy. Nvidia bought access, not elimination. Which means a hyperscaler could go directly to Groq tomorrow and license the same architecture Nvidia just paid twenty billion for.
Nina Park: Wait — the same exact technology?
Walt Garner: The non-exclusive structure permits it, yes. And Jonathan Ross is inside Nvidia now, which is — I mean, that is a remarkable piece of the deal to sit with. The man who built the case that GPUs are wrong for inference is now an Nvidia employee.
Nina Park: That's either the smartest talent acquisition in recent memory or the most expensive way to silence a critic. And I genuinely cannot tell which. Because Blackwell — the B200, with the fifth-generation tensor cores and that decompression engine the arXiv study documented — that hardware says Nvidia is trying to fix the memory problem in silicon. But buying Groq says they know the fix might not be enough.
Walt Garner: Now, the benchmark reality makes that tension almost unbearable. Cerebras and Groq's throughput advantages are real — five to ten times on inference workloads — and yet GPU-based serving is what actually runs at scale across the industry. So the question is whether the benchmark gap and the deployment reality are telling different stories entirely.
Nina Park: Picture a VP of infrastructure at a mid-size AI company — Sunday night, signing off on a rack order. Twelve Blackwells. She's seen the Groq throughput numbers. She orders the Blackwells anyway, because the tooling, the support ecosystem, the engineers who already know CUDA — that's not a benchmark. That's a switching cost.
Walt Garner: And that gap — between what the benchmark shows and what the procurement decision reflects — is, I think, the actual unanswered question underneath the acquisition. Because if HBM supply stays constrained, SK Hynix and Samsung and Micron holding the prepayment agreements, then the switching cost calculus eventually breaks.
Nina Park: So Nvidia buys Groq not because Blackwell fails, but because if HBM scarcity gets bad enough, Blackwell's memory advantage evaporates — and they need somewhere else to stand.
Walt Garner: Which is why — and I keep returning to this — the actual tell is Blackwell Ultra. August 2025, two hundred and eight billion transistors, dual-reticle design, and the architectural direction is still bandwidth-and-efficiency. Not raw compute parallelism. If the generation after that follows the same line, that's conviction in silicon. That's not narrative management.
Nina Park: We don't need to resolve whether Huang actually changed his mind. We just watch what they build next. If the post-Blackwell generation prioritizes inference decode over training parallelism — that's the answer. The Groq deal bought time. It didn't buy the answer.
Walt Garner: Indeed. And the sources genuinely don't settle which it is — conviction or reframe. That uncertainty is real, not rhetorical. The silicon will say it more plainly than Huang ever will.
Nina Park: Watch the next chip. Everything else is inference.
Walt Garner: Ha. Yes. Quite literally.
Nina Park: Good dig today. I mean that.
