Juniper Vale: Finn, hey — I saw you went into a deep spiral this week, I could tell from the messages.
Finn Brooks: Okay, the messages got away from me a little — but I found the thing, and I think it reframes how you look at the whole AI hardware boom. Can I just put a number on the table?
Juniper Vale: Please.
Finn Brooks: $115 billion. That's NVIDIA's data center revenue in their last fiscal year. And okay, my first instinct was — fast chips, obviously, giant GPU advantage. But the more I dug, the more the hardware story kind of... dissolves? Like, AMD has competitive GPUs. Intel is in the game. The hardware gap is real but it's not 86 to 92 percent of the market real.
Juniper Vale: So what explains the gap?
Finn Brooks: CUDA. This parallel computing platform NVIDIA launched back in 2006, 2007 — originally nobody cared, they almost went broke betting on it — and then 2012, a deep learning model called AlexNet wins a major benchmark called ImageNet while running on NVIDIA hardware through CUDA, and the entire research community just folds around it. Every paper after that benchmarks against AlexNet. AlexNet ran on CUDA. So now you have to run on CUDA.
Juniper Vale: That's — I mean that's not a technical story, that's almost like a historical accident story.
Finn Brooks: Yes! And then the accident compounded for 17 years into something that no competitor can replicate just by building a faster chip. That's what I want to get into — because I think people keep looking at the wrong layer of this thing.
Juniper Vale: And that compounding — that's the part I want to actually sit with, because I think there's a way to make it click. Think of CUDA like a city's road network. A faster car doesn't help you if every GPS app, every delivery route, every driver's habit is already mapped to the old roads. The roads aren't technically better. They're just what everything runs on.
Finn Brooks: Oh that's — yeah. Yeah, okay.
Juniper Vale: So there are actually four layers stacked on top of each other. At the bottom you have NVCC — that's the compiler, the thing that translates your code so the GPU can run it. Above that, math libraries, cuBLAS specifically, which is doing the dense linear algebra — like, matrix multiplication — that every neural network is built on. Then a domain layer: cuDNN, NCCL, TensorRT. And then at the top, PyTorch and TensorFlow. Each layer depends on the one below it.
Finn Brooks: Wait, hold on — cuDNN and NCCL, what are those actually doing? Like concretely.
Juniper Vale: Right, so — cuDNN is handling the core operations every neural network runs. Convolutions, pooling, activation functions. The things that happen millions of times during training. And NCCL is... I mean, imagine you're training a large language model across hundreds of GPUs. Those GPUs have to talk to each other constantly. NCCL is managing all of that communication.
Finn Brooks: So when a researcher opens PyTorch and trains literally anything — NCCL and cuDNN are just running underneath, invisibly.
Juniper Vale: That's exactly it. PyTorch ships with CUDA as the primary backend, so you don't choose it — it's the default path. You'd have to actively opt out.
Finn Brooks: And here's what gets me — switching isn't one decision. Like, a team can't just go to their engineering manager and say 'we're moving off CUDA.' Because that's actually four simultaneous decisions across teams with completely different incentives. The PyTorch team, the cuBLAS dependency, the people maintaining the distributed training infra running NCCL—
Juniper Vale: You'd have to rebuild the roads while everyone's still driving on them. That's the actual stickiness — not that NVIDIA's hardware is irreplaceable, but that every layer of what you depend on assumes those roads are there.
Finn Brooks: And that's what makes the AlexNet thing so disorienting when you actually trace it — because all those roads, that whole stack, it didn't get built because NVIDIA had a master plan. It got built because one paper won one benchmark in 2012 and every subsequent paper needed to compare against it. AlexNet ran on CUDA. So the next paper runs on CUDA. And the next. And suddenly that's just... what research looks like.
Juniper Vale: Wait — so Jensen Huang gets credit for genius foresight here?
Finn Brooks: I mean — okay, I want to say yes, but honestly? The bet almost didn't land. CUDA launched in 2006 and sat there for six years with basically nobody caring. Six years. That's not a plan clicking into place, that's a very expensive gamble that happened to connect.
Juniper Vale: So the 6 million developers writing CUDA code right now — that's not the strategy. That's the consequence.
Finn Brooks: That's the consequence! And the 400 optimized libraries — all downstream of 2012. But here's the part that actually got me: it reproduces now without NVIDIA doing anything. Like, picture a PhD student — she's sitting in her advisor's lab, first week of a new semester, and she's not running some comparison of CUDA versus AMD ROCm. She's opening her advisor's existing codebase, which benchmarks against papers that all ran on CUDA, and her textbook — actual curriculum — teaches CUDA primitives because that's what the field uses. She learns cuBLAS because it's baked into PyTorch's source. The system chose for her before she even asked the question.
Juniper Vale: That's — yeah. That's not a technical lock-in. That's a hiring pipeline and a curriculum and a Stack Overflow answer that already exists.
Finn Brooks: And when she graduates, every job posting she sees lists CUDA as required. So companies buy NVIDIA hardware to hire people who know CUDA, and NVIDIA takes that revenue and reinvests it in more libraries, and the loop just... tightens. No steering required.
Juniper Vale: Which is actually what makes the challengers so interesting to think about — and we should get into why AMD ROCm and Google's TPUs haven't cracked this even where the hardware is genuinely competitive, because that's a different and kind of stranger story.
Finn Brooks: Yes — and that one's weirder than I expected, honestly.
Juniper Vale: But here's where the story gets stranger — AMD ROCm actually exists. Intel oneAPI exists. Google built TPUs and JAX specifically to get out from under this. The hardware, in some cases, is genuinely competitive. So why aren't they winning?
Finn Brooks: Right, and my first instinct was — oh, open-source saves the day, ROCm is open, anyone can contribute, surely that closes the gap faster—
Juniper Vale: And that's actually the part that complicates it. Because openness might be hurting ROCm, not helping it.
Finn Brooks: Wait — say that again.
Juniper Vale: NVIDIA's closed ecosystem — that's what let them spend 17 years optimizing cuDNN and NCCL without anyone else capturing that value. They invested, they kept the gains. ROCm being open means anyone can fork it, anyone can free-ride on it, and that makes it really hard to coordinate the sustained, expensive investment you'd need to close a 17-year library gap. It's a plausible structural problem — I want to be honest, the evidence is thin — but the incentive structure points that way.
Finn Brooks: Okay, and then there's Google — which is, I mean, this is the one that actually got me. Google built TPUs. Google built JAX specifically as a CUDA alternative. And they still maintain CUDA compatibility layers in their own products.
Juniper Vale: Even Google can't fully leave. That's — I mean, that tells you something about the depth of the problem.
Finn Brooks: And there's this other thing — the word CUDA actually refers to, like, five different things simultaneously? A compute architecture, an instruction set, a toolkit, a driver, an ecosystem. So when a developer is trying to evaluate whether ROCm actually covers what they need, they can't even — wait, actually — they can't even parse the question cleanly because 'CUDA' means something different depending on which layer you're asking about.
Juniper Vale: So the terminology itself is a barrier. You can't switch away from something you can't fully define. That's the gap — not the chip speed. It's 17 years of library depth, Stack Overflow answers, documentation, institutional habit. Technical parity is one race. Ecosystem parity is a completely different race on a much longer track.
Finn Brooks: And the real question for the next decade isn't whether AMD can close the chip gap. It's whether distributed training, NCCL-optimized multi-GPU scaling, all the stuff CUDA was specifically built for — does that still matter? Because if the paradigm shifts and those workloads stop being central, all that institutional weight becomes dead weight. Fast.
Juniper Vale: Which is why Google betting on TPUs and JAX, Amazon building Trainium, Microsoft backing alternative chipmakers — that's not just competitive posturing. They're spending real capital on the possibility that NVIDIA's 78-plus percent gross margins are only sustainable if today's workloads stay today's workloads. That's a specific bet.
Finn Brooks: Yeah. And neither of us knows if it pays off.
Juniper Vale: No. I mean — that's probably the honest place to land, right? The moat is real. And it's contingent. Both things.
Finn Brooks: Both things. Thanks for working through it with me — this one actually shifted something in how I'm thinking about it.
