Hope Sterling: Juniper, hey — okay I need to tell you what happened to me literally last night because it is so relevant to what we're doing today.
Juniper Vale: Go ahead, I'm ready.
Hope Sterling: My friend is a machine learning engineer at this mid-size startup and she was like, hey, we're switching to AMD GPUs, save thirty percent on cloud costs — and I was like, cool, AMD just dropped the MI355X and it's matching NVIDIA on benchmarks, so that should be fine, right? And she looked at me like I had said something genuinely unhinged.
Juniper Vale: Yeah, because hardware parity and market reality are not the same thing at all.
Hope Sterling: But wait — like, isn't that the whole thing? If AMD's MI355X posts competitive MLPerf Inference 6.0 results in April 2026, which it did, that means the chip is actually good, so why does NVIDIA still have somewhere between 86 and 92 percent of the data center GPU market? That number is — I mean, that's basically a monopoly.
Juniper Vale: It is. And Jensen Huang basically told us why — he doesn't talk about transistor counts, he talks about 6 million CUDA developers and 400 optimized libraries. That's his answer to every competitive threat. Not the chip.
Hope Sterling: So the hardware won and still lost — that's what we're trying to figure out today?
Juniper Vale: Exactly — and here's the part that makes it click: CUDA is basically English as the language of global business. Not because English is linguistically superior. Because every textbook, every contract, every university course, every job posting is already written in it. You could invent a language that's objectively better tomorrow and it wouldn't matter, because you'd have to rebuild all of that.
Hope Sterling: Okay wait — so switching GPUs isn't like switching phones, it's like — telling an entire industry to go re-learn everything in a new language?
Juniper Vale: That's it. And NVIDIA has had since 2007 to write those textbooks. That's 19 years of accumulated libraries before most people even knew AI infrastructure was a thing.
Hope Sterling: Okay but how deep does that actually go? Like — is it just a few libraries?
Juniper Vale: Four layers deep — and this is the part that actually matters. Bottom layer is the CUDA toolkit itself, the basic compiler, the programming model. Then above that you get math utilities — cuBLAS is the one doing matrix multiplication, which is, I mean, that's basically all neural network math underneath. Then the AI-specific layer: cuDNN handles the convolution and pooling operations that every model runs, TensorRT handles inference optimization, NCCL handles communication across multiple GPUs. And then at the top you have the frameworks engineers actually touch — PyTorch, TensorFlow.
Hope Sterling: Wait — so PyTorch is sitting on top of all of that?
Juniper Vale: It's not just sitting on top of it — PyTorch and TensorFlow are calling into cuDNN and cuBLAS at runtime. Like, a researcher writes a line of PyTorch, hits run, and the actual computation is happening inside NVIDIA's libraries. They're not even aware it's happening.
Hope Sterling: So when my friend's startup tried to switch to AMD, they didn't just have a hardware problem — they had a, like, four-floors-of-invisible-plumbing problem?
Juniper Vale: And most engineers only ever see the top floor. They're writing PyTorch. They don't know cuDNN is underneath it running the actual math. So when it breaks on AMD hardware, it's not obvious why — it's somewhere in layers they've never had to think about.
Hope Sterling: But okay — that's the part that actually broke my brain a little. Because like, forget the libraries for a second. Even if AMD rebuilt every single one of those four floors overnight, there's still a — a human problem. Like, I'm picturing my friend's junior colleague, brand new ML engineer, gets handed this AMD inference task on a Friday afternoon. She sits down, opens documentation, and every course she took assumed CUDA. Every paper she reproduced, every Stack Overflow answer she bookmarked — CUDA, CUDA, CUDA.
Juniper Vale: The knowledge she'd have to leave behind is the actual switching cost.
Hope Sterling: Right — but then she tries HIP, HIPIFY, whatever — AMD's translation tool — and it gets her partway and then just... stops. Because the custom kernel code, the stuff written specifically for CUDA's threads-blocks-warps-grids execution model, there's no automatic translation for that.
Juniper Vale: Custom CUDA kernels are the hardest category. You can't HIPIFY your way out of hand-written GPU code — that stuff was optimized for a specific execution model and someone has to manually rewrite it.
Hope Sterling: Wait — so she's not stuck because AMD's chip is worse. She's stuck because the accumulated answers to her questions don't exist on AMD.
Juniper Vale: That's the VHS-Betamax thing, you know — Betamax was technically better, most people agree. VHS won because it had more titles at the rental store. More developers writing CUDA means more tutorials, more Stack Overflow threads, more research code. Which pulls in more developers. It compounds.
Hope Sterling: And like — generationally, right? University courses teach CUDA, so every new engineer enters the workforce already inside the ecosystem, and then they publish papers assuming CUDA, and then the next engineer reproduces that paper and — it's not a moat, it's like a tide that keeps rising.
Juniper Vale: That 6 million developers isn't a headcount. It's 19 years of everyone writing the textbook in the same language.
Hope Sterling: Okay — and there are actually signs the moat is cracking, ROCm 7, Anthropic's MI455X commitment — but the part that comes next is genuinely strange, because even where the crack is real, certain pieces of the stack like FlashAttention 3 and TensorRT-LLM and NCCL-based distributed training just... don't move. And I need Juniper to explain to me why that creates two totally different realities depending on what you're building.
Juniper Vale: And that two-tier thing is real — like, genuinely real, not a hedge. ROCm 7 lands in September 2025 with day-zero PyTorch and vLLM support. That's not a press release claim. That's a production-readiness threshold actually crossed. Standard ML workloads — inference on a known architecture, PyTorch training runs — those work on AMD now.
Hope Sterling: Wait, so that's like — that's genuinely new? Day-zero support?
Juniper Vale: That's what makes Anthropic's move make sense. They committed to up to 2 gigawatts of MI455X capacity — which is, I mean, that's not a pilot. A frontier AI lab betting that much on AMD only happens if ROCm has cleared a real bar.
Hope Sterling: Two gigawatts — like, that number is actually staggering. So is it actually breaking open?
Juniper Vale: For the standard stuff? Yes. But — okay, imagine a researcher at a well-funded lab who's just finished training a big model. Now she wants to deploy it. She reaches for TensorRT-LLM, which is how you actually squeeze production inference performance out of a model — quantization, layer fusion, all of that. And there is no AMD equivalent in production use. Nothing. She can't port that step.
Hope Sterling: So it's not that AMD is slower — it's that the optimization tool literally doesn't exist on their side.
Juniper Vale: Right. And NCCL is the same story — that's what handles communication across a giant distributed training cluster, and replicating that at scale is one of the hardest unsolved problems for any CUDA competitor. Intel's been trying with oneAPI for years and hasn't closed that gap either, which — actually, that's the tell. Because if this were just AMD executing badly, Intel would've cracked it. They haven't. So it's structural.
Hope Sterling: Oh, that's — wait, Intel too? That actually changes what this means.
Juniper Vale: Nineteen years of compounding. FlashAttention 3, TensorRT-LLM, NCCL at scale — those aren't features AMD forgot to ship. They're the product of a decade-plus of people writing code that only had to run on one thing. The crack is real. The core still holds.
Hope Sterling: The thing that I keep — okay wait, that's not quite right — the thing that I actually can't shake is that this wasn't inevitable. Like, NVIDIA made choices for 19 years that nobody else made, or nobody else made consistently enough, and the lock-in is the accumulated weight of those choices. It's not physics. It's just... a lot of Tuesdays.
Juniper Vale: That's the honest version of it, yeah. Anthropic paying the switching cost on 2 gigawatts of MI455X proves it's not unpayable — it's just that most organizations aren't Anthropic. The race is whether enough of them move before the next chip cycle buries the ecosystem another layer deeper.
Hope Sterling: And we don't know if they will. That's — I mean, that's actually where we land, isn't it. Not resolved. Just understood.
Juniper Vale: Which honestly feels right for this one. Thanks for bringing your friend's Friday nightmare into it — that made it real.
Hope Sterling: She's still debugging, by the way. Custom kernels.
