Cole Bryant: Marcus, my sister called me last night asking whether she should trust some health study she saw — and I realized I had no actual answer. Like, I know the words 'correlation isn't causation,' but I couldn't tell her when it actually matters.
Marcus Vale: That's exactly the right question to not be able to answer, honestly.
Cole Bryant: Which — okay, that's a little alarming coming from you.
Marcus Vale: Here's the deal. Correlation versus causation sounds like a philosophy lecture until you learn that in 1997, observational studies showed hormone replacement therapy cut heart disease risk by 35% — and then a Randomized Controlled Trial in 2002 ran the same question and found HRT actually increased that risk. Same question. Opposite answer. The trial was right.
Cole Bryant: Bro — okay hold on. Thirty-five percent protective, then the randomized trial flips it to harmful?
Marcus Vale: Complete reversal. And women were already on it. That's not a refinement of a finding. That's the floor dropping out.
Cole Bryant: So the observational study wasn't just imprecise — it was structurally pointing the wrong direction. Which means... I mean, what is an observational study actually measuring, versus what the Randomized Controlled Trial is measuring? Because clearly they're doing something different.
Marcus Vale: That's the whole engine of what we're getting into. Observational studies track what correlates with outcomes. Randomized Controlled Trials isolate what causes them. That gap — that's where the HRT disaster lived.
Cole Bryant: And the gap is big enough that you get opposite answers.
Marcus Vale: Big enough that guidelines changed twice. So — how does the same question produce answers that aren't just different, but inverted? That's what I want to work through.
Cole Bryant: Okay but — the gap being that big makes me want to understand what actually causes it. Like, I keep saying 'confounding' but I don't think I could actually explain it to my sister. And she'd ask. She'd absolutely ask.
Marcus Vale: Picture this. People who carry lighters get lung cancer at higher rates. Statistically, undeniably true. Does the lighter cause the cancer?
Cole Bryant: No — no, the lighter is just... the smokers have the lighters.
Marcus Vale: Smoking is the confounder. It's associated with carrying a lighter and it's associated with cancer. So the lighter and cancer look connected in your data. They're not. That's a spurious association — it's a statistical relationship that isn't real.
Cole Bryant: That's — wait, that's the whole thing, right? That's the core mechanism. A hidden third variable touching both sides of what you're measuring.
Marcus Vale: That's it. One sentence: a confounding variable is a third thing associated with both your exposure and your outcome, and it makes them look causally connected when they're not.
Cole Bryant: Now Digoxin — because I feel like this is actually alarming. Like, imagine a nurse, she's going through a pharmacy database at 2 a.m., and every chart she pulls where someone died — Digoxin is right there. Every single one. She's not — I mean she's not imagining it.
Marcus Vale: She's not wrong about the correlation.
Cole Bryant: Right! The correlation is real. But what she's actually reading is — she's reading a mirror of how sick those patients were already. The sickest patients are precisely the ones who get Digoxin prescribed. So of course they're dying at higher rates.
Marcus Vale: The observational studies on Digoxin — they adjusted for known confounders. Age, diagnosis severity. They were doing it right, or at least what 'right' looked like. Still showed increased mortality. Then a prospective randomized trial ran the same question and found no mortality increase. Zero.
Cole Bryant: Wait — they adjusted and were still wrong?
Marcus Vale: Still wrong. Because the confounders they couldn't measure — the unmeasured residual disease severity — weren't in their adjustment model. You can only adjust for what you know to look for. The stuff you don't know is still in the data, pulling the signal.
Cole Bryant: That's — dude, that's the actually scary part. It's not that the researchers were sloppy. They did the adjustment work. They thought they had it. The unknown unknowns just — they were baked in underneath everything they could see.
Marcus Vale: Which is why the randomized trial is doing something fundamentally different. It's not adjusting for confounders. It's scrambling them — distributing them across both groups so they cancel out. And that move handles the ones you'd never think to name. That's actually Fisher's insight from the 1920s, and nobody's topped it.
Cole Bryant: Wait — nobody's topped it. That's a wild thing to say about a guy doing math in the nineteen-twenties.
Marcus Vale: Fisher's proof is basically this: if you assign people to treatment or control randomly — coin flip, lottery, whatever — then every characteristic about those people, age, genetics, how sick they already are, stuff you'd never think to measure — gets shuffled across both groups. Before the drug is even given. The groups are balanced on everything. Known and unknown.
Cole Bryant: The unknown part is — I mean, that's the part I keep getting stuck on. Because with Digoxin, the researchers were adjusting for disease severity. They had that variable. And they were still wrong. So the unknowns they couldn't name were doing more damage than the knowns they could fix.
Marcus Vale: Exactly. And Fisher's move sidesteps that problem entirely. You don't have to name the confounder. You don't have to measure it. Randomization distributes it.
Cole Bryant: So if you'd randomized the Digoxin patients at the start — like, literally just flipped a coin on who gets the drug — disease severity would've been roughly equal across both groups before anyone took anything.
Marcus Vale: Right — and that's exactly what the prospective randomized trial did. Sicker patients ended up in both arms. The treatment effect you're measuring is clean. That's what internal validity actually means — the design itself rules out the alternative explanation.
Cole Bryant: No, wait — internal validity is the design doing the work, not the statistics cleaning up after.
Marcus Vale: That's the distinction. Statistical adjustment is retrofitting. Randomization is structural — it happens before data collection, so there's no cleanup required.
Cole Bryant: Which is — what makes Fisher's insight actually durable. Because it's not 'here's a better adjustment algorithm.' It's closer to — it's almost architectural? Like, he changed what the experiment is, not how you analyze it after.
Marcus Vale: That framing is right. And frankly, every causal inference method built since — propensity scoring, instrumental variables, the machine learning approaches on EHR data — they're all attempts to approximate what randomization does structurally. None of them fully get there. Which is a thread worth pulling on in a minute, because the places where an RCT can't be run are exactly where this gets uncomfortable.
Cole Bryant: Hold on — uncomfortable how?
Marcus Vale: The Randomized Controlled Trial has an Achilles heel that Fisher's math doesn't fix. But that's — we'll get there.
Cole Bryant: Bro, you can't just — okay. Fine. But the structural thing is what I want to land on because I think people underestimate it. Fisher in the nineteen-twenties basically said: stop trying to think of every way your data could be lying to you, because you will miss one. Instead, build the study so the lies get distributed equally and cancel out. That's — that circles back to why nobody's replaced it. It's not a technique. It's a different premise.
Marcus Vale: But here's the crack in Fisher's architecture — and it's real. The RCT gives you internal validity. That's clean. But internal validity and external validity pull in opposite directions. The tighter you control the trial, the less it looks like the actual world.
Cole Bryant: Wait — what do you mean, less like the actual world?
Marcus Vale: You run an RCT, you need a clean population. So you exclude people over seventy-five, pregnant women, people with multiple diagnoses. You monitor adherence obsessively. You're basically running the drug in a lab with humans. And then you take that finding and apply it to a seventy-eight-year-old woman with three other conditions who skips doses.
Cole Bryant: She was never in the study.
Marcus Vale: Never in the study. That's external validity — how far the finding actually travels into the real world. And observational studies, just by nature, capture that seventy-eight-year-old because she's already in the clinic. She's already in the Electronic Health Records.
Cole Bryant: Wait, I mean, this is almost the inverse of the HRT problem, right? Like the RCT is the thing that saved us from the bad observational data on HRT. But now you're saying the RCT's own control is the thing that breaks it for real-world use?
Marcus Vale: That's exactly the tension. And it gets worse in practice — loss to follow-up, nonadherence. People drop out of trials. People don't take the drug. So even the internal validity you paid for starts leaking.
Cole Bryant: So the gold standard has rust on it.
Marcus Vale: Structural rust, yeah. And then there's a wall you literally cannot cross. You cannot randomize people to smoke for thirty years. You cannot randomize a community to live next to a refinery. The exposure is the harm. Ethics blocks the door permanently.
Cole Bryant: Which is why — Bradford Hill Criteria, right? Like, that whole framework exists because causal reasoning from observational data wasn't a workaround, it was the only option anyone ever had for stuff like that.
Marcus Vale: Exactly. Bradford Hill didn't invent those criteria because he loved observational data. He invented them because sometimes randomization is permanently off the table and you still need a disciplined way to argue causation. Dose-response, biological plausibility, consistency across studies — it's a framework for reasoning carefully when you can't run the experiment.
Cole Bryant: And then Mendelian randomization comes along and it's like — someone found a cheat code? Because your genes are assigned at conception, not chosen by your behavior, so they actually approximate the randomness of an RCT without — I mean, without running one.
Marcus Vale: That's the instrumental variable move. Find a lever that pushes the exposure but has no direct path to the outcome. Genes fit because nobody chose their genetics based on their diet or lifestyle. So if a genetic variant predicts, say, alcohol metabolism, you can use it to estimate what alcohol actually does to cardiovascular risk without telling anyone to drink.
Cole Bryant: Bro, that's — okay, the specific thing that gets me about that is the gene is already randomized. At conception. Fisher's whole move, but nature did it.
Marcus Vale: Nature did it. And now the frontier is machine learning on Electronic Health Records and insurance claims — massive observational datasets, algorithms trying to surface causal signals at a scale no trial could touch. But here's where I get nervous. If the algorithm can't identify every confounder a human researcher missed — and it can't, by definition — then confounding hasn't been solved. It's just moved into the black box. You've made it harder to see, not smaller.
Cole Bryant: The confounding migrates. It doesn't disappear.
Marcus Vale: That's the unresolved part. The EHR data is real-world, which means external validity is strong. But the causal signal is still dirty. And nobody's shown me the proof point where the ML caught a confounder that human adjustment missed AND a later randomized trial confirmed it was right. That case doesn't exist yet.
Cole Bryant: And that's the thing I keep — I mean, that's what I'd tell my sister now. Not 'correlation isn't causation,' because that's useless. It's more like... every method we have for getting from correlation to causation just moves the problem somewhere harder to see. Adjustment moves it into unmeasured variables. Machine learning moves it into the black box. Even Mendelian randomization requires the gene to have no back-door path to the outcome — and proving that is its own whole problem.
Marcus Vale: Confounding doesn't disappear. It migrates. That's the honest summary of where causal inference is right now. Observational studies tell you what correlates with outcomes in the real world. Randomized trials tell you what causes them. That distinction is still clean. The question nobody's answered — and I mean nobody — is whether any method applied to observational data can fully substitute for randomization, or whether we've just gotten very good at relocating the uncertainty.
Cole Bryant: And the HRT reversal happened with careful researchers. That's the part that doesn't let me go. They weren't cutting corners. Best available methods. Still got a 35% protective signal that was pointing exactly the wrong direction for years.
Marcus Vale: Yeah — and that's where I land. Not that the new tools aren't real. They are. But Fisher's move from a hundred years ago still hasn't been topped structurally. And until someone shows me a case where ML on EHR data called a causal effect that randomization later confirmed — not just plausible, actually confirmed — I think we're still in the same epistemological house Fisher built. We've added rooms. We haven't left the building. Anyway. That's a good place to stop turning it over.
