Onpode
Cover art for Why randomization and blinding are the only way to separate drug effect from expectation

Why randomization and blinding are the only way to separate drug effect from expectation

November 19, 2026 · 13 min

Marcus Kline & Ben Okonkwo

Tell me if this sounds familiar — a practitioner watches hundreds of cases, builds a mental model of who recovers, feels certain, is wrong. Not occasionally. Systematically. Mm, that's — that's the TB doctors before 1948, isn't it. The Medical Research Council's streptomycin trial. And the particular wrongness — it wasn't that the doctors were…

Randomized controlled trials (RCTs) are prospective study designs developed to establish whether an intervention causes an outcome, rather than merely correlates with it.

0:0013:18
Get the next episode on Health →

Follow it free — new episodes land in your feed.

Or make your own — any topic, in minutes

More Onpode episodes on Health →

About this episode

Randomized controlled trials (RCTs) are prospective study designs developed to establish whether an intervention causes an outcome, rather than merely correlates with it.

Grounded in 12 sources
The Efficacy and Safety of Knotless Barbed Sutures in the Surgical Field: A Systematic Review and Meta-analysis of Randomized Controlled Trials ↗ · doi.org
Assessing health system interventions: key points when considering the value of randomization. ↗ · doi.org
Determining Causal Exposure-Response Relationships With Randomized Concentration-Controlled Trials ↗ · doi.org
CLINICAL TRIALS AND DRUG REGULATION: A PHILOSOPHICAL INQUIRY ↗ · doi.org
The New Modern Medicine: Disease, Evidence, and Epidemiological Medicine ↗ · doi.org
Identification of tools used to assess the external validity of randomized controlled trials in reviews: a systematic review of measurement properties | BMC Medical Research Methodology | Springer Nat ↗ · link.springer.com
Large simple randomized controlled trials—from drugs to medical devices: lessons from recent experience | Trials | Springer Nature Link ↗ · link.springer.com
CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials ↗ · pmc.ncbi.nlm.nih.gov
Factors That Can Affect the External Validity of Randomised Controlled Trials ↗ · pmc.ncbi.nlm.nih.gov
Randomisation to protect against selection bias in healthcare trials ↗ · pmc.ncbi.nlm.nih.gov
Randomised controlled trials—the gold standard for effectiveness research ↗ · pmc.ncbi.nlm.nih.gov
A Reweighting Based Approach for Treatment Effect Estimation Under Unmeasured Confounding with Non-Representative Randomized Data ↗ · semanticscholar.org
Read transcript

Marcus Kline: Tell me if this sounds familiar — a practitioner watches hundreds of cases, builds a mental model of who recovers, feels certain, is wrong. Not occasionally. Systematically.

Ben Okonkwo: Mm, that's — that's the TB doctors before 1948, isn't it.

Marcus Kline: The Medical Research Council's streptomycin trial. And the particular wrongness — it wasn't that the doctors were incurious or lazy. The sicker patients came first, sought treatment most urgently, and improved faster because they started at the floor. Selection bias. The observation itself was the contamination.

Ben Okonkwo: Which is the definition of confounding — the factors that predict who seeks treatment are the same factors that predict recovery, so you can't separate the drug's effect from the patient's trajectory without a control group assigned independently of all of that.

Marcus Kline: And the MRC's answer was random number tables. Assign by chance. No clinician deciding, no preference, no pattern — just the numbers.

Ben Okonkwo: Wait — and the specific power there is that randomization balances both known and unknown prognostic factors across treatment and control. It's not just controlling for the things you've measured. It distributes everything, including the variables nobody thought to record.

Marcus Kline: That's the structural elegance. You don't have to know what you're controlling for. The chance assignment does it.

Ben Okonkwo: And yet — here's where I keep stopping — that move, that coin flip, answers something very specific. Within a defined population, under controlled conditions, over a fixed time window. The question it answers is not 'does this work for patients broadly.' It's narrower than that.

Marcus Kline: Which medicine, since the 1970s when RCTs became formally codified as the gold standard, has not always remembered.

Ben Okonkwo: No — and that gap between what the design actually proves and what the institution built on top of it, that's the real thing we're pulling on today.

Marcus Kline: One narrow question, answered very well — for 1948. The question is whether we ever noticed it was narrow.

Ben Okonkwo: And that narrowness is actually two separate problems disguised as one. Think about it this way — imagine you're testing whether a new coffee blend improves morning focus. If you let people choose their own coffee, the people who already wake up sharp will pick the premium blend, because they care more about focus to begin with. So their scores look great, but you don't know if it's the coffee or just... who they already were. Randomization takes that choice away. Coin flip decides who gets which blend. That's it. That's the whole move.

Marcus Kline: And blinding is the separate thing entirely.

Ben Okonkwo: Completely separate. Because now imagine the person rating your focus scores knows which coffee you got. Even unconsciously — they expect the premium blend to work, so they interpret your answers slightly more charitably. They're not lying. They just can't help it. Blinding is the fix for that. It operates after assignment, not before.

Marcus Kline: Pre-assignment, post-assignment. Two different failure modes. Two different mechanisms.

Ben Okonkwo: People lump them together and say 'that's the RCT.' But there's actually a third mechanism that's even more invisible, and I think this is the one that gets dropped from the story entirely. Allocation concealment.

Marcus Kline: Which is — walk me through that.

Ben Okonkwo: So you've built your randomization sequence. The coin has been flipped, so to speak. But if the researcher enrolling a patient can peek at the upcoming assignment — can see that the next slot is treatment — they can decide whether to offer this particular patient a spot in the trial. And suddenly the selection bias is back, before randomization even fires. Allocation concealment hides the sequence until after the patient is enrolled. It's a procedural lock on the moment of allocation.

Marcus Kline: So the randomization is intact on paper, and the bias has already walked in through the front door.

Ben Okonkwo: Exactly that. And this is — okay, actually, the sequencing matters here. Allocation concealment is pre-assignment, same moment as randomization, but it's a different problem. Randomization is about chance. Concealment is about preventing someone from gaming that chance before it resolves.

Marcus Kline: Three distinct mechanisms. And the design only holds if all three are intact.

Ben Okonkwo: Right — but blinding frequently fails. You cannot blind a patient to their own surgery. Can't blind a therapist running cognitive behavioral therapy. So for entire categories of intervention, you are running the trial with that post-assignment mechanism gone, and yet — the evidence hierarchy, Class Ib under the CONSORT framework, still treats those trials the same.

Marcus Kline: We're ranking a degraded design as though the full architecture is standing.

Ben Okonkwo: And here's what that means for the control group — which is the other piece. Without blinding, the control arm patients may change their behavior once they suspect they're on placebo. They seek out alternatives, adjust their diet, tell their family. The control group stops being a clean comparison. So the gap you're measuring isn't drug versus nothing — it's drug-plus-expectation versus placebo-minus-hope. The two safeguards aren't redundant. They're load-bearing in sequence.

Marcus Kline: And the specific scenario I keep returning to — a surgeon, a new knee repair technique. The patient wakes up. They can feel the incision. The physiotherapist is telling them exactly which exercises to do next, which is different from the other group. The nursing team is responding to them differently. You cannot blind any of them. And that trial still exits the system stamped Class Ib.

Ben Okonkwo: Class Ib. Same classification as a double-blinded drug trial where the mechanism is fully intact.

Marcus Kline: So the hierarchy — the evidence hierarchy codified since the seventies — is treating a trial that never had access to its own post-assignment mechanism the same as one that did.

Ben Okonkwo: And I want to name something specific here, because I think this is the part that gets genuinely lost. Biased outcome assessment is a distinct failure mode — it's not just 'blinding failed generally.' It's that an investigator who suspects a patient is in the treatment arm will unconsciously score a subjective symptom report more favorably. The randomization did everything correctly. The bias entered through the assessment window. That's a different problem.

Marcus Kline: After a clean assignment.

Ben Okonkwo: After a perfectly clean assignment, yes. The investigator looks at a pain score — and this is for subjective outcomes especially — and they're not falsifying anything. They're just... interpreting ambiguous signals in the direction their expectation points. And randomization cannot touch that. Only blinding can. And in a surgical trial, blinding is, the word the research actually uses is, 'practically impossible.'

Marcus Kline: Practically impossible — and yet the classification stands.

Ben Okonkwo: Now, the WHO has had to sit with a version of this that's even more uncomfortable — health system interventions. You want to randomize a hospital policy, an infrastructure change. You can't, ethically or logistically. So the field has had to decide what to do when the ideal design runs into real-world conditions that just... block it. And the gap between what the design promises and what it can actually deliver in those cases is not theoretical.

Marcus Kline: So the gold standard is partly design and partly aspiration — and the distance between those two things depends entirely on what you're testing.

Ben Okonkwo: Right — and I think that's the tension the CONSORT guidelines are still trying to work through. They updated again in 2025. Which is — okay, I find that interesting, because if the standard were settled, you wouldn't need to keep revising the reporting framework. The fact that it's still evolving tells you something.

Marcus Kline: An evolving standard for a gold standard. The phrase itself starts to buckle.

Ben Okonkwo: And there's a layer underneath this we haven't touched yet — the stricter you make the trial design, the narrower the population you're actually testing. Which means the answer gets more precise and potentially less transferable at exactly the same time. That's what I think is worth pulling on next.

Marcus Kline: Consider — the cleaner the trial, the more precisely it answers a question that may not be the question in front of any actual patient. That is a problem I want to stay inside for a while.

Ben Okonkwo: And that's the internal validity trap — randomization does its job perfectly, the trial is clean, and then the answer transfers to... almost nobody. Because the population that produced the clean result was selected to be testable, not to be representative.

Marcus Kline: Walk me through what that actually looks like. The exclusions.

Ben Okonkwo: Okay — so a physician has a patient in front of them. Seventy-eight years old, diabetic, early heart failure, three other conditions, taking maybe seven medications. And there's a trial — good trial, rigorous — but the enrollment criteria excluded patients over seventy, excluded diabetes, excluded polypharmacy. The internal validity is real. The answer inside that trial is real. But the patient in that exam room was not in the room when the evidence was made.

Marcus Kline: The evidence is clean. The question it answers is not theirs.

Ben Okonkwo: And this is — okay, I want to be careful here, because the scale of it. Medicine can accumulate a large body of trials. Meta-analyses, Class Ia evidence, the top of the hierarchy. And collectively those trials answer a narrow question with high confidence while leaving the broader clinical question — does this work for my specific patient — structurally underanswered. Not accidentally. Structurally.

Marcus Kline: Structurally. Because the design that maximizes internal validity requires the exclusions that create the gap.

Ben Okonkwo: Exactly — and that's not a flaw that better reporting fixes. Jonathan Fuller's 2025 book, 'The New Modern Medicine,' treats this as a genuinely philosophical question. Is RCT primacy a methodological necessity, or is it partly a regulatory and institutional convention we built and then forgot we chose?

Marcus Kline: And the hierarchy itself — the codification. That didn't come from the 1948 MRC trial. That came later.

Ben Okonkwo: The 1970s. That's when it was formalized — single RCTs as Class Ib, meta-analyses as Class Ia. Mattia Andreoletti has pushed on this from the drug regulation side, contesting whether that primacy is a pure epistemological fact or whether the regulatory apparatus shaped what counted as sufficient evidence. The standard didn't descend from the 1948 results. It was built, institutionally, two decades later.

Marcus Kline: And CONSORT — updated in 2025 — is the live evidence that the building is still happening.

Ben Okonkwo: Right — if the standard were settled, you wouldn't need to keep revising what counts as adequate reporting. The 2025 update exists because prior RCT reporting was inadequate. Which means what we call the gold standard is partly methodological fact and partly an ongoing institutional negotiation about which kinds of evidence medicine decided to trust.

Marcus Kline: So the gold standard is not a fixed point. It is a direction of travel.

Ben Okonkwo: And that matters because — wait, this is the part that I think is genuinely hard — the physician looking at that Class Ia meta-analysis isn't just asking 'is this evidence strong.' They're asking 'can I trust this was conducted fairly.' And those are, actually, different questions.

Marcus Kline: One is methodological. One is about institutional legitimacy.

Ben Okonkwo: And the CONSORT guidelines — the external validity gap — none of it dissolves that second question. The physician still has to decide whether to trust a clean answer to the wrong patient. And that's not something the evidence hierarchy was designed to solve.

Marcus Kline: And yet — randomization is still the only design that actually balances the unknown prognostic factors. No observational study can replicate that. That's not institutional loyalty, that's just structurally true. So I'm not arguing against the coin flip. The coin flip in 1948 was an act of genuine humility — medicine conceding it couldn't trust its own clinical eye. What I can't resolve is what humility looks like now, when the coin flip itself has the limits we've named.

Ben Okonkwo: The gap I can't close is a practical one. Trials are getting narrower — stricter inclusion criteria, cleaner populations. The patients accumulating in front of physicians are getting older, more complex, more medicated. Those two curves are moving in opposite directions, and CONSORT reporting — even the 2025 revision — doesn't address that drift. It addresses transparency. That's not the same problem.

Marcus Kline: No. It isn't.

Ben Okonkwo: And the thing I genuinely can't settle is — if a drug works in that tightly controlled trial and doesn't work for the patient who was excluded from it, the trial wasn't wrong. It may have been right. It just answered a question that wasn't hers. I don't know what a physician does with that, honestly. That's not a reporting failure. That's a structural feature of how we decided to build the evidence.

Marcus Kline: What we're still working out — seventy-some years after those random number tables — is how to hold both things. The design is sound and the trustworthiness is earned, trial by trial, through whether the implementation was honest. And even then the answer might not transfer. Those aren't contradictions. But they don't resolve, either.

Ben Okonkwo: No. They really don't. Good conversation to sit inside for a while.