David Sterling: Hey — before we start, I want to just say: this is a heavier one. I think we should let it be heavy.
Megan Skiendel: Agreed. No spinning it into something tidy.
David Sterling: Here's where I want to start. Amgen — the pharmaceutical company — sets out to replicate fifty-three landmark preclinical cancer biology studies. Published in top journals. Cited extensively. Foundational work. They reproduced the results of six. Six out of fifty-three. That's an 89% failure rate on the science that the field built on.
Megan Skiendel: And — wait, I want to make sure we sit on that — these weren't obscure papers. These were the ones that sent researchers in a direction for years.
David Sterling: Cited hundreds of times. Shaping drug-trial design. The cost — the misdirected research spend, the downstream clinical choices — that cost was already paid long before anyone audited the original work. That's the lag. That's what makes the reproducibility crisis something other than a quality-control glitch.
Megan Skiendel: It's the gap between when the error enters the system and when anyone actually catches it — if they catch it.
David Sterling: So the question I want to work through is: how does this happen repeatedly, inside a system full of smart people, publishing in journals like the New England Journal of Medicine, funded by the NIH? Because the answer isn't incompetence.
Megan Skiendel: No. Listen, the answer is that the incentive architecture makes this the rational outcome. No one has to intend it. It emerges.
David Sterling: That's the thesis. And it's uncomfortable precisely because it means you can't fix it by finding the bad actor.
Megan Skiendel: There isn't one. That's the whole problem.
David Sterling: But saying 'it emerges' still lets everyone off the hook a little too cleanly. What's the actual mechanism?
Megan Skiendel: Okay — think of it like this. Imagine every film critic only got paid when they reviewed a blockbuster opening weekend. Over time, the archive of criticism skews toward hits. Not because critics are lying. Because the ones who reviewed small films quietly went broke. That's publication bias. That's the whole thing.
David Sterling: And the small films never get made either, eventually.
Megan Skiendel: Exactly — the null results disappear upstream, before publication even. A 32-year-old assistant professor up for tenure submits the study that worked. Not the three that didn't. She can't afford the CV hit. That's not misconduct. That's math.
David Sterling: She's not lying. She's optimizing. The NIH has actually named this — they've called publish-or-perish the driver. Explicitly. The funding agency that runs the system is diagnosing the system.
Megan Skiendel: And still the metric doesn't change.
David Sterling: Right. So where does novelty bias fit? Because journals want citations, and citations follow surprise. That's a separate lever from tenure, but it's pulling the same direction.
Megan Skiendel: It's downstream of the same thing — honestly, I mean, funders want innovation, editors want impact factor, researchers want publication. Every single actor is rational. The crisis is what falls out in aggregate. No one sat in a room and decided to corrupt the literature.
David Sterling: Principal-agent problem with a fifteen-year feedback loop. Patients bear the cost. Researchers capture the reward.
Megan Skiendel: And it gets more insidious than selective submission. There's a whole spectrum — questionable research practices — biased experimental designs, selective outcome reporting, running your stats five different ways until one clears the significance threshold. None of it is fraud. All of it is a rational response to the architecture. And all of it inflates false-positive rates.
David Sterling: Defensible at every individual step.
Megan Skiendel: Every step. And that's actually what makes it harder to fix than fraud — you can't prosecute someone for being rational inside a broken system. The 2009 data on this is brutal: positive trials were 3.9 times more likely to be submitted and published than negative ones. That's not noise. That's the filter, quantified.
David Sterling: And 3.9 times means the literature your postdoc inherits is not a sample of reality. It's a sample of what cleared the incentive.
Megan Skiendel: And not a sample of what's true — that's the distinction that matters. The literature isn't biased toward interesting. It's biased toward positive. Those aren't the same thing.
David Sterling: Right. The 2009 number — 3.9x — that's not capturing sloppiness. That's capturing suppression. The null results exist. They just never move.
Megan Skiendel: Picture it concretely. Tuesday morning, a postdoc finishes her analysis. The intervention didn't work. No effect. She stares at that for an hour — I mean, an actual hour of sitting with it — and then she sets it aside. That's publication bias. Not a submission that got rejected. A submission that never happened.
David Sterling: The silence is the data point.
Megan Skiendel: Exactly. And the New England Journal of Medicine flagged this — not some fringe methods journal, the most prestigious address in medicine — they named promotion committees specifically. That reviewers favor innovative over replicating. So the bias is baked in at the credentialing layer, not just the publishing layer.
David Sterling: Which means two filters. The researcher doesn't submit the null. And even if she did, the committee that evaluates her tenure doesn't reward it.
Megan Skiendel: She can't win on that study. At all.
David Sterling: So what accumulates in the literature is not — frankly, it's not a record of what works. It's a record of what surprised someone enough to submit and what a tenure committee would reward on a CV. Those are different populations.
Megan Skiendel: And P. Angadi's 2026 editorial — this is the part that stops me — it frames irreproducibility as a live, ongoing problem. Not historical. Not 'we identified this and fixed it.' Still diagnosing it, right now.
David Sterling: Still unresolved. After decades of naming it.
Megan Skiendel: Which is — honestly, that gap between knowing and acting is its own problem. And it connects to something we haven't touched yet: what happens when that skewed literature gets used under pressure, in real time, for actual clinical decisions. That's where the cost becomes impossible to ignore.
David Sterling: The downstream. That's the question. The 3.9x distortion isn't abstract once someone is building policy on the pool it produced.
Megan Skiendel: And we have a pressure test for exactly that — which we'll get to. But the reform question is already biting: Medical Teacher adopted Registered Reports in 2026, first health professions education journal to do it. One. That number tells you something about the gap between the diagnosis and the institutional will to act.
David Sterling: One journal. And the thing that breaks the clean story is — who's adopting it? Medical Teacher is a niche outlet. Health professions education. That's not the tenure-track oncologist's target. That's not where the pressure lives.
Megan Skiendel: Dario Cecílio-Fernandes and Jennifer Cleland announce it in 2026, and it's framed as a milestone. Which — it is. But it's also an indictment. First in category. After decades.
David Sterling: That's the adoption paradox. The journals with the most power to fix this — high-impact, high-prestige — have the most to lose from decoupling novelty from publication. Their impact factor is built on surprise.
Megan Skiendel: And the least institutional pressure to change. No one is forcing them.
David Sterling: NIH has named it. Explicitly — publish-or-perish rewards favorable results, under-rewards replication. That's the funding agency saying the system is broken. And then the funding structure stays.
Megan Skiendel: Oh, honestly — that's what gets me. They diagnosed it. In public. And the metric didn't move.
David Sterling: So what does Registered Reports actually solve, structurally? Peer review before data collection, acceptance on methodology not outcome — that directly kills the mechanism. But only for the researchers who submit to those journals.
Megan Skiendel: Which brings us to COVID. Because that's — I mean, that's where the abstract becomes a body count question. Fast-tracked pandemic literature, urgency driving every editorial decision, and the existing incentive architecture didn't pause. It accelerated.
David Sterling: The 2025 meta-research study. Elevated risk of bias, low reproducibility — specifically in the meta-analytic evidence that was shaping policy.
Megan Skiendel: That's the stress test result. The system didn't buckle under pressure — it performed exactly as designed. Speed plus existing incentives equals a policy-shaping evidence base that is, structurally, unreliable.
David Sterling: And it was treated as settled. Meta-analyses going straight into clinical protocol.
Megan Skiendel: A pediatric rheumatologist reading a corticosteroid meta-analysis, twenty-three papers in the pool, doesn't know that the positive-result filter was running the whole time. She sets a protocol. Kids get enrolled in studies downstream of that decision. The lag problem David named at the top — that's what it looks like when it lands on a person.
David Sterling: And the open science practices — preregistration, data sharing — they're real reforms. But they sit alongside the unchanged incentive architecture. Not inside it. That's the gap.
Megan Skiendel: Registered Reports are structurally correct. That's not the debate. The question is whether a format adopted by one health professions education journal can scale into the venues where the high-stakes literature actually lives.
David Sterling: And the institutions most responsible for the crisis have the least incentive to change. That's not pessimism — that's the principal-agent problem, applied to the reform itself.
Megan Skiendel: The reform is real. The adoption problem might be terminal. And that's the hardest thing to sit with — not that no one tried, but that the trying is structurally contained to the places that can afford to try.
David Sterling: And I don't have a clean answer for this — but I think we may not be watching a system that's failing to reform. We may be watching the two-tier system actually form. Right now. High-prestige journals publishing unreliable novelty, Registered Reports living in the margins. Medical Teacher, niche outlets, places where the tenure-track oncologist is not submitting. That's not a temporary gap while reform catches up. That might just be the equilibrium.
Megan Skiendel: And the knowing-versus-acting gap — listen, that gap has been decades wide. The New England Journal of Medicine was naming promotion committee bias before most of these researchers were in grad school. NIH called it out in public. The diagnosis has been sitting there. The institutions most resistant to changing are the ones that have known the longest.
David Sterling: Which reframes the question. It's not whether the structural reforms are correct — Registered Reports are correct, the mechanism is sound. The question is whether those reforms can be adopted by the institutions most resistant to them. And the answer, so far, is: one journal. First in category. 2026.
Megan Skiendel: One. After all of it.
David Sterling: We started with Amgen. Fifty-three studies. Six replicated. That number felt like an indictment of a few bad papers. By now it reads differently — it's the expected output of a system running exactly as designed. The 89% failure rate isn't a scandal. It's a measurement of the architecture.
Megan Skiendel: And the architecture is still running. That's where I land. Not in despair, exactly — more like, we know what it produces, we can see it producing it, and the question of whether it changes is genuinely open. I don't have an answer for that. I'm not sure one exists yet.