Onpode
Cover art for Anthropic quietly lobbies for AI safety rules that only trigger after 50 deaths or $1B in damages

Anthropic quietly lobbies for AI safety rules that only trigger after 50 deaths or $1B in damages

October 18, 2026 · 9 min

Eliza Ward & Brian Reed

Anthropic privately circulated draft legislation to Congress that would limit federal AI enforcement to a single agency — the Commerce Department — and only when an AI system foreseeably and imminently causes both 50 or more deaths and over $1 billion in damages, across just three qualifying risk categories.

Anthropic, the AI safety company and developer of the Claude chatbot, has privately circulated a draft federal AI safety bill to U.S. lawmakers that would set high thresholds for government intervention in advanced AI model deployment.

0:009:23
Get the next episode on Artificial Intelligence →

Follow it free — new episodes land in your feed.

Or make your own — any topic, in minutes

More Onpode episodes on Artificial Intelligence →

About this episode

Anthropic has quietly circulated draft federal legislation that would set the threshold for government AI intervention at 50 or more deaths and $1 billion or more in damages — and both conditions have to be met simultaneously, alongside one of three specific risk categories. No documented AI incident has ever cleared all three at once. This episode digs into what that conjunctive structure actually means in practice: a recruiting algorithm causing hundreds of millions in economic harm to workers doesn't trigger it. Neither does most of what AI critics would call serious, real-world damage. But the thresholds, as striking as they are, might not even be the central issue. The draft consolidates all federal AI enforcement authority into a single agency — the Commerce Department — while explicitly blocking HHS, the EPA, the FBI, and every other department from acting independently. There's one court on appeal. And the safety evaluators who would flag imminent risks? Paid by the labs they evaluate, with no verification mechanism in the bill itself. Anthropjc framed this to Congress as regulatory predictability. Critics call it a framework calibrated to remain permanently idle. This episode works through the legal structure, the competing bills currently stalled in Congress, and what it actually means that the company best known for AI safety is also the author of the enforcement architecture.

Frequently asked

What are the thresholds in Anthropic's proposed AI safety bill?

Anthropic's draft bill requires a foreseeable, imminent risk of both 50 or more deaths and over $1 billion in damages — and the harm must fit one of three categories: bioweapon creation, an AI-enabled cyberattack, or an AI model circumventing its own developer's safety controls. All conditions must be met simultaneously.

Which agency would enforce AI safety under Anthropic's proposed bill?

Under Anthropic's draft legislation, the Commerce Department would be the sole federal enforcer of AI safety rules. Every other federal agency — including HHS, the EPA, and the FBI — would be explicitly barred from acting independently, with disputes routed to a single federal court.

Has any real AI incident ever met the thresholds in Anthropic's bill?

According to AI safety researchers cited in reporting on Anthropic's draft bill, no documented AI incident has simultaneously hit both the $1 billion financial threshold and one of the three qualifying risk categories. Critics argue this makes the enforcement framework functionally idle against real-world harms.

Who evaluates AI models for safety under Anthropic's proposed legislation?

Under Anthropic's draft bill, AI labs would pay their own independent evaluators to assess model safety. The bill states payment must not affect independence, but includes no audit mechanism to verify that condition. If an evaluator flags an imminent risk, they have 72 hours to report it to the Commerce Department.

How does Anthropic's AI safety bill compare to the FRONTIER Act?

The FRONTIER Act, a bipartisan House bill from Representatives Trahan and Obernolte, gives the government power to halt deployment of risky AI models without restricting which agencies can act. Anthropic's draft, by contrast, consolidates all enforcement authority in a single agency and requires a conjunctive set of conditions before any action can be taken.

Grounded in 11 sources
AI rapidly improving, could vastly boost U.S. GDP, Anthropic's Clark tells policymakers - CNBC ↗ · cnbc.com
Anthropic wants outside input on Claude’s values. But who gets the final say? - NBC News ↗ · nbcnews.com
Anthropic pitches AI safety bill to Congress ↗ · politico.com
Anthropic bill sets $1B/50-death thresholds for AI halts ↗ · reuters.com
Anthropic proposes AI safety bill with strict intervention thresholds - Politico ↗ · investing.com
Daniela Amodei warns against AI girlfriends, Business Insider reports - The Next Web ↗ · thenextweb.com
Bill Gates leans on CEOs as he plans a bigger AI safety push, Axios reports - The Next Web ↗ · thenextweb.com
Anthropic Quietly Pushes AI Safety Bill That Would Limit Government Power to Block Dangerous Models | Law Commentary ↗ · lawcommentary.com
Anthropic Pitches Congress a $1B AI Safety Bill ↗ · shattered.io
Anthropic Pitches $1B AI Safety Bill to Lawmakers ↗ · tech-insider.org
Anthropic's proposed AI safety bill only kicks in if 50 deaths or $1B in damages ↗ · tomshardware.com
Read transcript

Brian Reed: Hey. Did you get through the Politico piece?

Eliza Ward: Yeah, twice actually.

Brian Reed: The thing that landed for me — and I want to see if it hit you the same way — is that Anthropic shopped this privately. Not a white paper, not a public comment. They circulated draft legislative language directly to lawmakers.

Eliza Ward: October 16th. And the draft has the Commerce Department as the sole federal enforcer — every other agency explicitly blocked from acting on its own.

Brian Reed: Right, and the threshold for Commerce to do anything at all is a foreseeable, imminent risk of over a billion dollars in damages, or fifty or more deaths. Those are the numbers Anthropic put in the bill.

Eliza Ward: Fifty deaths as a — as a floor.

Brian Reed: As a floor, yeah. And I don't know if that's a carefully modeled number or if it's — let me see — if it's just a number they think they can defend. That distinction matters a lot.

Eliza Ward: It does. And there's nothing in what's been reported that tells us which one it is.

Brian Reed: But hang on — because the number thing is actually the smaller story. The part that stopped me cold is that it's not a billion dollars or fifty deaths. It's both. And the harm has to fit one of three specific boxes.

Eliza Ward: Right. Conjunctive. The legal text is conjunctive.

Brian Reed: Which means — okay, think about a smoke detector that only sounds if there's a fire and a gas leak and the flames are a specific color. Miss any one condition, no alarm.

Eliza Ward: And the three colors are: biological weapon creation or release, an AI-enabled cyberattack, or an AI model circumventing its own developer's safety controls. That's it. Those are the only qualifying risks.

Brian Reed: So a recruiting algorithm that wipes out a hundred thousand job opportunities — real harm, real dollars — doesn't qualify. It's not a bioweapon. It's not a cyberattack.

Eliza Ward: Doesn't even get Commerce to the door. And AI safety researchers outside the industry are saying — I mean, this is the part that's actually new — no documented AI incident has simultaneously hit both the dollar threshold and one of those three categories. Not one.

Brian Reed: That's — I keep sitting with how strange that is. Anthropic is grounding this in their Advanced AI Framework from June, presenting it as calibrated policy, and the calibration apparently points to a bar nothing real has ever cleared.

Eliza Ward: Anthropic's answer is: these target catastrophic frontier risks, not ordinary harms. But critics are basically saying that framing makes the bill functionally permissive — wait, not just permissive. Permanently idle.

Brian Reed: And the foreseeable and imminent standard on top of all that — the risk can't be speculative, it has to be both foreseeable and imminent before Commerce can move. So what documented incident would've actually cleared the whole test?

Eliza Ward: That's the part I want to push on — because everyone's treating the thresholds as the story. But even if you cut the numbers in half, the structure still leaves you with one agency. The Commerce Department. Every other federal department is explicitly barred from acting independently. HHS, EPA, FBI — no independent lane.

Brian Reed: That's the take that isn't circulating enough. The numbers get the headlines. The single-agency clause is the actual mechanism.

Eliza Ward: Anthropic calls it regulatory predictability. Which — I mean, that's a real thing companies want. But predictability for the lab and redundancy for the public are not the same design goal.

Brian Reed: And the part that doesn't fit the 'predictability' framing is — hang on, let me get this right — if Commerce is slow, or underfunded, or just makes the wrong call, there's no other lever. It all routes through Commerce and then one federal court. That's it.

Eliza Ward: One court.

Brian Reed: Which is why the FRONTIER Act is actually the structural comparison worth making. Trahan and Obernolte — bipartisan House bill — gives the government power to stop deployment of risky models. Doesn't restrict which agencies can act. That's a meaningfully different architecture.

Eliza Ward: And Anthropic is circulating its draft into a specific vacuum. September 29th — Mark Warner tried unanimous consent on a Senate AI safety bill. Ted Cruz objected, blocked it. The bill is stuck in the Senate Commerce Committee right now.

Brian Reed: So the legislative space is genuinely open and Anthropic moves into it with something that consolidates enforcement authority. That's not an accident of timing.

Eliza Ward: And wait — the evaluator piece actually makes the single-agency problem worse, not better. The conflict is harder to defend — and we'll get to the full mechanics of it in a minute.

Brian Reed: The evaluator thing is — okay, the mechanics of it are actually stranger than they sound. The labs pay their own evaluators. That's in the draft. Provided payment does not affect independence. But there's no mechanism in the bill to verify that condition. None.

Eliza Ward: No audit of the auditor.

Brian Reed: And if the evaluator finds an imminent enumerated risk — one of those three boxes — they've got 72 hours to report it to Commerce. That's it. No parallel path. If the company disputes it, that goes to federal court. A judge decides whether the bioweapon risk is real.

Eliza Ward: Which is — wait, that's not how pharmaceutical risk gets adjudicated. Or aviation. A federal judge isn't the first call when a drug fails a safety flag.

Brian Reed: Right. And here's the scenario I can't get past. Picture a hiring algorithm — not bioweapon territory, not a cyberattack — just a model systematically filtering out women in mid-career transitions. Deployed by a Fortune 500 company, say, late 2027. Documented harm to hundreds of thousands of workers. Eight hundred million dollars in measurable economic damage. Under this draft, that triggers nothing. Below the financial threshold, outside all three risk categories.

Eliza Ward: Eight hundred million dollars. Not a rounding error.

Brian Reed: And Commerce can't move. HHS can't move independently. The EEOC might have a separate lane, but under single-agency exclusivity, the federal AI enforcement response is — nothing. The framework just doesn't see it.

Eliza Ward: And this is where Anthropic's dual identity gets genuinely — I mean, it's not cynical, exactly. Their representative told Congress, reported by CNBC on October 17th, that AI could vastly boost U.S. GDP. That same week the Advanced AI Framework from June is the literal template for the bill. So Anthropic is simultaneously the safety company, the economic growth witness, and the author of the enforcement architecture. Those roles aren't contradictory. But they're also not the same role.

Brian Reed: The thing to watch — and I think this is actually the concrete signal — is whether the FRONTIER Act sponsors, Trahan and Obernolte, respond directly to the evaluator independence language. Because if that conflict survives into any merged draft, the oversight mechanism is built on a structural conflict with no public verification. That's the line.

Eliza Ward: Congress might not actually test the calibration question before it votes. The draft hasn't been formally introduced. As of October 16th it was still privately circulating. So there's no committee markup, no CBO score, no moment where someone has to sit in front of Trahan or Obernolte and say: show me the incident inventory. Show me what this threshold was modeled against.

Brian Reed: And the Warner bill — still sitting in Senate Commerce after Cruz blocked it September 29th — that's the other live piece. Because if that goes nowhere, and the FRONTIER Act stalls, Anthropic's draft is the thing with momentum. Not because it was introduced. Just because it exists and the others don't.

Eliza Ward: Which is — wait, that's actually the floor-versus-ceiling question in concrete form. Is Anthropic setting the minimum standard Congress builds on, or is it occupying the space where a stricter bill might've gone?

Brian Reed: And I genuinely don't know. That's not a hedge — I mean, the safety credibility Anthropic built is real. Claude exists, the Advanced AI Framework from June is a real document, their representative was in front of Congress making GDP arguments the same week. They have standing. The question is whether that standing substitutes for scrutiny on the threshold numbers.

Eliza Ward: No confirmed next step. That's where we are. Privately circulated, not introduced, no hearing date. What would actually settle it is someone in the Senate Commerce Committee running the documented incident inventory against the conjunctive threshold — publicly, on the record. Until that happens, we don't know if this is a floor or a ceiling. And I'm not sure Congress knows either.

Anthropic quietly lobbies for AI safety rules that only trigger after 50 deaths or $1B in damages · Onpode