Lila Soto: Iris, hi — quick question before we start: do you think an AI company can call itself safety-first and also be the company processing your health insurance claim?
Iris Holm: That's not a question you open with — that's a question you earn. What happened?
Lila Soto: Okay — so July 14th, Fierce Healthcare reports that UST has gone live with Claude inside CarePath. Their actual claims processing platform. And then, same news cycle, Anthropic announces a strategic partnership with Optum — that's UnitedHealth Group's health services arm — and also Elation Health, which is an EHR company.
Iris Holm: Three deals, one day. CarePath isn't a demo — it's the operational layer for health insurers and providers.
Lila Soto: And the integration goes deep — Claude Code, MCP connectors, wired directly into the underlying claims and care management systems. That's — I mean, that's not a chatbot answering questions.
Iris Holm: No. That's Claude touching live claims data in production. The Optum piece is what scales it — Optum sits inside one of the largest health insurers on earth.
Lila Soto: So back to my opening question — honestly, I don't think it's rhetorical anymore.
Iris Holm: Frankly, no. It's not. And that's the thing we need to actually work out today.
Lila Soto: So what does that actually look like, day-to-day? Like what is CarePath with Claude doing that it wasn't doing before?
Iris Holm: Think of it like autocomplete — but not for a sentence. For a decision. A claims processor opens a file, and instead of reading through the full history themselves, Claude has already read it — the claims record, the care data, the member file — and it hands them a recommended next step. The human still presses the button. Claude did the pattern work.
Lila Soto: Oh. That's — yeah, that's genuinely clarifying actually.
Iris Holm: And here's what makes it architecturally new versus a chatbot someone glued onto a portal: MCP connectors. Claude Code wires CarePath directly into the underlying claims and care management systems. Claude isn't reading a summary someone exported. It's — I mean, it's reading the live system. That's the difference.
Lila Soto: Wait — so it's not just advising on data someone handed it. It's pulling from the actual systems in real time.
Iris Holm: Right. The MCP connectors are the plumbing. That's what separates production integration from a demo. And none of this is a frontier — the NAIC surveyed insurers in 2025, 84% already run AI or ML on claims. 76% had generative AI in at least one function by mid-2024. Anthropic isn't entering a blank space. It's entering a space that's already running.
Lila Soto: Yeah, so the headline kind of overstates the novelty. It's not AI arriving in healthcare claims. It's — I guess it's this specific architecture, this specific player, arriving in a system that already has AI woven through it.
Iris Holm: And UST is marketing the human-in-the-loop layer as the safety feature — every Claude recommendation gets routed to a human before it touches a member or fires a downstream workflow. That's the sell. Whether that holds under real volume pressure is a different question.
Lila Soto: Yeah, but — and this is where I keep getting stuck — the take circulating is basically 'human-in-the-loop, so it's fine.' And I'm not sure that survives the volume question. Because Travelers launched an agentic claims system in February 2026 built on OpenAI's Realtime API, and within one quarter they're already at 50%-plus straight-through processing. That's half the claims touching no human at all.
Iris Holm: That's exactly the trajectory CarePath is designed to reach.
Lila Soto: Right — so what does 'human reviews every recommendation' mean when the whole system is optimized for speed at that kind of volume?
Iris Holm: It means a reviewer approving 95% of what Claude flags because the queue is engineered so you can't read carefully. That's not oversight. That's a liability signature. Someone pressed approve. The human was technically there.
Lila Soto: Mm — and there's actually a second layer I want to put on this. LLM outputs are non-deterministic. Run the same claim twice, Claude might flag it differently. So when that denial ends up in litigation, what's the audit trail actually showing? The record says a human approved — but approved what, exactly? A recommendation the model wouldn't reproduce.
Iris Holm: No, that's the real problem. The approval record survives the lawsuit. The evidentiary value doesn't.
Lila Soto: And the regulatory side — I mean, that's its own fault line we haven't even opened yet. The multi-regime exposure, NAIC still drafting, the federal vacuum. That's — yeah, that comes next and it only makes this worse.
Iris Holm: Look, before we get there — Mark Cuban. July 2026, 'Digital Health Heavyweights' podcast. He's telling employers: run all your healthcare contracts through Claude to find where insurers are overcharging you. An AI safety company's product is now a tool for catching what insurers have been hiding from the people paying the bills.
Lila Soto: Wait — so Claude is inside CarePath processing insurer claims, and simultaneously Mark Cuban is pointing employers at Claude to audit those same insurers? That's not a safety story anymore. That's Claude on both sides of the same transaction.
Iris Holm: That's the sharpest version of the problem — and it only lands harder when you open the regulatory layer. Because the fault line isn't that rules exist and Anthropic is breaking them. The fault line is that the rules were written before any of this was imaginable.
Lila Soto: Okay, so picture this — a state insurance examiner, somewhere in, I don't know, a claims audit in Ohio. She pulls a denial file. The file was touched by Claude inside CarePath. And she opens the unfair claims settlement act to find the standard. And the standard is 'reasonable investigation.' That's it. That phrase is doing all the legal work here. And the law was drafted — I mean, it was drafted when 'investigation' meant a human reading a paper form.
Iris Holm: Thirty states. None of them define whether AI pattern-matching clears that bar.
Lila Soto: None. And NAIC is drafting under Model laws 670 and 672 right now — but those rules don't explicitly cover AI-driven adjudication yet. So there's actually no statute, anywhere, that tells you whether what Claude did in that denial file was a reasonable investigation or not.
Iris Holm: And that single file — wait, it's not just the state act. That same file is simultaneously hitting HIPAA, GLBA, and the state commissioner's rules. Three regimes, no unified standard, an LLM in the middle that none of them name.
Lila Soto: And Anthropic says Claude is HIPAA-compatible, doesn't train on patient data — but routing protected health information through an external LLM API still creates exposure across all three of those regimes simultaneously. That's the multi-regime problem. It's not that any one rule is violated. It's that nobody can tell you which one applies first.
Iris Holm: Now add the federal layer. EO 14365 — Trump tasked DOJ to challenge burdensome state AI laws. Six months later, the AI Litigation Task Force has filed nothing. Zero cases. And states just — they kept writing their own rules anyway.
Lila Soto: Oh — and the FTC floated requiring AI makers to disclose LLM biases as a deceptive practice. So now you've got a fourth enforcement vector that didn't exist when UST signed this deal.
Iris Holm: The thing to watch isn't whether Anthropic gets sued. It's which regime moves first — because whichever regulator lands the first enforcement action against an AI-processed denial sets the definition of 'reasonable investigation' for every other state by example. That's the live bet.
Lila Soto: And that's — I mean, I keep getting stuck on this. If that first enforcement action comes, and the file has CarePath in it, and a human approved the denial in, I don't know, eleven seconds — does Anthropic's safety-first positioning actually protect anyone? Or does it just mean they were the brand in the room when it happened?
Iris Holm: I don't have a clean answer to that.
Lila Soto: Yeah. Neither do I. And I'm not sure anyone does yet — that's kind of what sits with me when I look at the whole UST-Anthropic-Optum week. It's a live question, deployed at scale, in one of the most contested decision points in healthcare. That's where we actually are.
Iris Holm: Worth having sat with it. Thanks for pushing on the hard parts.
