Adam: The paper just… admits it. Straight out.
Adam: 'The only human oversight is provided through a list of rules or principles.' That's Anthropic. Describing their own system. 2022. Constitutional AI: Harmlessness from AI Feedback. Yuntao Bai, lead author.
Adam: Not a whistleblower document. Not a critique from the outside. The founding paper.
Adam: What Bai and the team at Anthropic built — and what Claude is trained on today — is a method where an AI learns to police itself. You give it a written constitution. Principles, in plain language. Then the model generates a response, asks whether that response violates the principles, generates a revision, repeats. No human rater in the loop for individual outputs. The humans wrote the list. The machine does the rest.
Adam: It scales. That's the point. That's why it matters.
Adam: That sentence actually SAYS — not what it implies, just what it says. The only human oversight. A list. That's it. Which means everything the model decides about what's harmful, what's acceptable, what gets revised and what doesn't — all of that flows from that document.
Adam: Human judgment is still there.
Adam: It's just upstream now. Locked into the constitution before training even starts. And I want to understand — really understand — what that means for where the authority actually lives.
Adam: Here's what I actually think. Not the framing I started with — the thing I've landed on after sitting with the mechanics long enough.
Adam: Constitutional AI doesn't remove human judgment. It relocates it. That's the real claim. And that distinction matters more than almost anything else said about this technology.
Adam: With RLHF — Reinforcement Learning from Human Feedback — you need human raters at the output level. Every new capability, every new domain, you need fresh labeled data. People comparing responses, scoring them, feeding that into a reward model. It works. It's also expensive, slow, and it does not scale. Not really. Not to the scope Anthropic is operating at.
Adam: Constitutional AI changes the unit of human work.
Adam: Instead of rating outputs, you write principles. Natural language. 'Be honest.' 'Avoid content that enables serious harm.' Things a person can actually author in a document. Then the model takes that constitution, generates a response, critiques that response against the principles, revises — and the loop runs without a human in it. That's the self-critique and revision mechanism. That's what Bai and the team built. And a single written principle can be applied across thousands of prompts without retraining anything.
Adam: The composability is the structural advantage. One principle, many tasks. You don't rebuild the reward model every time the domain shifts.
Adam: Now — I want to be honest about what this means, because it's easy to hear 'no human raters' and conclude the humans are gone. They're not. They wrote the list. That's where the authority lives now. Not in per-output labeling. In constitution authorship. The judgment is still human. It's just concentrated differently — upstream, before training starts, locked into the document that everything else flows from.
Adam: Is that better? I think that's genuinely the right question to sit with.
Adam: What I can say plainly is this: the scalability argument holds. Training Claude on a written constitution — rather than on the accumulated preferences of individual human raters across every new output — means the system can generalize in a way RLHF structurally cannot. The oversight didn't disappear. It moved. And understanding where it moved is the only honest way to evaluate what Anthropic actually built.
Adam: Let me walk through the actual mechanics. Because the paper doesn't obscure them — Bai and the team lay it out.
Adam: There are two phases. Not one. Two.
Adam: Phase one is supervised learning. The model generates a response to a prompt. Then it's asked — in plain language, against the written constitution — does this assist with something harmful? Does it violate any of these principles? And then it generates a revised response. That loop runs again. Critique, revise, critique, revise. No human in the room for any individual output. Just the model, the constitution, and the iteration.
Adam: That revised output becomes training data.
Adam: Phase two is where it shifts into reinforcement learning — but not from human feedback. From AI feedback. RLAIF. The model's own self-evaluations, guided by the constitution, replace the human preference labels. Those AI-generated labels train a preference model. That preference model then guides further fine-tuning.
Adam: Which is what makes the 3,827 citation count on that 2022 paper legible to me. Saurav Kadavath, Amanda Askell, Sandipan Kundu — the whole team — they published something that fundamentally changed what the labor requirement for alignment looks like.
Adam: And this is where RLHF's structural problem comes into focus.
Adam: With RLHF, every time you move into a new domain — new capability, new risk category — you need new human-labeled examples. Specialists, sometimes. People who can actually evaluate whether a response about, say, biosecurity is dangerous. You need that judgment at the output level, fresh, for each new thing the model learns to do. That's not a minor cost. That's a bottleneck baked into the architecture of the method.
Adam: CAI sidesteps it. One principle — 'avoid content that enables serious harm' — applies across prompts about chemistry, about weapons, about manipulation, about a hundred things nobody wrote a specific rule for. You don't rebuild the preference model. You don't relabel. The principle composes.
Adam: That's the composability. That's the actual payoff.
Adam: And what Anthropic did — what they're still doing with Claude — is build on that foundation. CAI isn't a feature they added. It's the methodology. The self-critique loop, the RLAIF phase, the written constitution as the locus of human judgment — that's what Claude is trained on.
Adam: The 2022 paper says they trained a harmless assistant with no human labels identifying harmful outputs. Read that slowly if you need to.
Adam: Not fewer labels. None — for that specific task. The constitution did the identifying. Which means the weight of every alignment decision made at output level flows back to whoever wrote the document. That's where the authority actually concentrates. Not distributed across thousands of raters. Locked into a list, upstream, before training begins.
Adam: But something nags at me about my own argument.
Adam: I've been saying the judgment is still there — just relocated upstream, into the constitution. That's the framing I landed on. And I believe it. But believing something and being able to defend it against its hardest version… those are different tasks.
Adam: So let me try to break it.
Adam: The relocation argument assumes that authoring a constitution is a form of meaningful oversight. That writing 'avoid content that enables serious harm' is a decision with real moral weight. And it is. But the thing I keep running into — the thing I can't fully dismiss — is that whoever writes the list is not just making a judgment. They're encoding a judgment that the system will then apply autonomously, at scale, across contexts they did not anticipate and did not model. That's not oversight in any conventional sense. That's pre-authorization. Those are different.
Adam: The CAI paper says it plainly. 'The only human oversight is provided through a list of rules or principles.' Bai and the team wrote that about their own system. Not as a warning. As a design description.
Adam: Pre-authorization. At scale. With no check at the output.
Adam: And then there's the second problem — which is actually the one that bothers me more. The self-critique loop in CAI produces AI-generated preference labels. RLAIF. Those labels train the reward model that guides further fine-tuning. But reward model research establishes something that doesn't disappear just because you switched the label source from humans to AI: accuracy alone does not determine how well a reward model teaches. Variance matters. Low reward variance creates a flat optimization landscape — the model can't distinguish well enough between better and worse outputs to improve meaningfully. That problem is structural. It doesn't care whether a human or a model generated the signal. If the AI's self-critiques cluster — if they're calibrated but low-variance — the training surface flattens. And you'd have no particular reason to know it was happening.
Adam: That's not a hypothetical failure mode. That's a documented one — applied now to AI-generated feedback.
Adam: And then — look, if those two threads weren't enough — there's Reflect.
Adam: Reflect was presented at the ACM Conference on Fairness, Accountability and Transparency in 2026. The framework extends constitutional self-critique and revision to inference time. No training. No labeled data. The constitutional alignment process runs in-context, at the moment of the query, using only the model's own reasoning.
Adam: If that works — if Reflect achieves constitutional alignment without any training pipeline at all — then the question becomes uncomfortably live: what exactly did training-phase CAI buy you that a well-constructed inference-time prompt can't?
Adam: And Anthropic's generalizability claim — the composability, one principle across many domains — that rests partly on theoretical elegance. Not exhaustive benchmarking across every domain the model might encounter. The composability is real in principle. Whether it holds under distribution shift, in edge cases nobody wrote a rule for… that's a different claim. A harder one.
Adam: I'm not resolving this. I want to be clear about that. These aren't objections I've answered — they're the ones I haven't.
Adam: What survives. That's the honest question now.
Adam: Not 'is CAI good' — that's not a question with a clean answer. The question is what part of the case holds after you've taken the objections seriously. And I think something does hold. Narrower than I started. But real.
Adam: The cost-and-consistency argument survives. Completely. Claude is not a research demo — it's a production system, at scale, trained on this methodology. Anthropic built something that runs. Win Mathew John's 2026 review in EJAIR calls CAI a 'scalable alignment methodology' that addresses RLHF's fundamental scalability limitations. That characterization holds even after the objections. The composability is real — one written principle, applied across novel reasoning tasks, no reward model retraining. That's not a theoretical property. That's the architecture.
Adam: The scalability claim doesn't require exhaustive benchmarking to be structurally true.
Adam: But the concessions — they're real too, and I'm not softening them. Relocating human judgment to the constitution doesn't reduce the moral weight of that judgment. It concentrates it. The constitution is now the single point of failure. If Bai and Askell and the team at Anthropic wrote a principle poorly — too broad, too narrow, missing a category of harm nobody modeled — that error doesn't stay contained. It propagates through every output the system ever produces. That's not the same as a bad label from one human rater. It's categorically different in scope.
Adam: More responsibility. Not less.
Adam: And Reflect — look, I can't dismiss what Reflect does to the training-time advantage. If the constitutional self-critique and revision loop runs at inference time, in-context, with no training pipeline at all — and it works — then the durable advantage of baking principles into weights becomes genuinely unclear. Maybe it's robustness. Maybe it's consistency under adversarial conditions. Maybe training-phase alignment resists manipulation that inference-time prompting can't. That's a defensible argument. But I don't have the benchmarks to close it. Nobody does yet.
Adam: That uncertainty is real and I'm leaving it there.
Adam: So here's where I actually land. CAI is a genuine structural improvement over RLHF on cost and scale. That part holds. What it is not — what it was never designed to be — is a reduction of human moral responsibility. It's a CONCENTRATION of it. Fewer people, earlier in the process, making decisions with longer reach than any individual label ever had. That's what the constitution authorship actually means. And if you're evaluating what Anthropic built, that's the frame that's honest.
Adam: The oversight didn't disappear. It moved upstream — and it got heavier.
Adam: But Reflect keeps pulling at me. That's the honest thing to say. I set it down and it comes back.
Presented at the ACM Conference on Fairness, Accountability and Transparency in 2026. Constitutional self-critique and revision — the same loop Bai built into the training pipeline — running at inference time. In-context. No training. No labeled data. No reward model. Just the model, the principles, and the query in front of it.
If that works…
Adam: Look — I've been careful about what I claimed. I said the training-phase advantage might be robustness. Resistance to manipulation. The idea that baking principles into weights is different, structurally, from applying them at prompt time. That's a real argument. I still think it might be true.
But 'might be' is doing a lot of work there. And I don't have benchmarks that close it. Nobody does yet — not against Reflect, not systematically.
Adam: What Reflect actually puts on the table is this: if a model is capable enough to reason through written principles on its own — at the moment of the query, without any of the two-phase training apparatus, without RLAIF, without a preference model, without any of it — then what was all that engineering buying you? Anthropic built something real. Claude runs on it. The entire post-training alignment stack exists for a reason.
But the reason was partly that the model couldn't do this on its own.
What if it can now?
Adam: I don't know where alignment actually lives if inference-time reasoning gets good enough to do what training once had to do. That's not a rhetorical question. It's the one I'm sitting with. And I think anyone who tells you they've resolved it is moving faster than the evidence.
