Onpode
Cover art for A new tool makes AI coding agents quiz developers on their own code before deployment

A new tool makes AI coding agents quiz developers on their own code before deployment

October 18, 2026 · 8 min

Iris Holm & Otis Kim

Sphinx, an open-source Git pre-push hook by GitHub user parthh01, blocks developers from shipping AI-generated code until they pass a comprehension quiz about their own diff. But the quiz is open-book — meaning developers can ask the same AI that wrote the code to answer it, making the check more ritual than verification.

Sphinx is an open-source Git pre-push hook created by a developer identified as "parthh01" on Hacker News, where it appeared as a Show HN submission that reportedly reached 156 points within eight hours of posting. The tool intercepts commits to protected branches, reads the code diff, and generates comprehension questions about edge cases and failure modes using a separate AI reviewer model.

0:008:10
Get the next episode on Artificial Intelligence →

Follow it free — new episodes land in your feed.

Or make your own — any topic, in minutes

More Onpode episodes on Artificial Intelligence →

About this episode

Ninety-six percent of developers say they don't fully trust AI-generated code. Only forty-eight percent always verify it before committing. The gap between those two numbers is what a tool called Sphinx is trying to live inside. Sphinx is an open-source Git pre-push hook built by a GitHub user named parthh01. Before you can ship to a protected branch, it reads your diff, sends it to a separate AI reviewer model, and makes you answer a comprehension question about what the code actually does under pressure — edge cases, failure modes, real behavior. Wrong answer gets you a follow-up. Right answer lets you push. The episode pulls on whether that mechanism holds up. The quiz is explicitly open-book: you can consult your AI coding agent while answering. The same agent that wrote the code. Which means the reviewer model and the code author are often drawing from the same systems — and a developer can route the question straight back through the agent, copy-paste the answer, and pass. That's not verification. That's a receipt that you asked the AI a question. Then there's yolo mode: skip the quiz entirely, push anyway, Sphinx logs the skip. No enforcement. Just a trail — one that, when something breaks in production, points at the developer rather than the org that chose the tool. The episode doesn't conclude that Sphinx is useless. Making evasion legible is real. But naming a gap and closing it are different things, and the design doesn't always distinguish between the two.

Frequently asked

What is Sphinx AI coding tool and how does it work?

Sphinx is an open-source Git pre-push hook built by GitHub user parthh01 that intercepts pushes to protected branches and requires developers to answer comprehension questions about AI-generated code in their diff. A separate AI reviewer model generates the questions; correct answers let the push proceed. It installs as the npm package sphinx-understand and requires Node 22.19 or higher.

What percentage of developers verify AI-generated code before committing?

Only 48% of developers always verify AI-generated code before committing, according to Sonar's 2026 survey of over 1,100 developers. Notably, 96% of those same developers say they don't fully trust AI-generated code — meaning the majority who skip verification are not unaware of the risk.

Can you bypass or skip the Sphinx code quiz?

Yes. Sphinx includes a 'yolo mode' that lets developers skip the comprehension quiz entirely and push immediately. The skip is logged, but there is no enforcement mechanism tied to that log. Critics argue this makes yolo mode a liability ledger rather than a genuine safety feature, documenting evasion without preventing it.

Is Sphinx's AI code quiz actually effective at preventing AI-generated code incidents?

Sphinx's quiz is explicitly open-book, allowing developers to consult the AI agent that wrote the code while answering. This means a developer can paste the quiz question into Claude Code or Cursor, copy the answer, and pass — creating a circular check where the AI effectively grades its own output rather than testing developer comprehension.

Who is liable when AI-generated code passes a Sphinx quiz and causes a production incident?

Sphinx's design trails liability to the developer. If a developer skips via yolo mode or passes by routing questions through an AI agent, the log records their choice. No case law yet governs this scenario, but structurally the organization that deployed Sphinx gains documented cover while the individual developer bears the traceable decision.

Grounded in 3 sources
Show HN: Sphinx – a hook that quizzes you on your AI's code before shipping ↗ · news.ycombinator.com
AI News Today, October 18: Top Stories | AI Weekly ↗ · aiweekly.co
Show HN: Sphinx – A hook that quizzes you on your AI's... ↗ · daily.dev
Read transcript

Otis Kim: Iris, tell me — when you see code you didn't write, what's your gut move?

Iris Holm: Read it. Or — look, I try to. Why, what's the number?

Otis Kim: The number is forty-eight percent. That's how many developers always verify AI-generated code before committing, according to Sonar's 2026 survey of over eleven hundred people. The other number — ninety-six percent — is how many of those same developers say they don't fully trust it.

Iris Holm: Wait. So the people not verifying — they're not confused about whether to trust it. They already know they don't.

Otis Kim: That's the thing that doesn't sit right. It's not a knowledge gap, it's a behavior gap. And that's exactly the problem a GitHub user called parthh01 decided to build a tool around — Sphinx. Open-source, installs as a Git pre-push hook, blocks you from shipping to a protected branch until you can answer questions about what the AI-generated code in your diff actually does.

Iris Holm: Hacker News received this how?

Otis Kim: hundred and fifty-six points in eight hours — which sounds like a win, except the reaction was split. Half the comments were basically 'finally, someone's making comprehension non-negotiable.' Other half: 'this is just friction and I'll skip it.' And there is a skip — it's called yolo mode, which is its own thing we should get into.

Iris Holm: The tagline though — 'outsource your thinking, not your understanding.' That's doing work.

Otis Kim: It's doing all the work. That's what we're here to pull on — whether that line survives contact with how Sphinx actually operates.

Iris Holm: So what does that actually look like — at the moment it intercepts?

Otis Kim: Okay, picture this. Someone's shipping code their agent wrote — say, a webhook handler. They run git push. Sphinx stops it. Not with an error, more like — a tap on the shoulder. It reads the diff, sends it to a separate AI reviewer model, and that reviewer generates a comprehension question. Something like: if this webhook retries three times and the database is down the whole time, what does your code do? You answer. Wrong answer — it explains the gap and asks a different question targeting the same failure mode. Right answer — push goes through.

Iris Holm: The reviewer is a separate model — not the one that wrote the code?

Otis Kim: That's the design, yeah. Whether that separation actually changes anything is — I mean, the question's fair, but mechanically yes, the quiz generation is a distinct step. The point is it's blocking a protected branch push until you've demonstrated you know what the code does under pressure.

Iris Holm: That's the click. It's not reviewing the code — it's reviewing the developer.

Otis Kim: Exactly that. And it installs as sphinx-understand, npm package, needs Node 22.19 or higher. Targets the agent workflow specifically — Codex, Claude Code, Cursor, OpenCode, Pi.

Iris Holm: Wait — Pi?

Otis Kim: Yeah, Pi's in there. It's a shorter list than you'd expect for something trying to cover the whole agent ecosystem, actually. But the point isn't breadth — it's that the hook lives at git push, not inside any one tool. Doesn't matter which agent wrote the code. The gate is at the boundary.

Iris Holm: The boundary being the only moment where you can't pretend you didn't have a choice.

Otis Kim: But that boundary is gameable — and that's the part nobody's really sitting with. The quiz is open-book. Explicitly. The docs say you can consult the code or your AI coding agent while you're answering.

Iris Holm: The agent that wrote the code.

Otis Kim: The agent that wrote the code. Picture a developer — it's 11 PM, deployment window for a financial reporting job is closing in twenty minutes. Sphinx fires a question about edge-case retry logic. The developer pastes the question straight into Claude Code, gets the answer, types it back. Quiz passed. Code ships.

Iris Holm: That's not verification. That's — a receipt that you asked the AI a question.

Otis Kim: Which is exactly the Hacker News read. If the book is the AI that wrote the code, the quiz is the AI grading itself. And the intent — I want to be fair here — the intent is behavioral understanding, failure modes, not rote memorization. The open-book format is supposed to be a feature. But the mechanism doesn't actually distinguish between 'I understood this' and 'I routed it through OpenRouter and copy-pasted.'

Iris Holm: So what does yolo mode add to this picture.

Otis Kim: Yolo mode is actually — okay, it's the tell. You hit yolo, you skip the question entirely, push goes through, and Sphinx logs the skip. That's it. No enforcement mechanism. No contractual obligation to act on that log. Someone built a liability ledger and called it a safety feature.

Iris Holm: It documents that you chose not to understand the code.

Otis Kim: And that log sitting there — who it belongs to when something breaks in production — that's a question we should probably get to, because the answer is messier than parthh01's README suggests.

Iris Holm: That log pointing at the developer — that's the part I want to pull on. Because parthh01 is one GitHub username. No verified team, no institutional backing, nothing confirmed beyond the repository. Who owns the liability architecture here?

Otis Kim: Yeah, and that's — actually, that's where the design gets strange. Because the org deploys Sphinx, the agent writes the code, the developer clicks yolo, and the log exists. When production breaks, that trail ends at the developer. Not the tool. Not the org that chose the tool.

Iris Holm: The org gets cover by deploying it.

Otis Kim: That's the move. And I want to be careful here because we're inferring from design, not case law — there's no litigation on this, it's too new. But structurally, if you're a CTO and something breaks and your log shows the developer skipped the Sphinx quiz via yolo mode, that's a very convenient document to have.

Iris Holm: And the quiz they skipped — it was generated by a separate AI reviewer, potentially running through OpenRouter. Which the developer could have used to answer it anyway.

Otis Kim: The independence of that check has completely collapsed. Sphinx uses OpenAI Codex, Claude Code, Cursor — the same systems — as supported backends for the reviewer model. So the circle is: agent writes code, reviewer model questions it, developer asks agent to answer, developer passes. That's not a check. That's a ritual.

Iris Holm: That's the Hacker News split, isn't it. Half the thread is — finally, comprehension is non-negotiable. Other half is looking at exactly this circle.

Otis Kim: And neither side is wrong, which is the uncomfortable part. The supporters aren't naive — forcing even a procedural stop has some value. The skeptics aren't lazy — they're pointing at a real circularity that the design doesn't solve. I could be wrong, but I think the split tracks something genuine about what people want from accountability tools versus what those tools can actually deliver.

Iris Holm: The failure mode is baked in before anyone writes a line of case law about it.

Otis Kim: Which is almost the answer to the question you opened with. When you see code you didn't write, what's your gut move. Forty-eight percent of developers — their gut move is nothing. And Sphinx doesn't change that gut. It just makes the nothing visible.

Iris Holm: The tagline is genuinely right, actually. 'Outsource your thinking, not your understanding.' That's the principle. A tool with a yolo button can't enforce a principle.

Otis Kim: No. It can name it — I mean, that's not nothing. Naming where the gap is, logging who skipped it, making the evasion legible. That's real. It's just not the same as closing it. I could be wrong, but I think parthh01 might know that.

A new tool makes AI coding agents quiz developers on their own code before deployment · Onpode