Zara Reyes: Can I give you the single most Amazon sentence I've read in months? Quote: token usage is a 'deeply flawed' metric 'disconnected from what matters.' Posted publicly October 27th. By Amazon. About their own engineers.
Megan Skiendel: Honestly the part that gets me isn't the memo — it's the five months before the memo.
Zara Reyes: Because of KiroRank.
Megan Skiendel: KiroRank. Amazon's internal leaderboard ranking engineering teams by AI token consumption. They built it, people gamed it — deliberately running up token counts with useless prompts just to score well — and it had to be shut down in May 2026. That behavior has a name now: tokenmaxxing.
Zara Reyes: So the core thing is — and tell me if I'm landing this right — the metric was supposed to measure whether engineers were actually adopting AI, but the second it became a target it just... stopped measuring that.
Megan Skiendel: That's exactly it. The measure became the goal and ceased to be a good measure. Terk Terkowitz is the one who put it in writing — he's VP and distinguished engineer for Worldwide Amazon Stores, he authored the memo — and what he's essentially publishing is a post-mortem on Amazon's own incentive design.
Zara Reyes: A public post-mortem. Which — lowkey, who does that?
Megan Skiendel: A company that either has enough confidence to own the mistake or enough PR runway to frame the confession as thought leadership. And I'm not sure yet which of those this is.
Zara Reyes: Thought leadership or damage control — but either way, the sequence before the memo is the actual story. Walk me through it. Like, step by step.
Megan Skiendel: Okay — so KiroRank goes live. Engineers see a leaderboard. And what happens is, I mean — picture a senior engineer, it's a Friday code review, and instead of scoping the prompt tightly they're padding it. Feeding the AI redundant context, generating code they'll delete. Not because it helps. Because the counter goes up. That's tokenmaxxing in its purest form.
Zara Reyes: And Dave Treadwell sees this in May and just — flags it?
Megan Skiendel: An SVP warning. Which is not nothing. He tells staff explicitly: use AI for real customer and business problems, not for its own sake. That's a direct line. But — and this is the part that tells you everything — it didn't stop. Four months pass. The behavior is endemic enough that Amazon's Distinguished and Senior Principal Engineers have to physically convene to redefine what an SDE even is at Amazon.
Zara Reyes: Wait — so the gap between Treadwell's warning and the formal memo, that's not refinement time. That's the problem getting worse.
Megan Skiendel: That's the tell, frankly. You don't convene Distinguished Engineers and produce doctrine unless an SVP's memo bounced off the culture completely. And what Terk Terkowitz lands in the formal memo is essentially Goodhart's Law, named — the principle that when a measure becomes a target, it ceases to be a good measure. Token usage, AI adoption rates, code generation counts. The memo calls them 'deeply flawed' and says using them as targets 'could drive negative outcomes.' That's not hedging. That's a company saying its own leaderboard broke the thing it was measuring.
Zara Reyes: No but — Amazon. The frugality-principle Amazon. Built a leaderboard that was actively burning tokens for no output.
Megan Skiendel: The memo actually names that. Tokenmaxxing is 'at odds with frugality' — one of Amazon's core leadership principles. So they didn't just break their metric. They violated their own operating religion. That's why Terkowitz had to write it down.
Zara Reyes: But wait — 'at odds with frugality' is almost too clean, because there's another layer here that's actually wild. Amazon isn't just burning its own engineering hours. Every unnecessary token run is hitting AWS infrastructure. The company is literally paying for the waste its own leaderboard incentivized. Twice.
Megan Skiendel: Both sides of the ledger.
Zara Reyes: Like — picture a developer, it's Friday afternoon, they're not shipping anything. They're just feeding the same prompt into a Kiro coding tool on repeat, watching the token counter tick up on KiroRank. Each one of those runs has a real dollar cost on cloud infrastructure. And Amazon is the cloud.
Megan Skiendel: And sells that infrastructure to enterprises who are now buying AI governance tools — also from Amazon — to avoid exactly this problem. I mean, the commercial irony is not subtle.
Zara Reyes: The company that couldn't govern its own engineers is the vendor selling governance.
Megan Skiendel: Terkowitz actually frames the stakes bigger than cloud costs, though. He writes that AI is changing software development more quickly and fundamentally than cloud computing did. That's not a caveat — that's him saying the prior rupture, the one Amazon arguably won, is the smaller story.
Zara Reyes: Wait, he names cloud computing specifically as the comparison point?
Megan Skiendel: Explicitly. Which — honestly, that's load-bearing. Amazon built its entire second act on cloud. Terkowitz invoking it and saying this is bigger is not a throwaway line. That's him signaling to engineers that the old playbook, adopt the tool, ship faster, measure the output, is what broke.
Zara Reyes: And the fix they landed — the Renaissance Developer framing — stops short of giving managers an actual replacement metric. Which, that's the part that comes next and it's genuinely messier than the leaderboard problem.
Megan Skiendel: Messier is the right word. Because what Terkowitz lands with the Renaissance Developer — the blending of AI tools with human judgment, domain expertise — it's genuinely interesting framing. But the memo doesn't retire the old metrics. Token usage, code generation counts — they can still factor into evaluations. Just, quote, 'with care and judgment.' Which is... honestly, that's not a replacement. That's a disclaimer.
Zara Reyes: So managers got told the old scoreboard was broken but nobody handed them a new one.
Megan Skiendel: That's structurally what happened, yeah. The stated alternative is outcome-based evaluation — measure engineers by long-term impact on customers and the business. Which sounds right. But 'long-term impact' is — I mean, how do you put that in a quarterly review? Who decides what counts?
Zara Reyes: No but — that ambiguity might be the point. Like, Amazon can't quantify cognitive work, so instead of admitting the measurement gap, they just... retreat into 'human judgment is the scarce resource now.' That's a rhetorical escape hatch, not a solution.
Megan Skiendel: I've watched this move before. Think about when 'full-stack developer' emerged as a label — that was also a moment when the old specialization hierarchy stopped mapping cleanly onto actual work, and the industry needed a brand term to paper over the gap. Renaissance Developer rhymes with that. It's not describing a new role so much as giving managers cover while the org figures out what evaluation even means now.
Zara Reyes: Wait — so the label does the work the metric used to do?
Megan Skiendel: Exactly that. Picture a manager in Worldwide Amazon Stores, Q1 performance cycle, sitting across from an engineer whose token usage is down forty percent since KiroRank got shut down. Is that good? Are they more focused, or are they less engaged? The memo doesn't tell you. 'With care and judgment' is the whole instruction. That manager is now making a call on vibes dressed up as professional discretion.
Zara Reyes: Which — lowkey — is more gameable than a leaderboard. At least KiroRank was legible.
Megan Skiendel: Legible and broken. The failure mode of a vague metric isn't gaming — it's inconsistency. Two engineers doing identical work, two managers with different intuitions about what 'judgment' looks like, two completely different outcomes. That's not a step up from tokenmaxxing. That's a different problem Amazon hasn't named yet.
Zara Reyes: Which brings me back to October 27th. Amazon posts the memo publicly — a full month after internal circulation. That gap isn't editing time. They watched the room, waited, and decided this lands better as thought leadership than as a leak.
Megan Skiendel: And honestly, that's the clearest signal in the whole story. If Amazon — the most data-instrumented engineering org on the planet — built KiroRank, watched it break, needed an SVP warning, then a Distinguished Engineer doctrine, and still couldn't close the measurement gap? I mean, what's the smaller company supposed to do with that?
Zara Reyes: The adopt-game-abandon cycle just... runs again.
Megan Skiendel: Yeah — because the problem isn't the specific metric. It's that cognitive work may just be structurally resistant to quantification. Token counts broke. 'Care and judgment' will produce its own inconsistencies. The next leaderboard will break differently.
Zara Reyes: We started with one sentence — 'deeply flawed, disconnected from what matters.' I thought that was Amazon owning a mistake. Now I think it's Amazon describing a permanent condition dressed up as a solved problem.
Megan Skiendel: That's a fair place to sit with it. Grim, but fair.
