Simon Willison's writeup of a fireside chat with Cat Wu and Thariq Shihipar from Anthropic's Claude Code team is less about a product announcement and more about how the team building an AI coding tool actually uses it on itself. Three details stand out: an internal Slack integration now lands 65% of the team's product engineering pull requests, the tool's own system prompt has shrunk by 80%, and feature releases are gated on retention data rather than launch buzz. Together they sketch what building for a model that improves quickly actually looks like in practice.
This article covers why a shrinking system prompt is a signal about model capability rather than a cost-cutting move, what it means that most PRs now land through a chat interface instead of a traditional editor flow, and why retention — not adoption — is the metric driving what ships.
A shrinking system prompt is a capability signal, not a cleanup
A system prompt is the standing set of instructions given to a model before any user request — the rules, examples, and constraints that shape its behavior. Anthropic's team reports cutting Claude Code's system prompt by 80%, attributing it to newer models like Fable needing fewer examples and less explicit constraint to behave correctly.
That's a meaningfully different story than "we simplified our prompt for maintainability." A system prompt is scaffolding for a model's weaknesses — the more explicit guidance a task needs, the more the model needed hand-holding to do it reliably. Cutting scaffolding by 80% because the model no longer needs it is a direct, measurable readout of how much more capable the underlying model has become, not a matter of engineering taste.
The system prompt is a report card, written for the previous model's weaknesses. Shrinking it is the newer model turning in a shorter list of things it still can't do on its own.
65% of PRs landing through chat changes what "using the tool" means
Claude Tag — Anthropic's Slack integration for Claude — now lands 65% of the Claude Code team's own product engineering pull requests. That's not a minor internal efficiency stat; it means the majority of code changes at the company building the tool start as a conversation in a chat channel rather than an editor session.
This matters for anyone evaluating AI coding tools because it shifts the unit of work. If most PRs originate in chat, the tool's chat interface — not its IDE integration — is carrying the bulk of real engineering throughput. Teams evaluating "does this tool fit into how we work" should weigh the chat-to-code path as seriously as the editor-integrated path, since the team that built the tool is routing most of its own work through exactly that path.
Automated review changes what "non-critical" means
The team relies heavily on automated code review for non-critical changes, freeing human reviewers to focus on changes that carry more risk. This is a triage decision as much as an automation one — someone has to define what counts as "non-critical" reliably enough that an automated reviewer can be trusted with it unsupervised.
Get that triage boundary wrong in either direction and it costs you: too conservative, and automation barely reduces reviewer load; too permissive, and risky changes slip through under a category that was supposed to exclude them. The team's willingness to lean on this split suggests they've validated the boundary against real incident data, not just intuition — a detail worth probing before importing the same pattern into a different codebase with a different risk profile.
Retention, not launch metrics, decides what ships
Perhaps the most transferable detail: the team uses internal retention metrics, not adoption or launch-day numbers, to decide which features actually ship broadly. A feature that gets tried once and abandoned reads very differently in retention data than one that gets used repeatedly, even if both generate similar initial adoption numbers.
This connects to a broader theme in harness engineering — the discipline of building the tooling and constraints around a model rather than the model itself. Deciding what to ship based on whether people keep coming back, rather than whether they show up once, is a harness-engineering decision as much as a product one: it treats the tool's usefulness as something to be measured continuously, not declared at launch.
What this means for builders
If you're building AI-assisted developer tooling, treat your system prompt's size as a metric to track over model upgrades — a shrinking prompt on a newer model is a signal you can act on, not just tidy up. Separately, audit where your own team's real work happens (chat, IDE, CLI) rather than assuming the interface you designed first is the one people actually use, and consider retention over adoption as the gate for promoting a feature from experimental to default.
Conclusion
The most useful part of this fireside chat isn't a specific feature reveal — it's a look at how the team building an AI coding tool measures its own tool's fitness: shrinking scaffolding as a capability signal, chat as a majority code-delivery path, automated review scoped to a validated risk boundary, and retention as the actual ship gate. Those are process decisions any team building on top of fast-moving models can borrow directly.