{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# System Prompts Don’t Get Shorter. They Shift.\n", "\n", "Claude Opus, Fable, and the harness layer around the model" ], "id": "774af2a2-1ba5-41b4-b3be-0cb7bebc9ab1" }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/html" }, "source": [ "" ], "id": "ebd36df2-cd29-4581-8dab-0f7a3b8fadaf" }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"Bar\n", "\n", "I keep hearing the model labs tell a story that as models get better,\n", "you need to give the model less direction, and the coding agent\n", "harnesses around them should get simpler.\n", "\n", "I like that story, it sounds so intuitive. But like a good engineer, I\n", "also do not quite trust it.\n", "\n", "So I pulled together a small visual experiment around Claude’s published\n", "system prompts. The original version looked at Opus 4 through Opus 4.7.\n", "I have now extended it to the latest Opus row I could verify, **Claude\n", "Opus 4.8**, plus **Claude Fable 5**.\n", "\n", "The short version: some prompt complexity really does\n", "move into training. But the total system does not simply get smaller.\n", "Complexity moves into safety policy, tool discovery, product context,\n", "memory, skills, and app-specific harnesses.\n", "\n", "## The interactive\n", "\n", "The chart below is the experiment. Click a model tab to see the category\n", "notes, or switch the chart from absolute words to percent share.\n", "\n", "\n", "\n", "## What changed\n", "\n", " Opus 4, May 2025\n", " 1,714\n", "

words in the first Claude 4 prompt snapshot in this comparison.

\n", "\n", " Opus 4.7, Apr 2026\n", " 3,686\n", "

words after safety, agentic behavior, and product scaffolding expanded.

\n", "\n", " Opus 4.8, May 2026\n", " 2,888\n", "

words in the latest Opus prompt body extracted from Anthropic's docs.

\n", "\n", "The most interesting update is that the curve is no longer going up. For\n", "a while, Opus system prompts grew from 1,714 words to 3,686 words in\n", "less than a year. The newer Opus 4.8 and Fable 5 are using smaller\n", "system prompts. But digging into the content shows it has shifted.\n", "\n", "## The pieces that moved\n", "\n", "The behavior patches are the easiest part to understand. Older prompts\n", "carried little runtime hacks: count letters carefully, restate puzzle\n", "constraints, do not over-apologize, avoid certain linguistic tics. Those\n", "are exactly the sort of instructions you would expect to move into\n", "training as the model gets better.\n", "\n", "The structural parts are stickier. Safety instructions remain large.\n", "Product and model identity keep changing as the product surface changes.\n", "Tool discovery becomes explicit in Opus 4.8: before Claude says it\n", "cannot do something, it is told to check for deferred tools, personal\n", "context, and skill files.\n", "\n", "This is Anthropic making product decisions. They are deciding what\n", "context and capabilities are visible, when to reveal them, and how\n", "Claude should behave when something might exist outside the visible\n", "prompt.\n", "\n", "## Why this matters\n", "\n", "If you are building agents, you should not take the model labs at face\n", "value. You should measure what is actually changing.\n", "\n", "Here we found that despite the notion of models needing less guidance,\n", "the labs are still using a lot of guidance. It just shows up\n", "differently.\n", "\n", "## Want to get hands-on?\n", "\n", "If this framing is useful, the next step is to change a harness yourself\n", "and watch the trace move.\n", "\n", "- Start with my hands-on course, [Learn Harness Engineering with\n", " OpenHands](https://rajshah4.github.io/learn-openhands-harness/). It\n", " walks through Agent Server and Agent Canvas, then turns model\n", " routing, retrieval, memory, security, critic loops, and goal\n", " scaffolding into runnable projects.\n", "- The source repo is\n", " [rajshah4/learn-openhands-harness](https://github.com/rajshah4/learn-openhands-harness),\n", " if you want to fork the exercises.\n", "- The broader companion repo is\n", " [rajshah4/harness-engineering](https://github.com/rajshah4/harness-engineering),\n", " which collects the prompt-evolution experiment, references, and\n", " other small harness investigations.\n", "- For the conceptual frame, the annotated talk is here: [Harness\n", " Engineering: Why the System Around the Model Decides Agent\n", " Performance](https://rajivshah.com/blog/harness-engineering.html).\n", "\n", "## Data notes\n", "\n", "- Opus 4 through Opus 4.7 use Simon Willison’s\n", " [`simonw/research`](https://github.com/simonw/research/tree/main/extract-system-prompts)\n", " mirrors of Anthropic’s published system prompts.\n", "- Opus 4.8 and Fable 5 were extracted from Anthropic’s [published\n", " system prompt\n", " page](https://platform.claude.com/docs/en/release-notes/system-prompts),\n", " with model context checked against Anthropic’s [model\n", " overview](https://platform.claude.com/docs/en/about-claude/models/overview)\n", " and [Opus 4.8\n", " docs](https://platform.claude.com/docs/en/about-claude/models/whats-new-claude-4-8).\n", "- Word counts are approximate and categorized by primary function." ], "id": "93fbbc60-6afa-4c59-a2a8-ccbb191e23bf" } ], "nbformat": 4, "nbformat_minor": 5, "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } } }