Compaction on demand: summarizing a conversation when you decide, not when the API does

Every long-running Claude conversation eventually hits the same wall: the history grows until it no longer fits in the context window, or until it is so long that every request costs more than it should. The standard fix is compaction — replace the old part of the conversation with a summary and carry on.

Since February the API has offered one way to do that: threshold compaction. You set a token threshold in context_management, and when a request crosses it, the API writes a summary partway through that request. It works, but the API decides when it happens, and the request that trips the threshold pauses while the summary gets written.

On September 14, 2026 Anthropic added a second way. With the compact-2026-09-04 beta header you send a top-level compaction parameter on a request of your choosing. The API returns a single signed summary block and no reply. You then put that block at the front of your history in place of the messages it summarizes.

What this gives you that threshold compaction does not

1. You pick the moment. Compact at the end of a task phase, when a user goes idle, or before a known-expensive step — whatever your application knows that the API does not.
2. It can run in the background. The summary request is separate from your conversation turns. Your agent keeps working on the full history while the summary is being written, and you swap the block in when it arrives. Nothing pauses.
3. You can keep recent turns word for word. Leave the last few turns out of the compaction request and put them after the block. Claude sees a summary of the old material and the exact text of the recent material.

The third point matters more than it sounds for agents. On models with preserved thinking, the thinking blocks in those kept turns can stay valid after the swap, so a long-running agent keeps its train of thought instead of starting its reasoning over from a summary.

Rule of thumb: use threshold compaction when you are happy for the API to manage context inside ordinary requests. Use the compaction parameter when your app needs to control timing, cannot pause for a summary, or must keep recent turns intact.

Which models support it

Claude API only (not Bedrock or Vertex) on Fable 5.1, Mythos 5.1, Fable 5, Mythos 5, Mythos Preview, Opus 5, Opus 4.8, Opus 4.7, Opus 4.6, Sonnet 5, and Sonnet 4.6. You can confirm per model by calling the Models API with the beta header and reading capabilities.compaction.

Step 1: request a summary

Send the conversation as it stands, plus "compaction": {"type": "summarize"}. Send the same system prompt and tools you use for the rest of the conversation — the summarizer reads them, and kept thinking only stays valid if they match.

import anthropic

client = anthropic.Anthropic()

SYSTEM = "You are a data-modeling assistant for a recipe app."

history = [
    {"role": "user", "content": "I am building a recipe app. Help me name the main entities in the data model."},
    {"role": "assistant", "content": "Start with Recipe, Ingredient, and Step. Add a RecipeIngredient entry that holds the quantity and unit."},
    {"role": "user", "content": "Good. Now suggest field names for Recipe."},
]

summary = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=8000,          # caps the summary AND any thinking before it
    system=SYSTEM,
    messages=history,
    betas=["compact-2026-09-04"],
    extra_body={"compaction": {"type": "summarize"}},
)

print(summary.stop_reason)   # "compaction" when it worked

The response has one content block of type compaction with a content string and a signature, and stop_reason is "compaction". Top-level input_tokens and output_tokens are zero because no reply was generated; the real cost shows up in usage.iterations as an entry of type compaction. It is billed and rate-limited like any other request.

A few things the API will reject on this call:

• A last assistant turn that ends in a tool call with no result yet. Send the tool result first.
• stop_sequences, structured-output output_config.format, or tool_choice of type any or tool.
• context_management on the same request. You cannot mix the two compaction styles in one call.

The conversation still has to fit in the model's context window, so compact before you outgrow it.

Step 2: swap the block in

Replace the messages you sent with the returned assistant message. Keep the block exactly as returned, signature included, and send it first on every later request with the beta header.

The background case is where people get this wrong. If your agent took more turns while the summary was being written, drop exactly the messages you sent in the compaction request and keep everything appended since:

sent_count = len(history_at_compaction_time)

if summary.stop_reason == "compaction":
    block_message = {"role": "assistant", "content": summary.content}
    history = [block_message] + history[sent_count:]
# otherwise: keep the full history and try again later

Two assistant messages in a row after the block are fine. Three rules the API enforces:

Rule	What happens if you break it
The block goes first in messages	Summarized messages left in front of it return a 400 (compaction_block_misplaced)
Remove the summarized messages	Left after the block, they are not rejected — they are sent to the model again and you pay for them twice
Exactly one block, on every later request	A request without the block reaches Claude without the summary

To compact again later, send compaction on a history that already starts with a block. The new block summarizes the old summary plus everything after it. From then on, send only the newest block.

Keeping thinking valid in the kept turns

On models with preserved thinking, thinking blocks in the turns after the summary stay valid when both of these hold:

• The kept turns directly followed the summarized messages (the easiest way: compact exactly the messages of a request you already made).
• system and the tools not marked defer_loading: true are unchanged from the compaction request.

Do not edit history between sending the compaction request and making the swap, and swap on the first request after the block arrives. If you need to change the system prompt or tools, compact the whole conversation first (no kept turns), then change them on the next request.

Custom summaries

Pass instructions (up to 16,384 characters) to replace the default summarization prompt entirely:

{
  "compaction": {
    "type": "summarize",
    "instructions": "Summarize this support conversation. Keep the customer's account ID, every order number mentioned, what has already been tried, and the open question. Do not call tools; respond with the summary text only."
  }
}

Always tell the model not to call tools. The summarizer reads tool definitions and the earlier thinking; if it calls a tool instead of writing text, you get no summary.

The five ways it returns nothing

A failed summary is still an HTTP 200 with empty content, and it is still billed. Check stop_reason every time:

stop_reason	Meaning	Fix
max_tokens	Summary got cut off	Raise max_tokens — allow several thousand
model_context_window_exceeded	No room for the summarization prompt	Shorter instructions or fewer messages
tool_use	Model called a tool instead	Add "do not call tools" to instructions
refusal	Declined under normal safeguards	Check stop_details
end_turn	No text returned	Retry or continue without a summary

A transient server problem returns a retryable 529 with error.details.error_code of compaction_unavailable. And if you forget the beta header, the error is a generic compaction: Extra inputs are not permitted that never mentions the header — worth knowing before you spend twenty minutes on it.

Things the summary drops

Images, documents, container_upload blocks, and fetched URLs inside the summarized range are gone once the block replaces them. Mid-conversation role: "system" messages in that range get summarized and stop applying. If a later turn still needs any of it, restate or re-upload it after your next user turn.

Also: cache_control on the block places a prompt cache breakpoint right after the summary, which is usually where you want one. Token counting ignores the compaction parameter. And a task budget's remaining value cannot be sent with compaction or on requests carrying the block.

When to reach for it

• Chat products with long sessions: compact in the background after every N turns, keep the last 4–6 turns verbatim, and users never notice a pause.
• Agents on your own loop: compact at phase boundaries (research done, now implementing) so the summary captures a finished chunk of work. If you run on Managed Agents, Anthropic handles context for you and this parameter does not apply.
• Cost control: a long history resent on every turn is often the largest line on the bill. See cost optimization for the rest of that picture.

Sources

• Compaction docs — compact on demand
• Claude Platform release notes