http://akjamie.github.io/tags/context-compression/ Recent content in Context Compression on Jamie's Blog Hugo en-us Sun, 24 May 2026 00:00:00 +0000 http://akjamie.github.io/post/2026-05-24-context-compressor-deep-dive/ Sun, 24 May 2026 00:00:00 +0000 http://akjamie.github.io/post/2026-05-24-context-compressor-deep-dive/ <h1 id="designing-context-compression-for-production-agents-a-deep-dive-into-hermes">Designing Context Compression for Production Agents: A Deep Dive into Hermes</h1> <blockquote> <p>Staff-engineer-level notes on <code>agent/context_compressor.py</code>: how Hermes preserves task continuity when a long-running agent outgrows the model context window, and what the implementation teaches about summarization, compression, and failure-tolerant agent design.</p> </blockquote> <hr> <blockquote> <p>[!NOTE]</p> <h3 id="executive-tldr">Executive TL;DR</h3> <p>Hermes context compression is not &ldquo;summarize the chat when it gets long.&rdquo; It is a transcript rewrite algorithm with strict invariants:</p> <ul> <li><strong>Head / middle / tail partitioning:</strong> keep the system prompt and first turns intact, summarize the middle, and protect the recent tail by token budget.</li> <li><strong>Active task anchoring:</strong> the latest user message must stay outside the summary. A summarized &ldquo;pending ask&rdquo; is reference material, not a live user turn.</li> <li><strong>Tool-aware compaction:</strong> old tool outputs are deduplicated, summarized, and pruned before any LLM call; tool call/result pairs are sanitized afterward so providers never receive invalid message history.</li> <li><strong>Iterative summaries:</strong> second and later compactions update the existing handoff instead of recursively summarizing summaries as ordinary turns.</li> <li><strong>Multimodal budgeting:</strong> images are charged a fixed token estimate so image sessions do not accidentally preserve far more context than the model can fit.</li> <li><strong>Failure visibility:</strong> if the summary model fails, Hermes inserts an explicit fallback marker and records dropped-turn metadata instead of silently losing context.</li> </ul> </blockquote> <hr> <h2 id="how-to-use-this-deep-dive">How to Use This Deep Dive</h2> <p>Read this document in four passes:</p>