When to reach for it
- Conversations or tasks outgrow the context window.
- Salient facts must survive across very long horizons.
- Which information is salient changes over the run.
Virtual Context Management.
The agent as its own memory manager: page facts in and out like an OS.
The agent treats its finite context window like RAM and an external store like disk, paging information between the tiers under its own control so it can operate over histories far larger than the window.
In practiceA legal-research agent managing a 200-page contract pages out early clause summaries to an external store and pages them back in only when a downstream question references that section.
When it backfires
- The task fits comfortably in-context.
- A plain truncation or summary buffer suffices.
- The extra LLM calls to manage paging are not justified.
The tradeoff
Effectively unbounded memory under a fixed window is gained against extra inference calls to manage the paging and the risk of evicting something the agent later needs.
What it actually does.
Content is paged in and out to fit the context window.
Two ways this pattern will hurt you.
The agent evicts something it later needs
Self-editing memory means the agent decides what to drop. Under token pressure it can page out a fact that becomes critical several steps later, then confabulate a replacement.
Fix · Pin invariant facts (task goal, constraints, identity) to a non-evictable region and only let the pager touch the discretionary tier.
Re-summarising on every page-out drifts from the source
Each eviction compresses already-compressed content. After many cycles the paged-back summary is a summary of a summary, and details mutate or invert relative to what was originally said.
Fix · Page the verbatim content out to the store and summarise only the resident working view; recall the original on demand rather than re-compressing repeatedly.
Where Virtual Context Management is native.
LettaMemGPT lineage; OS-style tiered memory.Native
Mem0Managed memory layer: fact extraction, summarisation, eviction.Native
ZepLong-term memory store with progressive summarisation.Native
LangChain / LlamaIndexBuffer / summary / entity memory modules.Native
Keep going