When to reach for it
- User context must be maintained across multiple turns.
- References to prior statements are expected.
- Conversational flow is critical.
Conversational Memory.
The running message history the agent reads each turn.
Recent conversation turns are retained and replayed so the model has context for the next turn — as a full transcript, a fixed-size window of the last N turns, or a running summary once the raw history outgrows the budget. This is the short-term tier of agent memory.
In practiceA travel-planning chatbot replays the last 20 turns at the start of each response so the model remembers the user said 'window seat only' three messages ago.
When it backfires
- Privacy regulations forbid storage.
- History consumes unnecessary tokens.
- Tasks are strictly stateless.
The tradeoff
Improved conversational context is gained against higher token costs, privacy risks, and context window exhaustion.
What it actually does.
Each turn is appended and replayed as context.
Two ways this pattern will hurt you.
Unbounded history exhausts the context window
Every turn appends to the buffer; eventually the prompt is mostly stale dialogue and the model truncates the most recent — and most relevant — turns.
Fix · Cap the buffer by token budget. Summarise older turns into a single block, or fall back to retrieval against an episodic store.
Summarisation silently drops the detail you needed
To stay under budget, older turns are compressed into a summary. The one concrete fact a later turn depends on — a booking code, an exact figure — is exactly what the summary paraphrases away.
Fix · Extract and pin durable facts (IDs, preferences, commitments) to a structured slot before summarising the prose, so compaction never touches them.
Where Conversational Memory is native.
Microsoft Agent Frameworkthread historyNative
Keep going