When to reach for it
- The agent must utilise domain-specific knowledge long-term.
- Facts and preferences are reused.
- Relationships between entities are crucial.
Semantic / Vector / Graph Memory.
What the agent knows, indexed for retrieval.
Long-term knowledge is stored structurally, as embeddings, or as entities and relations, independently of single conversations. Match the store to the access pattern and start simple: vector DBs for similarity recall, key-value (Redis) for fast exact lookup, relational for structured/auditable querying, and graph DBs only once vector-plus-relational becomes the bottleneck.
In practiceA support assistant stores the fact that a customer is on the Enterprise plan, and recalls it in a conversation weeks later without the user repeating it.
When it backfires
- Knowledge updates too rapidly.
- Governance for storage content is lacking.
- Simple prompt context is sufficient.
The tradeoff
Reusable, highly structured knowledge is gained against significant updating, modelling, and ranking challenges.
What it actually does.
Durable facts outlive any single run.
Two ways this pattern will hurt you.
Retrieval returns relevant-looking but stale facts
Embeddings score by surface similarity. A high-similarity match can be a year-old answer to a question whose ground truth has since changed.
Fix · Include a freshness signal in ranking and reject documents past a recency cutoff. Re-embed when the source updates, not on a schedule.
Duplicate and contradictory facts accumulate
Each write appends. The store ends up holding both 'the customer is on Free' and 'the customer is on Enterprise', and retrieval may surface either, so the agent contradicts itself between sessions.
Fix · Upsert facts by a stable entity key rather than appending, and resolve conflicts on write (latest-wins or explicit reconciliation) instead of at read time.
Where Semantic / Vector / Graph Memory is native.
OpenAI Vector Storesmanaged vector storesNative
Microsoft Agent Frameworkknowledge storeNative
CogneeOpen-source knowledge-graph memory built from unstructured data.Native
Keep going