Skip to content

Architecture Overview

agent-mem has three jobs. It records what coding agents do through their hooks, it thinks about it later in background processes that call the agent’s own CLI, and it recalls what matters, injected into the next session, searchable over MCP and visible in a local viewer. One SQLite file holds all of it.

Agents call the hook, which writes memory.db (or the spool) and returns context; observe and index work in the background with the agents' CLIs and mem0; the MCP server, the viewer and the CLI read memory.db.
Open in Excalidraw
ComponentCommandRunsRole
Hooksagent-mem hook <harness> [event]per event, ~65 msrecord, inject, warn, block; start the rest
Observeragent-mem observeafter requests, until idlefacts, titles, session summary, lessons, review, graph
Indexeragent-mem indexafter requests, until donespool replay, embeddings, notes, token accounting, consolidation, retention
MCP serveragent-mem mcpper agent sessionmem_search, mem_timeline, mem_get, mem_remember, mem_forget
Vieweragent-mem servealways on, port 37778a read-only page over the database
CLIagent-mem …on demandsetup, status, doctor, search, purge, export, …
LayerChoiceWhy
LanguagePython 3.11–3.14the hook path uses the standard library only
StorageSQLite (WAL), FTS5 with the trigram tokenizerone file, concurrent readers, substring search in any language
Vectorsfastembed (ONNX Runtime), paraphrase-multilingual-MiniLM-L12-v2, numpymultilingual, no PyTorch, about 220 MB
Fact extractionmem0 with the agent’s CLI as its model (claude -p, codex exec, copilot -s), optional Ollamano API key; the vendor that saw a request writes its facts
Fact indexQdrant in local path mode (through mem0)mem0’s deduplication; rebuildable from SQLite
Topic linksspaCy (en_core_web_sm or another language)mem0’s entity store
Time wordsdateparser“letzte Woche”, “8. April”, “last friday”
MCPthe official Python MCP SDK, stdioone process per session
ViewerPython http.server, a self-contained HTML page with its own scriptno build step, no network
Installeruv toolone isolated environment, in-place upgrades
  1. The agent fires an event and runs agent-mem hook <harness> with the payload on stdin.
  2. The payload is normalized to one Event, checked (internal, disabled, paused, excluded), cleaned of private sections and secrets, and written in one transaction: session, event, turn, recipes, rules, dead ends.
  3. If the database is busy or the 400 ms deadline passes, the event goes to spool/ instead.
  4. A closed turn is queued in observe_queue.
  1. Stop starts agent-mem observe --drain. It leases up to five turns of a session and builds a bundle: prompts, what the tools showed, answers, subagent answers.
  2. The agent’s own CLI extracts facts through mem0; agent-mem mirrors them into memories.
  3. A second call writes turn titles, the running session summary and lessons.
  4. Once a day per project the facts are reviewed for duplicates and outdated ones; new memories are read into a graph of things and relations.
  5. agent-mem index replays the spool, embeds new text, reads the agents’ own notes, measures token work and runs the daily consolidation.
  1. Session start: a briefing with your rules and the last session first, within 800 tokens.
  2. Each prompt: confident matches only, within 2,400 tokens.
  3. Before a tool: rule notes or a block, and the history of the files it touches.
  4. After a failure: how this error was fixed before.
  5. After compaction: session goal, decisions, open failures, files.
  6. On demand: the MCP tools, agent-mem search, the viewer.
MomentEventWhat agent-mem does
openSessionStartbriefing, viewer started, panel
askUserPromptSubmitclose the previous turn, open a new one, learn rules, recall
workPreToolUse / PostToolUsewarn or block, file history, record, fix hints, recipes, dead ends
doneStopanswer stored, turn closed and queued, observer and indexer started
compactPreCompact / SessionStart(compact)the compaction block
closeSessionEndlast turn closed, summary line
src/agent_mem/
core/ config, db + schema/, events, identity, privacy, signals, spool, timeutil, logutil, paths
standard library only; shared by every path
hooks/ hook (entry point), normalize/ (claude, codex, copilot, opencode), capture, inject
the hook path: payload in, SQLite write, context out; standard library only
memory/ store, search, semantic, learn, consolidate, views, indexer, notes, work
memories, retrieval, embeddings, learning, the background indexer
observe/ queue, bundle, providers, mem0_adapter, summary, review, relations, worker,
spacy_model, coding_instructions.md — the only place that imports mem0 and Qdrant
mcp/ server: the stdio MCP server
cli/ argument parsing, commands, install (setup --write, uninstall), health (status, doctor)
ui/ the self-contained page and the loopback server
bench/ evaluate: fixture and LongMemEval-S runner (not used at runtime)
plugins/ claude, codex, copilot, opencode: hook files, manifests, the OpenCode plugin

Dependencies point one way: core ← hooks, memory ← observe, mcp, ui, bench ← cli. Nothing in core, hooks or memory imports from observe, cli or ui. A test checks that the hook path never imports mem0, Qdrant, fastembed or numpy.

  • The agent is never blocked or broken. Every hook exits 0 and prints valid output or nothing.
  • No event is lost. Busy database, migration or deadline: the event waits in the spool and is replayed in order.
  • No turn is lost. It stays in observe_queue until facts are extracted or it is stored word for word; failures back off and end in failed, retryable.
  • A request stays with its vendor. A Claude Code turn goes to Claude, a Codex turn to Codex, a Copilot turn to Copilot, unless you set observe.cross_vendor.
  • SQLite is the truth. Everything in mem0/ can be rebuilt from it without a model call.
  • Nothing leaves the machine except model downloads and your agent’s own CLI talking to its own provider.