Rolling Memory Graph — Design Spec
Problem Statement
The current memory infrastructure (Memory Catcher, Memory Lane, injection hooks) is entirely post-hoc — it extracts after sessions end and retrieves via flat semantic search. There's no live tracking of understanding within a session, no topology between memories, and no mechanism to tune resolution based on relevance.
Three specific pain points:
- Mid-project context loss — compaction or new session kills the texture of where you were
- Cross-domain blindness — AI misses connections between projects/topics
- Resolution mismatch — AI knows too much (noise) or too little (gaps), no way to tune detail level
Design Decisions (12 from interview + stress test refinements)
1. Structure: Graph
Nodes (concepts/entities) + weighted edges (relationships between them). Graph provides machine-navigable topology while remaining renderable as human-readable narrative.
2. Updates: Heuristic-Gated Differential Probe
~~Each turn~~ Not every turn. A lightweight heuristic gate runs per turn. The full LLM probe only fires when the gate triggers.
Four heuristic triggers (any one fires the probe):
- New named entity not in current graph
- Domain shift (conversation moves between relationship/project/technical territory)
- Decision language ("let's go with", "decided", "the plan is")
- Correction language ("actually", "no", "wait", "not that")
Dead man's switch: Force probe after 8 turns of silence.
Probe output: JSON patches — nodes added/strengthened/weakened, edges created/modified. Patch model, not snapshot.
Cost impact: ~10-15 probes per 60-turn session instead of 60. 60-80% cost reduction.
3. Node Schema (Minimal)
Nodes (5 fields):
id — stable identifier (slugified label, e.g. rolling-memory-graph)
label — human-readable name ("Rolling Memory Graph")
domain — subgraph: relationship, project, technical, personal
weight — float 0.0-1.0, current relevance (decay operates on this)
last_activated — timestamp of last probe that touched this node
Edges (3 fields):
source/target — node IDs
relationship — verb describing connection ("blocks", "informs", "contradicts", "extends", "supersedes")
strength — float 0.0-1.0 (decays independently from node weight)
4. Contradictions: Contradiction Edges
When a decision changes (e.g., "use Resend" -> "use Bento"), both nodes stay. A supersedes edge connects the new to the old. Retrieval layer prefers the newer one but can surface the old decision if asked. Preserves decision history.
5. Retrieval: Two Modes
- Explicit mention -> full resolution restore (name a topic, branch snaps to high-res)
- Detected overlap -> hint/peripheral awareness ("related context over here if you need it")
Different thresholds prevent noise while maintaining useful proactive surfacing.
6. Lifecycle: Layered (Hot/Warm)
- Hot graph — session-local JSON sidecar, high resolution, fast updates, no compression tax
- Warm graph — persistent in PostgreSQL, lower resolution by default but deep when you drill in
- Reconciliation between layers at session boundaries
7. Reconciliation: Three Triggers (Event-Driven)
Not boundary-driven. Three specific triggers:
- Session end — full reconciliation, hot graph merges into warm
- Pre-compaction — before context gets compressed, flush hot graph to warm as safety net
- Session resume — pull relevant warm graph branches back into fresh hot graph
Crashed sessions: Post-session Memory Catcher hook does best-effort reconciliation from transcript.
Merge logic:
- For each hot node, check if matching node exists in warm
- If yes: update warm weight (higher of two, or average weighted by recency)
- If no: insert with slight weight penalty (hasn't proven itself cross-session)
- Cross-session frequency is itself a relevance signal — nodes appearing in multiple sessions gain weight
8. Decay: Distance-From-Focus with Time Floor
Model: new_weight = weight (1 - decay_rate distance_factor) * time_factor
distance_factor = graph hops from current focus cluster (capped at max)
time_factor = slow exponential decay from last_activated
- Focus cluster = 3-5 most active nodes in current session
Key property: Distance-from-focus sets the rate of decay. Recently activated nodes resist distance decay because their time factor hasn't eroded yet.
Anchor node floor: Nodes seeded from flat memories never decay below 0.2. They lose resolution but never disappear.
Why not pure activation decay: The whole point of building a graph is that structure matters. Distance-from-focus is the only decay model that actually uses the graph as a graph. Pure activation treats it as a list.
9. Context Injection: Layered LOD + Progressive Disclosure
Automatic Level-of-Detail (LOD) rendering:
- Focus cluster: full detail (all fields)
- 1-hop neighbors: summaries (label + domain + weight)
- 2+ hops: counts ("12 additional nodes in technical domain, weight 0.1-0.3")
Progressive disclosure escape hatch: LLM can request expansion of any compressed layer. Requesting expansion of a cluster is itself a signal to increase its weight.
Not progressive disclosure alone — system actively recalculates LOD every probe cycle based on where focus is. The LLM expansion request is the fallback, not the primary mechanism.
Token budget: ~500-800 tokens for layered injection vs. 2-3K for full graph dump.
10. Integration: Bidirectional with Flat Memory
Graph -> Flat (Graduation): Three criteria, ALL must be met:
- Cross-session frequency >= 3 (appeared in 3+ sessions)
- Warm graph weight >= 0.6 (currently relevant)
- Edge count >= 2 (connected to multiple concepts)
When met, reconciliation flags for graduation. LLM drafts flat memory content. Human review required before promotion (one-way door).
Flat -> Graph (Seeding): Existing flat memories seed warm graph as pre-weighted anchor nodes with decay floor of 0.2. Corrections become high-weight anchors. Stale insights fade faster.
Reverse flagging: When anchor node drops below 0.2 despite floor for 6+ weeks, flag for archive review.
11. Nesting: Domain Only for MVP
MVP: Four domain subgraphs (relationship, project, technical, personal) with cross-domain edges. This directly solves cross-domain blindness (pain point #2).
v2 — Scope nesting: Turn -> conversation -> project -> system levels. Partially approximated by hot/warm lifecycle in MVP.
v3 — Temporal nesting: Today (high-res) -> this week (compressed) -> this month (landmarks). Partially approximated by decay function in MVP.
Rationale: Domain nesting is the only dimension not already approximated by other design decisions. Scope and temporal are refinements that make a working system better, not things that make it work.
12. Reconciliation Engine: Hybrid (LLM + Algorithm)
- LLM probe identifies what is significant and why (semantic judgment)
- Deterministic algorithms handle weight math (decay rates, edge strengthening, merge logic)
- Creates audit trail — probe output explains why, algorithm is reproducible
Validation Signals
Three metrics, self-reported (no dashboard needed):
- Cross-domain surfacing rate — how often the graph surfaces connections between domains that flat search wouldn't catch. Target: 2-3 useful surfacings per day.
- Context recovery time — turns needed before AI is "caught up" after resume/topic switch. Current: 3-5 turns. Target: 0-1 turns.
- Graduation quality — percentage of graduated memories surviving 30 days without deletion/correction. Target: 80%+.
MVP Scope (4 Components)
In MVP:
- Heuristic gate — per-turn check in user-prompt-submit hook. Four triggers + dead man's switch.
- Probe — Haiku LLM call via host-bridge. Input: current turn + hot graph. Output: JSON patches.
- Hot graph — JSON sidecar in session directory. Four domain subgraphs. Updated by probe output.
- Layered injection — reads hot graph, renders LOD layers, injected via assistant-tool-use hook.
NOT in MVP:
- Warm graph in Postgres (sessions independent at first)
- Graduation to flat memories
- Temporal or scope nesting
/graph show interface
- Cross-session frequency tracking
- Contradiction edges (overwrite for now)
MVP validates:
- Pain point #1 (mid-session context loss) — fully
- Pain point #2 (cross-domain blindness) — within a session only
- Core question: does the heuristic-gated probe produce a useful graph?
Existing Infrastructure (composes with)
- Memory Catcher: Haiku-powered post-session extraction, 10 memory types, 16K+ memories
- Memory Lane: PostgreSQL + pgvector, 1024-dim embeddings, semantic search, dedup, recall analytics
- Memory Injection Hooks: User prompt hook (4 slots, semantic + entity search) + assistant tool-use hook
- Memories API: Full REST CRUD, semantic search, feedback loop, suggestions
- Embedding Service: Ollama mxbai-embed-large, HNSW index
Key Insight: Why Session State Vectors Failed
Session state vectors were the closest prior attempt. They died because they were:
- Snapshots, not differentials (no "what changed")
- Flat, not graph-structured (no topology)
- Stored data waiting to be found, not a retrieval mechanism itself
- No decay or relevance weighting
The rolling graph avoids all of these failure modes by design.
Next Steps