Contact Memory & Emotional Intelligence Spec

Last Updated: 2026-06-29 Status: Active

1. The Challenge

1. Chat Engine is stateless. It processes one message at a time. The 20-message sliding window is fine for support but destroys relationship continuity. An AI that forgets "your dog's name is Bruno" isn't building a relationship — it's breaking one.

2. AI Brain memory is per-tenant, not per-contact. Brain memories answer "what does this business care about?" not "what does this specific end-user care about?" They're different domains with different lifecycles.

3. Emotions matter in relationship contexts. A customer support bot can be consistently neutral. An AI companion, tutor, or coach must adapt: playful when the user is happy, supportive when they're sad, calm when they're anxious.

4. Relationship depth requires context density. A new contact needs discovery questions. A 30-day contact expects the AI to know their life. The system must scale context injection based on relationship stage — thin prompts for new contacts, rich prompts for deep relationships.

2. Memory Architecture

2.1 How It Differs from AI Brain Memory

2.2 Memory Types

Facts decay slowest (0.003/day). "Your dog's name is Bruno" should last for months. Episodes decay fastest (0.008/day). "We talked about the weather on March 5th" fades unless reinforced.

2.3 Memory Provenance (source field)

Every memory has a source column tracking how it was created:

2.4 Multi-Signal Retrieval

Retrieval scoring for L2 semantic search:

RetrievalScore = (embedding similarity × 0.35)
               + (recency × 0.25)
               + (importance × 0.20)
               + (access_frequency × 0.10)
               + (entity_match × 0.10)

Entity match weight is higher than AI Brain (0.10 vs 0.05). If the user mentions "Bruno," memories linked to that entity surface regardless of embedding similarity.

3. Extraction Architecture

Memory extraction runs in three tiers with different triggers, latency, and cost.

Tier 1 — Inline Rules-Based (every message, ~0ms)

Runs synchronously before POST /ingest returns 202. No LLM calls.

Implemented in: extractImmediateMemories(message) and extractEntities(message) in src/lib/entities.ts + @repo/brain

Fact detection patterns:

"I am {X}" / "I'm {X}"              → fact about identity
"I have {X}" / "I've got {X}"       → fact about possessions/relationships
"My {X} is {Y}"                      → fact about named entity
"I work at {X}" / "I study at {X}"   → fact about workplace/school
"I live in {X}"                      → fact about location

Preference detection patterns:

"I like {X}" / "I love {X}"         → positive preference
"I don't like {X}" / "I hate {X}"   → negative preference
"I prefer {X}"                       → comparative preference
"Don't talk about {X}"              → avoidance preference

Entity detection: rules-based regex matching for person names, pet names, workplaces, and places. Writes to both legacy ci_contact_entities (for backward compat) and the new ci_entities table.

Dedup: exact-match check on (contact_id, content) before inserting — prevents duplicates from repeated ingests of the same message.

Tier 2 — Session-Close LLM Batch (one call per session, ~2–5s)

Triggered when POST /ingest receives session_close: true + session_messages (the full transcript).

Implemented in: extractSessionMemories(transcript, existingMemories, llmProvider, model) in src/lib/extraction.ts

What the LLM extracts:

{
  "memories": [
    { "action": "ADD", "type": "fact", "content": "...", "importance": 0.8, "confidence": 0.9 },
    { "action": "UPDATE", "type": "fact", "content": "...", "supersedes_id": "mem_old", "importance": 0.85, "confidence": 0.95 },
    { "action": "DELETE", "supersedes_id": "mem_wrong" }
  ],
  "entities": [
    { "type": "person", "name": "Bruno", "properties": { "species": "dog", "breed": "golden retriever" } }
  ],
  "entityRelationships": [
    { "from": "Arjun", "to": "Bruno", "relationship": "linked_to", "confidence": 0.95 }
  ],
  "emotionalSummary": {
    "dominantMood": "frustrated",
    "moodScore": -0.4,
    "variance": 0.2,
    "keyTriggers": ["work_stress"]
  },
  "relationshipSignal": "stable"
}

Writes:

• New/updated memories with provenance (source: "ai_extracted", sourceSessionId)
• Entity graph nodes (ci_entities via upsertEntity())
• Entity graph edges (ci_entity_relationships via addEntityRelationship())
• Session emotional snapshot (ci_emotional_snapshots with period_type: "session")

Embeddings for new memories are generated asynchronously via c.executionCtx.waitUntil().

Tier 3 — Weekly Consolidation (9-step pipeline, Sunday 2am UTC)

Triggered by CF Cron. One queue message per active contact is enqueued to CONSOLIDATION_QUEUE. The queue consumer runs consolidateContact() in src/lib/consolidation.ts.

9-step pipeline:

1. Decay importance — multiply by (1 - decayRate) per day elapsed. Skipped if last_reinforced_at is within 14 days.

   • fact: 0.003/day, preference: 0.005/day, episode: 0.008/day, pattern: 0.003/day

2. Episode → pattern compression — when 3+ episodes share a theme, LLM summarizes them into a single pattern memory. Episodes remain in the DB but are no longer served as context.

3. Entity dedup — batchResolveEntities() checks entity pairs with Levenshtein similarity in the 0.80–0.85 range. Optionally asks LLM to confirm merges before applying them. Clear matches (≥ 0.85) are merged automatically.

4. Emotional arc rollup — session snapshots from the past week are aggregated into a single weekly emotional snapshot: average mood score, dominant mood, variance, trend direction.

5. Relationship trajectory snapshot — computes ci_relationship_snapshots row for the current week: stage, score, trajectory (moving_up/stable/moving_down/volatile), velocity (slow/moderate/fast).

6. Regenerate L1 core_summary — LLM writes a 1–3 sentence behavioral guidance string from the top memories + entities. Stored in contacts.core_summary. Replaces the previous summary.

7. Soft-prune low-importance memories — memories with importance < 0.05 have expires_at set to now + 7 days. They're not deleted immediately — a follow-up pass removes them.

8. Remove orphan entities — entities with confidence < 0.3 that haven't been seen in 30+ days and have no ci_entity_relationships edges are deleted.

9. Update last_consolidated_at — stamps the contact row so the next cron can filter by staleness.

4. Memory Conflict Resolution

The superseded_by column implements a ledger-style conflict resolution pattern. Old memories are never deleted — they're marked as superseded, preserving the history.

UPDATE action:

1. New memory inserted with superseded_by = null
2. Old memory: superseded_by = new_memory_id
3. Queries for active memories: WHERE superseded_by IS NULL

DELETE action:

Old memory: superseded_by = "DELETED"
(sentinel — prevents it from appearing in active queries)

POST /contacts/:id/correct (owner/contact correction):

1. Find highest-importance active memory WHERE content ILIKE '%hint%'
2. Insert new memory: source = "contact_corrected", confidence = 1.0
3. Old memory: superseded_by = new_memory_id
4. contacts.core_summary = null (force regeneration on next weekly run)

This means a business owner can always correct what CI "knows" about a contact, and the history of what was believed is preserved.

5. Entity Graph

The entity graph maps what CI knows about a contact's world — people, goals, motivations, places, and events — and the relationships between them.

5.1 Nodes (ci_entities)

Contact: Arjun
├── pet: Bruno (confidence: 0.98, properties: { species: "dog", breed: "golden retriever" })
├── place: Infosys (confidence: 0.92, properties: { role: "workplace" })
├── motivation: career_growth (confidence: 0.85)
├── aversion: politics (confidence: 0.9)
└── person: Mom (confidence: 0.88, properties: { location: "Chennai" })

5.2 Edges (ci_entity_relationships)

Bruno → linked_to → Arjun (confidence: 0.95)
career_growth → motivated_by → Infosys (confidence: 0.8)

5.3 Extraction

• Session-close LLM call extracts both nodes and edges from the full transcript
• upsertEntity() checks exact name match within (contact_id, type, name) unique index before inserting; merges properties on match
• Inline Tier 1 also calls extractEntities() (rules-based regex) but writes simpler nodes without relationship edges

5.4 Dedup

• Exact match: handled by unique index on (contact_id, type, name)
• Fuzzy dedup: batchResolveEntities() in weekly consolidation checks pairs with Levenshtein similarity ≥ 0.80; merges properties and redirects FK references

5.5 L2 Enrichment

When GET /context/:id?message=... is called, expandOneHop() finds all entity IDs mentioned in the message (by name match), then fetches their 1-hop graph neighbors. The neighbor names are appended to message_relevance as "Also known: ...".

6. Emotional Intelligence Layer

6.1 Mood Classification

Per-message (Tier 1 inline): LLM haiku-class call on every user message.

Moods: happy | neutral | frustrated | sad | anxious | excited | confused
Energy: high | medium | low

Fallback chain:

1. Haiku LLM classification (primary)
2. On timeout/error → keyword-based classifyMoodKeyword() (no API call, < 1ms)

Keyword fallback:

"haha", "lol", "amazing"         → happy, high
"sad", "crying", "miss"          → sad, low
"worried", "nervous", "anxious"  → anxious, low
"ugh", "annoyed", "frustrated"   → frustrated, medium
"bored", "meh", "whatever"       → bored, low
default                          → neutral, medium

6.2 Crisis Detection

The session-close LLM call detects crisis signals in the full transcript and sets contacts.crisis_level:

• none — normal conversation
• distress — sustained negative mood, language suggesting struggle
• crisis — explicit crisis language ("I want to die", "can't take this")

crisis_level is returned in every GET /context/:id response. Chat Engine is responsible for gating its response behavior — CI sets the signal, CE acts on it.

7. Context Injection by Relationship Stage

Different stages get different context density in session_core:

The message_relevance (L2) field adds message-specific context on top of session_core regardless of stage.

8. Memory Growth Over Time

Memory count doesn't grow linearly because weekly consolidation compresses episodes into patterns and prunes low-importance memories.