System Architecture — Contact Intelligence

Status: Active Last Updated: 2026-06-29

1. High-Level Design

1.1 Architectural Components

1.2 System Diagram

2. Memory Tiers

CI serves context to the chat engine in three tiers. Each tier has a different retrieval cost, token budget, and trigger condition.

L1 — Session Core (always injected)

• Source: ci_contacts.core_summary — generated by weekly consolidation job
• Location in prompt: system prompt, once per session
• Size: ≤ 50 tokens
• Format: behavioral guidance, not facts

[Priya | Engaged → stable | Mood: improving]
Approach: habit-focused not numbers, warm but not intense,
November wedding is the real motivator

Cold start: for the first 5 sessions (before enough context exists), L1 is filled from ce_bot_config.new_contact_brief — an owner-written default. At session 5, real extracted context takes over automatically.

L2 — Message Relevance (conditional)

• Source: pgvector semantic search on the current message text
• Location in prompt: internal note injected immediately before the user message
• Size: ≤ 150 tokens
• Trigger: cosine similarity > 0.7 OR message contains emotional signal OR entity name match detected
• When nothing qualifies: inject nothing — silence is correct

L2 is what makes the AI feel like it remembered something relevant, right now. L1 tells the AI who this person is; L2 tells the AI what matters in this specific moment.

L3 — Archival (on-demand)

• Source: full memory store, searched via pgvector
• Trigger: LLM explicitly calls recall_memory tool
• Use: rare, safety net for specific questions that need a deep search

Context Packet API Contract

GET /ci/context/:contactId?message=<encoded>&sessionCount=<n>

Response:

{
  "session_core": "[Priya | Engaged → stable | Mood: improving]\nApproach: ...",
  "message_relevance": "She mentioned her sister last week — they had a fight about the diet",
  "has_message_context": true,
  "relationship": {
    "stage": "engaged",
    "trajectory": "stable",
    "mood_trend": "improving"
  },
  "crisis_level": "none",
  "triggers": ["wedding_deadline_approaching"]
}

The chat engine assembles the prompt — CI delivers ingredients, not the assembled prompt. This keeps CI decoupled from chat engine prompt engineering.

Prompt placement order:

1. Bot persona + owner instructions
2. L1 session_core — in system prompt
3. Knowledge base RAG (if enabled)
4. Conversation history (last 6 messages)
5. L2 message_relevance — as internal note before user message (conditional)
6. User message

3. Memory Pipeline

Real-time extraction (every message — zero LLM cost)

Rules-based only, non-blocking, runs inline:

• Regex patterns: "my name is", "I am", "I have", "I like/hate", phone numbers, explicit dates
• Correction patterns: "that's wrong", "I never said", "actually I'm not", "no I meant"
• Writes to a session staging buffer — NOT directly to the memory store

Session-end extraction (one LLM call per session)

Triggered when session closes or 2 hours of inactivity. Single haiku-class LLM batch call returns:

1. New memories — fact, preference, episode, behavior, pattern (with type + importance score)
2. Conflict checks — compare against existing store → ADD / UPDATE / DELETE
3. Entity extraction — new nodes + relationships for the entity graph
4. Emotional session summary — dominant mood, variance, key triggers → writes ci_emotional_snapshots (session)
5. Relationship signal — depth score, breadth, emotional investment → feeds weekly trajectory

Conflict resolution

When new information contradicts existing memory, the LLM chooses one of:

Memories are never hard-deleted from the store (only soft-archived) — but superseded memories don't surface in retrieval.

Real-time correction (synchronous — only exception to session-end rule)

When the chat engine detects a correction mid-session:

POST /ci/contacts/:id/correct { hint, correction }

• Creates new memory: source: contact_corrected, highest confidence
• Marks old memory as superseded_by new
• Runs before the AI generates its acknowledgment response
• The fix is in place before the AI says "Got it"

4. Entity Graph

Two Postgres tables, no graph database needed.

ci_entities — nodes

type: person | goal | event | behavior | aversion | motivation | topic | place
name, properties: jsonb
confidence: 0.0–1.0
last_seen_at

motivation is the highest-value type — it's rarely stated directly, inferred by LLM from patterns across sessions. If you know what someone is actually motivated by, you know how to talk to them.

ci_entity_relationships — edges

relationship: motivated_by | triggered_by | linked_to | opposes | part_of | is_related_to
confidence

Entity resolution — C+B hybrid

• At extraction time (cheap): fuzzy name match 85%+ similarity + same type → update existing, don't create duplicate
• Weekly consolidation (LLM batch): candidate pairs sent in groups of 10 → "are these the same entity?" Confirmed matches merged; properties combined.

How entities improve L2

When building L2 context, CI extracts entities mentioned in the current message and expands 1 hop in the graph. This finds causally connected context — not just similar things, but related things. If the message mentions "the gym" and the graph shows gym → triggers_by → anxiety, the L2 context can include that relationship even if the word "anxiety" wasn't in the message.

5. Emotional Arc

Intra-session (chat engine — zero CI calls)

Simple sentiment shift on last 3 messages: "tone has dropped." The chat engine handles this locally. No CI call, no cost, immediate.

Cross-session arc (CI owns)

per-session snapshot  →  written at session-end extraction
weekly rollup         →  aggregate session snapshots
                          → dominant mood, trend direction, variance, key triggers
                          → written to ci_emotional_snapshots (weekly period)

Trend: arithmetic comparison of last 3–4 weekly mood scores → improving | declining | stable | volatile

Crisis detection — two levels:

False positives acceptable. False negatives are not.

6. Relationship Trajectory

Stage definitions

new       0–2 sessions
building  3–10 sessions, returning voluntarily
engaged   regular cadence, personal depth, goal progress
loyal     4+ weeks continuous, high frequency, deep memory
at_risk   50%+ frequency drop OR 2-week negative mood trend
dormant   14+ days no session

Trajectory = stage + direction + velocity

• Direction: compare relationship_score across last 3–4 weekly snapshots
• Velocity: slow | moderate | fast (rate of change)

Early intervention window

A contact still labeled engaged but showing 30% frequency drop + shorter sessions + declining mood = moving_down, moderate. CI fires outreach at this point, not when they've officially become at_risk.

This is the key insight: waiting for stage change is too late. Detecting drift within a stage is what actually prevents churn.

Computed weekly in the consolidation job (pure formula, no LLM). Feeds into L1 session_core and proactive outreach triggers.

7. Proactive Outreach

Two modes

Smart retention (CI decides): one toggle in chatbot settings. CI watches signals — at_risk trajectory, negative arc 2+ weeks, session gap approaching threshold — and fires outreach automatically.

Owner-configured rules: inactivity trigger, scheduled, milestone, recurring. Owner sets conditions and message template (or AI-generated). Wizard UI in dashboard.

Split responsibility

CI does not generate the final message. Chat engine does not decide when to reach out.

CI → intent + approach guidance:
  "Reach out to Priya. 6 days quiet, last session ended
   frustrated about slow progress. Tone: warm, don't mention
   the goal, ask how she's feeling."
 
Chat Engine → final message, using CI guidance + bot persona

Pull principle

Every proactive message is designed to elicit a reply — reference something specific from last conversation, end with an open thread. Never a self-contained information broadcast.

WhatsApp 24-hour window awareness

The window check lives in the outreach scanner (CI knows channel + last message timestamp).

Anti-spam guarantees

• Max 2 proactive messages per week per contact
• Min 24h between any two outreach messages
• Never during crisis state
• Contact says "stop" → outreach_disabled: true, permanent until owner re-enables

8. Multi-Channel Identity

Model

ci_contacts (master)
    ↑
ci_contact_channels (one row per channel)
  channel_type: whatsapp | telegram | website_widget | instagram
  external_id: "wa-+919876543210"
  confidence: 1.0 | 0.8

All memory, arc, graph, and trajectory live on the master contact. The channel record is just a lookup key to find the master.

Linking paths

1. Contact-initiated (confidence 1.0) — contact self-identifies on another channel, or widget asks "have you chatted with us before?"
2. Owner-initiated (confidence 1.0) — owner merges two contacts in dashboard after confirming they're the same person
3. Probabilistic suggestion (CI proposes, owner confirms) — nightly: same name + same entities + similar writing style → surfaces "possible match" card. Never auto-merges.

Merging two different people is catastrophic. The safety constraint: human confirmation always required.

Merge operation

• Memories: conflict-resolve (same as session-end)
• Entity graph: entity-resolve (same as weekly consolidation)
• Emotional arc: combine + weight recency
• Relationship trajectory: take the higher stage
• Conversation history: stays in CE sessions — not merged, both contribute to the same CI master going forward
• Merges are logged, reversible within 24 hours

9. Weekly Consolidation Job

Runs Sunday 2am UTC (0 2 * * 0). Per active contact (where last_consolidated_at < 7 days ago), in order:

Step 1  decay_memories()
        importance × decay_rate by type:
          facts: 0.99 | preferences: 0.95 | episodes: 0.90 | patterns: 0.98
        Skip if last_reinforced_at < 2 weeks ago
 
Step 2  compress_episodes()
        If 3+ similar episodes exist → LLM extracts the pattern
        Reduces individual episode importance, preserves originals as archived
 
Step 3  resolve_entities()
        LLM batch (10 candidate pairs per call): "are these the same entity?"
        Confirmed matches → merge properties
 
Step 4  rollup_emotional_arc()
        Aggregate session snapshots → write weekly ci_emotional_snapshots row
        Compute trend direction + variance
 
Step 5  compute_relationship_trajectory()
        Formula: stage + direction + velocity → write ci_relationship_snapshots row
        Check for early intervention window
 
Step 6  regenerate_l1_core()
        LLM call: fresh behavioral summary from all current data
        Write to contacts.core_summary + KV cache
 
Step 7  prune_memories()
        Soft-archive importance < 0.05 (never hard delete)
 
Step 8  cleanup_orphan_entities()
        Remove: 0 relationships + confidence < 0.3 + not seen in 30 days
 
Step 9  update last_consolidated_at

After all contacts:

Step 10  compile_owner_alerts()
         at_risk contacts, crisis flags, milestones → owner digest via COMMS_SERVICE

Cost

~1–3 haiku LLM calls per active contact per week. Under $1/week per 1,000 active contacts.

Resilience

Each contact is processed independently — a failed contact doesn't block others. last_consolidated_at tracks which contacts need retry. Large tenants fan out via Cloudflare Queue (contact = queue message), rate-limited to avoid LLM API rate limits.

10. Data Model Overview

New tables

ci_contact_channels — multi-channel identity
ci_entities — entity graph nodes
ci_entity_relationships — entity graph edges
ci_emotional_snapshots — emotional arc (session + weekly periods)
ci_relationship_snapshots — relationship trajectory (weekly)

Modified tables

ci_contacts — add: core_summary, last_consolidated_at, cold_start_used_until, crisis_level, channel_count

ci_contact_memories — add:

confidence and importance are separate dimensions:

• High importance + high confidence → inject prominently
• High importance + low confidence → inject with hedged framing ("she mentioned wanting to lose weight")
• Low importance + any confidence → likely decay candidate

11. Integration Points

Chat Engine → CI

CI → Chat Engine

Proactive outreach: CI generates intent → CI calls CE /internal/send with intent + approach guidance → CE generates final message and delivers via channel API.

AI Brain → CI

Brain consumes GET /ci/analytics/relationship-health for tenant-level insights. Brain does NOT write to CI.

Flows → CI (V2)

CI emits events: contact.at_risk, contact.dormant, contact.milestone, contact.returned. These become Flow trigger nodes. V1: CI owns the outreach engine. V2: outreach migrates to Flows, CI is the signal source.

12. Performance Budget

The context packet (L1 + L2) never blocks the user — the chat engine can also proceed with L1-only if L2 takes more than 10ms. Extraction never blocks the conversation.

13. Data Flow Guarantees

1. Context retrieval never waits for extraction. Extraction is async. Memories are available in the next session, not mid-session.

2. Outreach never spams. Rate limits enforced at DB level (unique constraints on trigger fire date) AND application level (scanner rate check before send).

3. Contact data isolation is absolute. Every query scopes to tenant_id + contact_id. Tenant A's contacts are never visible to Tenant B.

4. Correction is synchronous. The only real-time write in CI. The fix is in the DB before the AI says "Got it."

5. Memory deletion is complete. Forget-me request → cascade delete all rows in ci_contact_memories, ci_entities, ci_emotional_snapshots, ci_relationship_snapshots, ci_contact_channels for that contact. Synchronous, no orphans left.

6. Weekly job failure is isolated. One contact failing consolidation doesn't block others. The contact's last_consolidated_at stays old → it retries next week automatically.