Scorecard
Audited 2026-06-28. Overall: 7.8 / 10
| Parameter | Score | Notes |
|---|---|---|
| API Design | 8.0 | 25+ routes, SSE streaming, cursor pagination; analytics/events/cleanup endpoints documented but absent |
| Data Model | 8.5 | 5 tables, idempotent external_msg_id, SLA tracking, audit log; single migration |
| Security & Auth | 8.0 | AES-256-GCM credentials, HMAC webhook verification, PBAC; rate limiting not enforced, PII redaction absent |
| Multi-tenancy | 10.0 | Perfect — tenant_id on every table and every query; unique constraints prevent cross-tenant leakage |
| Billing | 8.5 | Accumulator model, pre-flight gate, out-of-credits fallback + owner notification |
| LLM Integration | 9.0 | Three model tiers, multi-provider fallback (Anthropic→Gemini), streaming + non-streaming |
| Memory System | 7.5 | 3-layer (mood / context / tool-recall) solid; KV session cache documented but never built |
| Channel Support | 8.5 | WhatsApp embedded signup + HMAC, Telegram webhook + secret token, Website Widget SSE |
| Widget Embedding | 8.0 | SSE streaming, non-streaming fallback, widget token; no server-side origin domain enforcement |
| Session Management | 8.5 | SLA DO alarms per session, escalation flow, sliding window, rolling summary |
| Knowledge / RAG | 8.0 | Delegates to brain-service, semantic search working; no confidence scoring |
| Testing | 4.0 | Extremely thin — no comprehensive test suite found |
| Observability | 5.0 | Audit log in DB; no structured logging; analytics endpoints unimplemented |
| Feature Completeness | 6.5 | Core chat across 3 channels working; analytics, real-time events, lead capture, PII redaction all absent |
Bugs
B1 — Lead capture rules stored but never enforced
Severity: High
File: src/db/schema.ts (ce_bot_config.lead_capture_rules), src/lib/orchestrator.ts
The lead_capture_rules JSONB column stores an array of { trigger, askFor, message } rules. The bot config GET/PATCH endpoints accept and save these rules. However, the orchestrator never reads or evaluates them during message processing. The feature is UI-visible but completely no-op.
Fix: In the orchestrator, after assembling the system prompt, evaluate leadCaptureRules against the current message. If a trigger matches and the contact is missing the askFor field, inject a capture rule into the prompt instructing the bot to ask for it.
B2 — SLA breach notification is a TODO comment
Severity: Medium
File: src/scheduler.ts (alarm handler)
The SessionScheduler DO alarm handler:
- Clears
sla_breach_atfrom the session ✓ - Logs the SLA breach ✓
- Has a
// TODO: Send alert via COMMS_SERVICEcomment — call never made
The dashboard shows the SLA breach indicator, but the owner receives no notification (email, push, webhook) when an escalated session goes unanswered beyond the SLA window.
Fix: Complete the COMMS_SERVICE call. Use COMMS_SERVICE.fetch(POST /internal/alerts/sla-breach, { sessionId, tenantId, slaBreachAt }).
B3 — Session history always loads from Postgres (KV cache never built)
Severity: Low
File: src/lib/orchestrator.ts, memory-management-spec.md
The memory-management-spec.md documents a CF KV cache for session history (24h TTL) to avoid a DB hit on every message. The code doesn't use BRAIN_KV or any KV namespace for session caching — every message triggers a Postgres query for the last 6 messages.
Impact: Minor latency overhead on every message; no correctness issue. At scale, this adds DB load that the KV cache was meant to avoid.
Fix: Cache last-N messages in KV keyed by session:{sessionId}:history. Invalidate on new message insert.
B4 — Analytics endpoints documented but return 404
Severity: Medium
File: docs/chat-engine/api-spec.md
The API spec documents:
GET /analytics/overview— session counts, resolution rate, avg response timeGET /analytics/unanswered— messages without a bot response
Neither endpoint exists in the codebase. Dashboard components that call these will receive 404.
Fix: Implement the endpoints with SQL aggregations against ce_messages and ce_sessions. The data is all available in the existing tables.
B5 — Real-time event stream documented but not implemented
Severity: Medium
File: docs/chat-engine/api-spec.md
GET /events/stream is documented as an SSE endpoint emitting:
chat:new_message— incoming customer messagechat:escalated— session escalated to humanchat:assigned— session assigned to agentchat:resolved— session closedchat:sla_breach— SLA timer fired
No implementation exists. Dashboard team chat inboxes that poll for new sessions must use polling instead of push.
Fix: Implement using a Durable Object or Cloudflare Worker SSE stream that fans out events to connected dashboard clients per tenant.
B6 — No server-side widget origin domain validation
Severity: Low
File: src/routes/widget.ts
The website widget channel stores an identifier (domain, e.g., mysite.com). The widget token validates that the caller knows the token, but the Chat Engine doesn't enforce that requests come from the registered domain. An attacker who obtains the widget token can embed the chatbot on any site and bill it to the legitimate tenant.
Fix: Read the Origin or Referer header on /chat/widget/* requests and validate it matches the channel's identifier domain. Reject with 403 if it doesn't match (or if no origin header is present).
B7 — POST /sessions/:id/escalate not documented
Severity: Low
File: src/routes/sessions.ts, docs/chat-engine/api-spec.md
The escalation endpoint exists and is used internally by the orchestrator (when the handoff trigger fires). It is not documented in api-spec.md. Dashboard developers who want to programmatically escalate a session have no documented path.
Fix: Add the endpoint to api-spec.md with its request body, response shape, and permission requirement (chatbot:sessions.write).
Areas of Improvement
G1 — No rate limiting enforcement
Priority: High
File: src/routes/*.ts
The security guide documents per-scope rate limits:
- Widget messages: 30/min per session
- Webhook ingress: 200/min per tenant
- Dashboard API: 100/min per user
None of these are enforced in code. A malicious actor can flood the widget endpoint, burning through the tenant's credits or triggering DDoS-level load on the LLM.
Fix: Add Cloudflare native rate limiting bindings in wrangler.toml (one per scope). Apply in middleware before route handlers.
G2 — No PII redaction
Priority: Medium
File: src/lib/orchestrator.ts
The security guide says user messages are scrubbed for PII (credit card numbers, government IDs, etc.) before being sent to the LLM and before being passed to Contact Intelligence. No scrubbing code exists.
Fix: Add a lightweight regex-based scrubber before generateResponse() and before POST /ingest calls. Patterns: credit card (\b\d{4}[\s-]\d{4}[\s-]\d{4}[\s-]\d{4}\b), Aadhaar, PAN card.
G3 — No Instagram webhook handler
Priority: Medium
File: src/routes/webhooks.ts
The ce_channels.platform enum accepts instagram as a valid value. No webhook handler exists for Instagram DMs (POST /webhooks/instagram/{tenantId}). The channel can be created but messages will never be processed.
Fix: Implement the Instagram webhook handler (similar to WhatsApp — both use Meta Graph API). Add channel creation flow for Instagram.
G4 — Conversation summary not returned to callers
Priority: Low
File: src/lib/orchestrator.ts
After each assistant message, the orchestrator async-fires a summary update: a Haiku model call compresses the conversation into a 1-line rolling summary stored in ce_sessions.summary. This summary is never returned to the widget or included in the GET /sessions response.
Fix: Include summary in the GET /sessions list response and GET /sessions/:id detail. This lets the dashboard show a conversation preview without loading all messages.
G5 — Lead capture rules config has no validation
Priority: Low
File: src/routes/config.ts
lead_capture_rules is accepted as raw JSONB with no schema validation. Invalid rule shapes are silently stored and would cause runtime errors when enforcement is added (B1).
Fix: Add Zod schema for lead_capture_rules items: { trigger: string, askFor: "name" | "email" | "phone", message: string }.
G6 — Internal maintenance endpoints not implemented
Priority: Low
File: docs/chat-engine/api-spec.md
Documented but absent:
POST /internal/sla/sweep— find sessions with breached SLA, trigger alertsPOST /internal/sessions/cleanup— close sessions idle for 24h
The SessionScheduler DO handles individual SLA alarms, but there's no sweep for sessions where the DO alarm was missed (e.g., if the DO was evicted).
Fix: Add a daily cron trigger calling runSessionCleanup() and runSlaSweep() against ce_sessions.
G7 — Knowledge base update from chat not wired
Priority: Low
File: docs/chat-engine/user-journey.md
The user journey describes a dashboard shortcut: "When the AI can't answer a question, a button appears to add the answer to the knowledge base." This shortcut calls POST /brain/knowledge to add the unanswered question + human answer as a new document. The shortcut is in the spec but no endpoint or UI flow connects it.
Action: Wire the "Add to Knowledge Base" action in the dashboard. No backend changes required — it's a direct call to POST /brain/knowledge.