AI Brain — Backlog

Scorecard

Audited 2026-06-28. Overall: 7.5 / 10

Bugs

B1 — POST /chat/confirm not implemented (501)

Severity: High
File: src/routes/chat.route.ts (line 506)

The destructive action confirmation endpoint returns { status: "not_implemented" } with a 501 status. This blocks any agent or UI flow that requires the user to confirm before executing a destructive action (e.g., deleting a blog post, clearing a conversation).

Fix: Implement the confirmation handler. The pattern should be: look up the pending action from conversation state (KV), verify it matches what the user confirmed, execute the tool call, clear the pending state.

B2 — Memory consolidation disabled pre-launch

Severity: High
File: wrangler.toml (lines 21–30 commented out), src/index.ts (queue consumer)

The Cloudflare Queue and its consumer for memory consolidation are commented out in wrangler.toml. This means:

• Memories extracted from conversations are never merged or deduplicated
• Importance decay and episode→pattern compression never run
• ai_memory table will accumulate raw extractions indefinitely
• ai_memory_entities will not be populated

The handler code (processConsolidationJob()) is written and ready; the queue binding is just commented out.

Fix: Uncomment queue bindings in wrangler.toml before launch. Also apply the brain-consolidation-dlq queue in the Cloudflare dashboard.

B3 — Insight generation disabled pre-launch

Severity: High
File: wrangler.toml (lines 33–39 commented out), src/index.ts (scheduled handler)

The cron trigger "0 */6 * * *" is commented out. The scheduled export handler calls scanAllTenants() + runPeriodicMaintenance(), but these never fire. No proactive insights are ever generated.

Fix: Uncomment cron trigger in wrangler.toml when ready to enable insights. Note: this will wake the Neon DB every 6 hours — at zero tenants it's wasteful (documented in wrangler.toml comment), but should be enabled as soon as any tenants are onboarded.

B4 — RLS policies not implemented in database

Severity: Medium
File: drizzle/0000_tricky_typhoid_mary.sql

The security guide explicitly calls for PostgreSQL Row-Level Security (RLS) policies as a second layer of isolation. The migration creates all 9 tables but adds no RLS policies. App-level tenant_id filtering is the only isolation guard.

If a query ever escapes the tenant_id filter (bug in middleware, SQL injection, or accidental cross-route call), the attacker can read any tenant's conversations, memories, and knowledge.

Fix: Add RLS migration. Pattern:

ALTER TABLE ai_conversations ENABLE ROW LEVEL SECURITY;
CREATE POLICY ai_conversations_tenant ON ai_conversations
  USING (tenant_id = current_setting('app.tenant_id'));

Set app.tenant_id at the start of each connection via Drizzle's db.execute(sqlSET app.tenant_id = ${tenantId}).

B5 — Silent knowledge failure in customer chat

Severity: Medium
File: src/routes/chat.route.ts (line ~386, catch block in /chat/customer)

If searchKnowledge() throws (Neon timeout, pgvector error, malformed query), the customer chat proceeds without any knowledge context. The user receives a response based only on the system prompt and conversation history — no RAG. No error is surfaced; no log entry indicates degradation.

Fix: Log the error with structured metadata and optionally include a degraded-mode indicator in the response for debugging (stripped in prod). Consider falling back to a "I'll check on that for you" response when knowledge search fails.

B6 — generateTitle() failure leaves conversation title null

Severity: Low
File: src/routes/chat.route.ts (line ~287)

generateTitle() is called fire-and-forget after the chat response. If the Haiku model call fails or times out, the exception is silently swallowed and the conversation title remains null indefinitely. There is no retry.

Fix: Add a fallback title: title = body.message.slice(0, 50) (first 50 chars of the user's first message) if generateTitle() fails.

B7 — No prompt injection safeguard

Severity: Medium
File: src/routes/chat.route.ts, src/routes/knowledge.route.ts

The security guide specifies input sanitization to detect and block prompt injection attempts (e.g., "Ignore previous instructions and..."). No sanitization code exists in brain-service routes. The @repo/brain orchestrator may handle this, but it's unverifiable from the service layer.

Fix: Add a lightweight injection-check before passing user input to the LLM. At minimum, check for common injection patterns; log and return 400 for clear violations.

Areas of Improvement

G1 — No structured logging

Priority: High
File: src/routes/*.ts

All error logging uses console.error() which produces unformatted text in Cloudflare's tail logs. There's no structured log format, no correlation IDs in logs, no log levels, no way to filter by tenant or conversation.

Add structured logging with at least: { level, msg, tenant_id, conversation_id, request_id, error }.

G2 — @repo/brain package is a black box

Priority: High
Docs: docs/ai-brain/agent-orchestration-spec.md

All agent orchestration (orchestrateChat()), memory extraction, tool registry, and LLM fallback logic lives in @repo/brain. There is zero documentation for this package — no README, no jsdoc, no internal spec. Brain-service is a thin HTTP adapter; the actual intelligence is untestable from outside the package.

Action: Write an internal doc for @repo/brain covering: module layout, orchestrateChat() parameters, tool registry shape, memory extraction pipeline, provider selection logic.

G3 — Knowledge ingest is synchronous (blocks request)

Priority: Medium
File: src/routes/knowledge.route.ts (line 94)

ingestDocument() — chunk + embed + insert — runs inside the HTTP request. For large documents, this could take 5–30 seconds and risk Cloudflare's 30-second CPU time limit.

Fix: Queue the ingest job after creating the doc record with status: "processing". Return 202 Accepted immediately. The queue consumer calls ingestDocument() and updates the doc status when done.

G4 — No memory visibility endpoint

Priority: Medium
File: src/routes/ (missing)

The ai_memory table accumulates facts, preferences, episodes, and patterns extracted from conversations. Users have no way to view, edit, or delete these memories. If the brain forms an incorrect belief about the business, there's no correction mechanism.

Add: GET /memory — list memories with type/importance filters. DELETE /memory/:id — allow user to remove incorrect memories.

G5 — No insight delivery push (polling only)

Priority: Medium
File: src/routes/insights.route.ts

Insights are generated in the background (when cron is enabled). Users must poll GET /insights to discover new ones. There's no push notification (WebSocket, SSE, or Cloudflare real-time channel) to alert the user.

Action: When an insight is created, push a notification via the existing Notification Service or a DO-managed WebSocket connection in the dashboard.

G6 — No feedback endpoint for decision outcomes

Priority: Low
File: src/db/schema.ts (ai_decisions.feedback_score)

The ai_decisions table has feedback_score REAL (-1.0 to 1.0) for tracking whether the AI's decisions were useful. There's no API endpoint to submit feedback. The column is defined but will always be null.

Add: POST /conversations/:id/feedback — takes { messageId, score: -1 | 0 | 1, comment? }. Write to ai_decisions.feedback_score and outcome. This enables future fine-tuning.

G7 — No real-time streaming for customer chat

Priority: Low
File: src/routes/chat.route.ts (line 345, /chat/customer)

Owner chat (/chat) streams tokens via SSE. Customer chat returns the entire response as a single JSON body. For long responses, the customer sees nothing for several seconds.

Fix: Add SSE streaming to /chat/customer (same pattern as /chat). The main difference is customer chat doesn't use tools, so the SSE events are simpler (only token and done events needed).

G8 — Plan limits not visible to users

Priority: Low
File: src/routes/usage.route.ts

GET /usage returns limits.max_interactions, used_interactions, and remaining. But there's no endpoint to check the current plan or available plans, and the dashboard doesn't clearly surface "X of Y interactions used this month." A user may not know they're close to their limit until they hit a 402.

Action: Add GET /usage/limits returning the full plan definition (plan name, all limits, period reset date). Surface a usage progress bar in the dashboard.