Functional flow
Click any node for responsibilities, I/O, repo paths, and operational notes. Dual-publish fan-out is the center of the write path.
1 · Manual agent write → index → query
REST source creation returns 202 after registry + publish; MCP write tools also return before workers finish. Indexing is async; query via Search primitives (LLM-free) or Ask (planner → executor → synthesizer).
2 · Slack event ingestion (Socket Mode)
Ops Connector creation stores config only. Deploy the persistent worker via infra/08-deploy-slack-connector.sh. New activity refetches a thread; bulk history backfill is not implemented. Credentials: Secret Manager alexandria-ws-{ws}-connector-{id}-*.
3 · GitHub scheduled / poll ingestion
Ops This is not a webhook. Connector creation stores config only; deploy via infra/09-deploy-github-connector.sh, trigger the first /sync, and schedule later calls. Initial sync is capped at 200 eligible files. Live PR listing goes through ACL-filtered connectors (github-acl.ts).
4 · Fan-out pipeline (source-ingested · document-ready · evidence-ready)
Shipped Embeddings listens only on evidence-ready. Legacy sub-embeddings on source-ingested is deleted by infra/01-pubsub-subscriptions.sh.
5 · Evidence records + embeddings + Pinecone / FTS
Partial VECTOR_BACKEND=pgvector adapter exists for benchmarks; production default is Pinecone. See infra/10-vector-notes.md.
6 · Wiki maintainer + lint
7 · Search vs Ask
MCP/REST: search, search_context, search_code, recent_prs, and wiki tools. MCP has no single Ask tool; clients orchestrate.
8 · Projects + ACL
Projects never grant access. Visibility: workspace | restricted (grants) | project (only when attached). Agents without userId see workspace-visible sources only. Standalone MCP search_code, recent_prs, and fetch_source have no project_id argument; Ask passes project scope to its retrievers.
Component architecture
Deployable surfaces and shared libraries. Click a component for detail.
- apps/web — browse, ask, settings, billing
- Auth: Supabase Google + email
- Calls API with JWT + x-workspace-id
- apps/api — Hono REST + MCP at /mcp
- Same core for agents and UI
- Answer path under services/answer/
- Pub/Sub, GCS, Voyage, vector adapter
- retrieval/hybrid.ts, scope, RRF
- OTel helpers + return modes
- apps/workers/evidence
- Processors: prose, code, slack
- Publishes evidence-ready
- apps/workers/embeddings
- Rejects non-evidence-ready payloads
- Upserts vectors + compatibility chunks; evidence rows generate FTS
- apps/workers/wiki
- Ordered subscription per workspace
- Indexes wiki sections as evidence
- apps/workers/connectors/slack
- Socket Mode + admin sync-thread
- apps/workers/connectors/github
- Sync endpoint; dual-publishes ingest
- GCS ws/<id>/{raw,wiki}/
- Supabase registry + FTS + RLS
- Pinecone namespace = workspace
MCP tools used by agents Shipped
| Tool | Role | Click |
|---|---|---|
add_source / write_note | Write into registry; triggers fan-out | |
search / search_context | Hybrid evidence search + citations | |
search_code | Code evidence / symbols | |
recent_prs | Live PRs from allowed GitHub connectors | |
fetch_source / fetch_skill | Full content with return_mode | |
read_wiki_page / search_wiki | Wiki navigation + grep | |
list_skills | Discover ingestion / workflow skills |
Example agent journeys
Concrete end-to-end paths an MCP agent (or human via Ask) would take today.
“Store that knowledge” Shipped write_note / add_source
Agent captures a decision, meeting note, or distilled answer so it compounds for later retrieval and wiki synthesis.
-
Authenticate with Authorization: Bearer alx_… (API key) or user JWT.
-
Call write_note (or add_source with tier=managed, source_type=note) with title + Markdown content and optional tags.
-
API writes raw + normalized text to GCS, inserts sources + documents, and dual-publishes fan-out. REST creation returns 202; MCP returns its tool result before workers finish.
-
Evidence worker builds prose evidence; embeddings indexes vectors + FTS; wiki maintainer updates entity/source pages asynchronously.
-
Later: search_context finds the note as cited evidence; or Ask synthesizes it with related sources.
“Index this codebase” Shipped Ops GitHub connector
Workspace admin configures a GitHub connector; sync materializes repo files as sources and feeds the same evidence pipeline.
-
Create/configure connector (admin) with repos + a least-privilege PAT stored through Secret Manager. This does not deploy or trigger the worker. Attach the connector to a project for relevance scoping.
-
Trigger sync: POST worker /sync (or schedule that endpoint). The initial run processes at most 200 eligible files per repository; later runs compare commits.
-
Each file path calls publishSourceIngested → evidence code processor (symbols / multi-granularity) → embeddings.
-
Agent queries with search_code (path_glob optional) or Ask planner selects search_code + recent_prs.
- ACL: live PR results map onto allowed repo sources via resolveAllowedGithubConnectors — restricted repos stay hidden.
“Get all relevant marketing context” Shipped project scope + Ask / MCP
Use a Marketing project (sources + Slack/blog connectors) so relevance is narrowed; ACL still filters restricted sources.
-
Resolve active project: explicit project_id or user’s default_project_id.
-
MCP path: start with read_wiki_page("index.md") for orientation, then search with project_id and tags like marketing; fetch top sources with llm_extract.
-
Ask path (web): planner may pick search + search_wiki; executor fans out under ACL ∩ project; synthesizer returns cited narrative + caveats; logged to query_runs.
-
Slack marketing channels (if connector attached to the project) contribute thread evidence via the Slack processor facets.
- Optional: agent write_note files the synthesis back so the next query compounds.