Search returns evidence
Hits keep assertion, document, revision, chunk and lane identity. A graph path keeps its supporting assertions and directions.
MESA CORE v0.9.3 · INSIDE THE ENGINE
Two paths: turning a source into stored memory, and finding evidence when an application asks a question. Each section below has a UML diagram, the reason behind the design and a link to its decision record.
THE WHOLE SYSTEM AT A GLANCE
In the supported V4 topology, one combined process owns the API, the background workers and all three stores. The boxes are modules inside that process, not separate servers.
Calls point down only: API/MCP → workers → memory → storage (ADR 0013). SQLite decides; LanceDB and Kùzu are projections written after it, in order (ADR 0009).
SQLite, LanceDB and Kùzu cannot share one transaction, and two writers on the same files make recovery ambiguous. So one process writes, SQLite records every step, and the other stores follow it.
ADR 0009 · Ledger and single owner →01 · THE WRITE PATH
Your application sends exact text with its document, revision and chunk identity. The API durably admits work and returns 202; processing continues in the background. Whole-document admission may wait briefly for queue room while retaining its chunk plan. Your application asks for the status until it reaches a final state.
Admission snapshots legal items separately from SPO facts. Legal rules default on; model-based SPO extraction defaults off. Each enabled store write follows the previous one; retries reuse idempotent IDs. Mutation polling: 2 s; revision activation: 1 s.
The server derives access scope from the authenticated session. The request identifies a dataset, document, revision and exact source chunk.
POST /v4/memory/insertA candidate, mutation ledger record and pipeline run are created. HTTP 202 confirms admission; it does not confirm completed processing.
MemoryCandidate · memory_mutations · pipeline_runsLegal rules default on and retain versioned items with original chunk offsets during admission. Model-based subject–relation–object extraction defaults off; enable MESA_EXTRACTION_ENABLED explicitly. Models-off writes retain BM25 search.
legal_items ≠ SPO · MESA_EXTRACTION_ENABLED=falseMode 0 uses deterministic checks; mode 1 adds one model validator; mode 2 requires two distinct provider/model pairs to agree.
MESA_TIER3_MODE = 0 / 1 / 2Canonical records go to SQLite. Enabled vector and graph lanes then write to LanceDB and Kùzu; disabled lanes end SKIPPED and can be backfilled later.
SQL_APPLIED → VECTOR_APPLIED → GRAPH_APPLIEDThe client checks the durable mutation status. COMMITTED is the successful final state.
GET /v4/mutations/{id}WHAT IF A STEP FAILS?
The three stores do not share a single database transaction. Durable work records, expiring ownership leases and idempotent receipts let the system retry or reconcile unfinished work without calling it a success.
COMMITTED means required lanes completed; disabled vector/graph lanes may be SKIPPED. Retries are bounded; exhausted work parks in the dead-letter queue (DLQ) until someone replays it. A failure stays visible instead of being hidden by compensation.
A failed projection should be visible as retry, DLQ or BLOCKED instead of being hidden behind a compensating write.
ADR 0009 · Ledger and single owner →02 · THE READ PATH
The session fixes which datasets may be read. Eligibility rules remove everything else before any ranking. Four lanes search the eligible set and their rankings are merged. With legal rules enabled, scoped citation factors, optional reranking and a bounded exact-match-first stage determine the final order.
Scope precedes ranking. Legal factors and optional reranking follow fusion; exact law/article or a decision’s own first-chunk reference may reserve at most three leading positions. Other results retain order. Legal rules off bypasses legal stages; PageRank remains telemetry.
Hits keep assertion, document, revision, chunk and lane identity. A graph path keeps its supporting assertions and directions.
The context endpoint packs evidence once into a token budget. A complete graph proof fits together or not at all; your application’s model then writes the answer.
Vector, keyword and graph scores live on different scales, so adding them is meaningless. Fusing rank positions avoids that; low graph centrality is never treated as evidence that a fact is false.
ADR 0010 · RRF and PageRank →Search and context default to a single query. When adaptive context is explicitly requested, an empty result or insufficient multi-lane support can trigger one planner call and up to two reformulations. Every search keeps the original scope; this does not generate an answer.
ADR 0015 · Adaptive query planning →WHO MAY READ WHAT?
Tenants own workspaces, workspaces own datasets, datasets own documents. A document has immutable revisions made of exact source chunks, and every assertion points back to the chunks that support it.
Filled diamonds mean ownership (composition): deleting a dataset removes its documents. Role is an association class on the Principal–Tenant link and inherits down to workspaces and datasets. A session is created by the server and freezes its dataset set. LegalItem snapshots belong to source chunks, remain separate from SPO assertions and are hidden from returned copies when legal rules are off.
An agent ID describes a persona, not an owner. Treating a client-supplied ID as the boundary made ownership ambiguous, so the server resolves the whole chain itself.
ADR 0011 · Tenant/dataset authorization →WHERE DOES IT RUN?
Stores are embedded under one storage root and owned by one combined process. MESA_GRAPH_ENABLED=false avoids opening Kùzu; models-off operation retains BM25 search. No separate database server is required.
The three stores are embedded files under one root, so they are backed up and restored together. Models-off combined operation accepts documents and supports BM25. Legal rules need no model; open-ended SPO extraction requires explicit enablement.
Configured limits from v0.9.3. Separate DEV measurements and CPU/GPU conditions are documented in the release notes; they do not certify production capacity.
03 · HOW THE CODE IS ASSEMBLED
mesa_api / mesa_mcpReceive HTTP or MCP requests and bind them to an authenticated session.
mesa_workersConsume durable ingestion, projection and maintenance work.
mesa_memoryOwn extraction, validation, embeddings and context building.
mesa_storageOwn canonical SQLite state, source ownership, projections and retrieval through MemoryDAO.
mesa_clientProvides synchronous and asynchronous Python clients for the HTTP boundary.
Imports may only point down the layers. A CI check rejects reverse imports, so storage code can never depend on the API.
ADR 0013 · Production dependencies →docs/architecture-v4.mddocs/concepts-and-ingestion.mdmesa_memory/extraction/service.pymesa_workers/projection_worker.pymesa_storage/dao.pymesa_storage/retrieval_scope.pymesa_storage/legal_rules.pymesa_storage/legal_identity.pymesa_memory/context_builder.pyVerified source baseline: Core v0.9.3. Diagrams simplify the runtime.