MESA CORE v0.9.3 · INSIDE THE ENGINE

How information moves through MESA.

Two paths: turning a source into stored memory, and finding evidence when an application asks a question. Each section below has a UML diagram, the reason behind the design and a link to its decision record.

THE WHOLE SYSTEM AT A GLANCE

One application outside. One storage owner inside.

In the supported V4 topology, one combined process owns the API, the background workers and all three stores. The boxes are modules inside that process, not separate servers.

UML · Component diagramModules, interfaces and stores inside Core
MESA Core · one combined process «component»Your applicationPython SDK · HTTPor an MCP client «component»mesa_api · mesa_mcpauth · session · admission «component»mesa_workersdurable queue consumers «component»mesa_memoryextract · validateembed · build context «component»mesa_storageMemoryDAO façadecanonical writes · search «database»SQLitesource of truth «database»LanceDBvector projection «database»Kùzugraph projection «component»Model providerLLM + embeddingsexternal REST /v4 · MCP enqueue use read · admit write calls 1 · SQL2 · vector3 · graph One owner process writesall three stores. A secondwriter on the same rootis not supported.

Calls point down only: API/MCP → workers → memory → storage (ADR 0013). SQLite decides; LanceDB and Kùzu are projections written after it, in order (ADR 0009).

WHY

SQLite, LanceDB and Kùzu cannot share one transaction, and two writers on the same files make recovery ambiguous. So one process writes, SQLite records every step, and the other stores follow it.

ADR 0009 · Ledger and single owner →

01 · THE WRITE PATH

From exact source text to usable memory.

Your application sends exact text with its document, revision and chunk identity. The API durably admits work and returns 202; processing continues in the background. Whole-document admission may wait briefly for queue room while retaining its chunk plan. Your application asks for the status until it reaches a final state.

UML · Sequence diagramWrite path: accept now, process in the background, poll for the result
«actor»Your app«boundary»API«entity»SQLite ledger«control»Worker«external»Model«entity»LanceDB · Kùzu insert(source chunk) legal items + mutation + run mutation_id 202 · mutation_id 202 = accepted, not processed.Processing continues below. claim run (lease) source chunk SPO only if enabled 0..N facts validate (mode 0–2) accept / reject alt[accepted] facts · SQL_APPLIED VECTOR_APPLIED / SKIPPED GRAPH_APPLIED / SKIPPED state = COMMITTED [rejected] state = REJECTED loop[every 2 s until a terminal state] GET /v4/mutations/{id} read state state final state

Admission snapshots legal items separately from SPO facts. Legal rules default on; model-based SPO extraction defaults off. Each enabled store write follows the previous one; retries reuse idempotent IDs. Mutation polling: 2 s; revision activation: 1 s.

Developer detail · the six steps with their code names
  1. 01

    Check the session and source

    The server derives access scope from the authenticated session. The request identifies a dataset, document, revision and exact source chunk.

    POST /v4/memory/insert
  2. 02

    Accept the work durably

    A candidate, mutation ledger record and pipeline run are created. HTTP 202 confirms admission; it does not confirm completed processing.

    MemoryCandidate · memory_mutations · pipeline_runs
  3. 03

    Keep legal items; optionally extract facts

    Legal rules default on and retain versioned items with original chunk offsets during admission. Model-based subject–relation–object extraction defaults off; enable MESA_EXTRACTION_ENABLED explicitly. Models-off writes retain BM25 search.

    legal_items ≠ SPO · MESA_EXTRACTION_ENABLED=false
  4. 04

    Apply the selected validation

    Mode 0 uses deterministic checks; mode 1 adds one model validator; mode 2 requires two distinct provider/model pairs to agree.

    MESA_TIER3_MODE = 0 / 1 / 2
  5. 05

    Write the stores in order

    Canonical records go to SQLite. Enabled vector and graph lanes then write to LanceDB and Kùzu; disabled lanes end SKIPPED and can be backfilled later.

    SQL_APPLIED → VECTOR_APPLIED → GRAPH_APPLIED
  6. 06

    Confirm completion

    The client checks the durable mutation status. COMMITTED is the successful final state.

    GET /v4/mutations/{id}

WHAT IF A STEP FAILS?

Every mutation ends in a state you can see.

The three stores do not share a single database transaction. Durable work records, expiring ownership leases and idempotent receipts let the system retry or reconcile unfinished work without calling it a success.

UML · State machineOne mutation from QUEUED to a terminal state
QUEUEDRUNNINGEXTRACTEDVALIDATEDPROJECTINGCOMMITTEDDLQRETRY_PENDINGREJECTEDBLOCKEDROLLED_BACK worker claimsfacts or fallbackchecks passSQL writtenlanes complete transient error (any step) [validation rejects] retry retries exhausted replay rollback[cleanup done] rollback [cleanup unfinished] REJECTED keeps the audit source but creates no search data.

COMMITTED means required lanes completed; disabled vector/graph lanes may be SKIPPED. Retries are bounded; exhausted work parks in the dead-letter queue (DLQ) until someone replays it. A failure stays visible instead of being hidden by compensation.

WHY

A failed projection should be visible as retry, DLQ or BLOCKED instead of being hidden behind a compensating write.

ADR 0009 · Ledger and single owner →

02 · THE READ PATH

From a question to source-linked evidence.

The session fixes which datasets may be read. Eligibility rules remove everything else before any ranking. Four lanes search the eligible set and their rankings are merged. With legal rules enabled, scoped citation factors, optional reranking and a bounded exact-match-first stage determine the final order.

UML · Activity diagramRead path: four lanes in parallel, one fused result
Resolve sessionauthorized datasets Filter eligibletenant · time · committed VectorLanceDB · meaningBM25SQLite FTS5 · wordsAssertionSQLite · structured factsGraph pathKùzu · ≤ 3 hops Fuse / rankRRF · legal · rerank Exact match firstif legal on · ≤ 3 hits Pack evidence≤ 2048 tokens fork: each lane runs on the same eligible set · join: one vote per lane per candidate

Scope precedes ranking. Legal factors and optional reranking follow fusion; exact law/article or a decision’s own first-chunk reference may reserve at most three leading positions. Other results retain order. Legal rules off bypasses legal stages; PageRank remains telemetry.

Search returns evidence

Hits keep assertion, document, revision, chunk and lane identity. A graph path keeps its supporting assertions and directions.

Context fits the model’s input

The context endpoint packs evidence once into a token budget. A complete graph proof fits together or not at all; your application’s model then writes the answer.

WHY

Vector, keyword and graph scores live on different scales, so adding them is meaningless. Fusing rank positions avoids that; low graph centrality is never treated as evidence that a fact is false.

ADR 0010 · RRF and PageRank →
Optional adaptive search: when is the question reformulated?

Search and context default to a single query. When adaptive context is explicitly requested, an empty result or insufficient multi-lane support can trigger one planner call and up to two reformulations. Every search keeps the original scope; this does not generate an answer.

ADR 0015 · Adaptive query planning →
Ranking formula and search limits

WHO MAY READ WHAT?

Scope comes from the server. Every fact points to its source.

Tenants own workspaces, workspaces own datasets, datasets own documents. A document has immutable revisions made of exact source chunks, and every assertion points back to the chunks that support it.

UML · Class diagramWho may read what, and where every fact comes from
Principal+ principal_id+ key (scrypt hash) Tenant+ tenant_id Workspace+ workspace_id Dataset+ dataset_id+ purge, rollback grants «enumeration»RoleOWNERWRITERREADER Session+ session_id+ dataset_ids {frozen} Agent+ agent_idpersona, not a boundary Document+ document_id+ title «immutable»Revision+ revision_id SourceChunk+ chunk_id+ exact content «snapshot»LegalItem+ version, type+ start, end+ source quote 0..* Assertion+ subject+ predicate+ object+ state 1..* 1..* 0..* 1..* 1..* **role 10..*starts *1acts as «scope» 1..* datasets 1..*0..*evidence Roles inherit downward.Request metadata nevergrants authority (ADR 0011).

Filled diamonds mean ownership (composition): deleting a dataset removes its documents. Role is an association class on the Principal–Tenant link and inherits down to workspaces and datasets. A session is created by the server and freezes its dataset set. LegalItem snapshots belong to source chunks, remain separate from SPO assertions and are hidden from returned copies when legal rules are off.

WHY

An agent ID describes a persona, not an owner. Treating a client-supplied ID as the boundary made ownership ambiguous, so the server resolves the whole chain itself.

ADR 0011 · Tenant/dataset authorization →

WHERE DOES IT RUN?

One process, one folder of data.

Stores are embedded under one storage root and owned by one combined process. MESA_GRAPH_ENABLED=false avoids opening Kùzu; models-off operation retains BM25 search. No separate database server is required.

UML · Deployment diagramOne process, one storage root, optional model providers
«device»Host · VM or Docker Compose service «execution environment»Python 3.11–3.13 · uvicorn «artifact»mesa · combined runtimeAPI + workers + MemoryDAO {replicas = 1} «volume»MESA_STORAGE_ROOT «artifact»SQLite«artifact»LanceDB«artifact»Kùzu owns «device»Client app · agent · SDK HTTP APIX-API-Key «external»Model providers only if models enabled extraction · validation · embeddings «device»Backup target offline Never point two processesat one storage root.Scale admission, not writers.

The three stores are embedded files under one root, so they are backed up and restored together. Models-off combined operation accepts documents and supports BM25. Legal rules need no model; open-ended SPO extraction requires explicit enablement.

Configured bounds

Configured limits from v0.9.3. Separate DEV measurements and CPU/GPU conditions are documented in the release notes; they do not certify production capacity.

Context budget
2048 tokens by default
Graph search
≤ 20 seeds · ≤ 3 hops · ≤ 500 hits
Graph request timeout
15 s by default
Rank fusion
RRF, k = 60, vector 8.0; other lanes 1.0
Mutation status polling
every 2 s by default
Revision activation polling
every 1 s when waiting
Storage writers
exactly one process per storage root

03 · HOW THE CODE IS ASSEMBLED

Separate modules, explicit responsibilities.

mesa_api / mesa_mcp

Entry points

Receive HTTP or MCP requests and bind them to an authenticated session.

mesa_workers

Background work

Consume durable ingestion, projection and maintenance work.

mesa_memory

Information processing

Own extraction, validation, embeddings and context building.

mesa_storage

Records and search

Own canonical SQLite state, source ownership, projections and retrieval through MemoryDAO.

mesa_client

Application connection

Provides synchronous and asynchronous Python clients for the HTTP boundary.

WHY

Imports may only point down the layers. A CI check rejects reverse imports, so storage code can never depend on the API.

ADR 0013 · Production dependencies →
Read the exact Core sources behind these diagrams

Verified source baseline: Core v0.9.3. Diagrams simplify the runtime.

NEXT STEP

Where to go from here