MESA Core · v0.9.3

Core mimarisi

v0.9.3 kaynak: docs/architecture-v4.md

Bu rehberin Türkçe ve İngilizce sürümleri aynı konuyu açıklar. Bu sayfayla ilgili kaynak bölümleri en altta yer alır.

Asıl kayıtlar SQLite’ta tutulur#

Kaynak kimliği, değiştirilemeyen belge sürümleri, bilgi kayıtları, değişikliklerin durumu ve çıktıların sahipliği SQLite’ta tutulur. LanceDB vektör araması, Kùzu ise ilişkiler üzerinde arama için kullanılır. Bu iki sistem, bir bilginin hâlâ geçerli olduğuna veya kimlerin erişebileceğine kendi başına karar vermez.

Veri alımı ve kurtarma nasıl ilerler?#

Bir istek kabul edildiğinde, yapılacak iş önce kalıcı olarak kaydedilir. Ardından bilgi çıkarımı ve seçilen doğrulamalar yapılır; kayıtlar SQL → vektör → çizge sırasıyla yazılır. Üç depo tek bir ACID işlemini paylaşmaz. Bu nedenle işin kime ait olduğu, hangi adımların tamamlandığı ve yeniden denendiğinde ne yapılacağı ayrıca izlenir. Yazma ancak COMMITTED durumuna ulaştığında tamamlanmış sayılır.

Ayrıntıları nerede bulabilirsiniz?#

Mimari sayfası veri akışını, depoları, aramayı ve kaynak izini ayrı ayrı açıklar. Uygulama ayrıntıları için veri alımı, arama, güvenlik ve işletim rehberlerine bakın.

Eşzamanlı arama ve kapsam önbelleği#

MESA_SEARCH_SCOPE_CACHE=true, kapsama özel geçici FTS indekslerini ve satır/metin özelliklerini dört kapsamlık LRU önbelleğinde yeniden kullanır. SQLite commit değişiklikleri önbelleği geçersiz kılar; her istekte yetkilendirme yeniden denetlenir. Sorgu embedding’i arama kilidinden önce hesaplanır; yeniden sıralama bağlantı bırakıldıktan sonra çalışır. scrypt doğrulama, anahtar oluşturma ve döndürme işlemleri event loop dışında yürütülür; yetki sonuçları önbelleğe alınmaz. Çizge kapalıysa başlangıç düğümlerinin hazırlığı atlanır.

Şemalar

UML · Bileşen şemasıCore içindeki modüller, arayüzler ve depolar
MESA Core · tek birleşik süreç «component»UygulamanızPython SDK · HTTPveya MCP istemcisi «component»mesa_api · mesa_mcpkimlik · oturum · kabul «component»mesa_workerskalıcı kuyruk işçileri «component»mesa_memoryçıkar · doğrulavektörle · bağlam kur «component»mesa_storageMemoryDAO cephesiasıl yazma · arama «database»SQLiteasıl kayıt «database»LanceDBvektör görünümü «database»Kùzuçizge görünümü «component»Model sağlayıcıLLM + embeddingharici REST /v4 · MCP kuyruk kullanır oku · kabul yaz çağrı 1 · SQL2 · vektör3 · çizge Üç depoya yalnızcabu süreç yazar. İkincibir yazıcı aynı kökükullanamaz.

Çağrılar yalnızca aşağı yönlüdür: API/MCP → işçiler → hafıza → depolama (ADR 0013). Kararı SQLite verir; LanceDB ve Kùzu ondan sonra sırayla yazılan görünümlerdir (ADR 0009).

UML · Dağıtım şemasıTek süreç, tek depolama kökü, isteğe bağlı model sağlayıcılar
«device»Sunucu · VM veya Docker Compose servisi «execution environment»Python 3.11–3.13 · uvicorn «artifact»mesa · birleşik çalışmaAPI + işçiler + MemoryDAO {replicas = 1} «volume»MESA_STORAGE_ROOT «artifact»SQLite«artifact»LanceDB«artifact»Kùzu sahip «device»İstemci uygulama · ajan · SDK HTTP APIX-API-Key «external»Model sağlayıcıları yalnız model açıksa çıkarım · doğrulama · embedding «device»Yedek hedefi çevrimdışı İki süreci aynı depolamaköküne bağlamayın.Kabulü ölçekleyin, yazıcıyı değil.

Üç depo tek kök altındaki gömülü dosyalardır; birlikte yedeklenir ve geri yüklenir. Modeller kapalı combined çalışma belgeleri kabul eder ve BM25 araması sunar. Hukuk kuralları model istemez; açık uçlu SPO çıkarımı ayrıca açılır.

Tüm şemalar ve gerekçeleri: Mimari →

Kendi sayfasına taşınan bölümler
Kaynak belgedeki ilgili bölümler

Bu bölüm, kaynak belgenin bu sayfayla ilgili kısımlarını değiştirmeden gösterir. Kaynak içindeki bir bölüme giden bağlantıyı açtığınızda metin otomatik olarak görünür.

Package version: 0.9.3, without production certification.

This document owns the current V4 design; production qualification remains

release candidate / NO-GO under the release policy.

The root architecture entry point links here. The obsolete

V3 whitepaper is available in Git history; see the

ADR index for decision applicability.

Module ownership#

ModuleResponsibility
mesa_storageCanonical SQLite, catalog repositories, ownership, retrieval and projection providers
mesa_memoryRuntime configuration, extraction/validation, embedding, context, security and rebuild orchestration
mesa_workersDurable ingestion, projection and maintenance consumers
mesa_apiHTTP schemas, authorized V3/V4 routes and control API
mesa_clientSync/async HTTP clients and optional integrations
mesa_mcpMCP service adapters, gateway, bindings and client lifecycle
mesa_evalsLocal repository diagnostics and deterministic retrieval contract tooling (not packaged in wheel)

Higher layers may depend on lower layers: API/MCP → workers → memory → storage.

Runtime composition roots are explicit exceptions; the source guard is

`check_layer_imports.py`. MemoryDAO

remains the compatibility façade; catalog transactions have a focused repository

boundary. See ADRs 0012 and

0013.

Runtime boundary#

V4 has one supported storage-writing topology: one combined process owns

SQLite, LanceDB and Kùzu. API admission and background consumers run in that

process. A second writer must never mount the same storage root.

V3 remains available as the lexical-core compatibility surface and keeps its

existing API/worker topology. V3 and v4 must not write the same physical

storage directories concurrently.

Canonical v4 fact extraction is owned by FactExtractionService.

Security and catalog hierarchy#

The v4 security boundary is a tenant, not an agent:

text
Principal
  └── Tenant
      └── Workspace
          └── Dataset
              └── Document
                  └── immutable DocumentRevision
                      └── SourceChunk

Roles are OWNER, WRITER and READER at tenant, workspace or dataset

scope. Roles inherit downward. Purge and rollback require explicit dataset

permissions. Agent IDs represent compute/persona context and do not grant

tenant access.

An authenticated session is generated by the server and binds one principal,

tenant, workspace, agent and immutable set of datasets. Insert, search,

context, status, replay and rollback derive their scope from this record.

API credentials use key_id.secret. Only the salted scrypt digest is stored;

plaintext is returned once by mesa-v4-admin issue-key. Multiple keys may

belong to one or more principals and can be revoked or rotated independently.

Canonical ingestion#

POST /v4/memory/insert durably admits exact source content and creates:

  • one MemoryCandidate;
  • one memory_mutations ledger row;
  • one pipeline_runs row plus append-only events;
  • ordered SQL, vector and graph outbox lanes.

The candidate explicitly carries tenant, workspace, dataset, document,

revision, chunk, source reference, evidence span, agent/session, pipeline,

extraction and embedding versions. Authorization fields are never read from

free-form metadata.

An authenticated dataset writer may additionally supply the typed

legal_identity object (required law_id and article_id; optional

statute_code and version/paragraph/clause identifiers). MESA validates it against the bounded

legal alias ontology, binds it to the immutable mutation provenance and stores

it on canonical assertions. Reserved _mesa_* keys in caller metadata are

always discarded, so free-form metadata cannot opt into legal ranking. The

producer remains responsible for verifying that the declared identity belongs

to the source; MESA_Data must not populate it from a bare article heading.

Validation is an explicit policy gate configured via MESA_TIER3_MODE (0 = deterministic-only, 1 = single LLM validator, 2 = dual-LLM consensus). A rejected mutation may keep its raw

audit source, but it cannot create an active SQL, vector, entity, edge or

assertion artifact. In Mode 0, deterministic admission checks gate progression with zero LLM validator dependencies.

Pipeline and failure model#

The legal success path is:

text
QUEUED → RUNNING → EXTRACTED → VALIDATED → PROJECTING → COMMITTED

Failure/recovery states are RETRY_PENDING, DLQ, ROLLING_BACK,

ROLLED_BACK and BLOCKED. State changes use compare-and-swap and append an

event. Claims use a random fencing token, bounded lease, heartbeat and bounded

retry. A worker that loses its token cannot finalize the claim.

V4 retries restart extraction from RETRY_PENDING, skip model calls after

validation, and atomically mark the dispatch and any not-yet-validated mutation

DEAD_LETTER and the raw log failed after five unsuccessful attempts,

preserving final states.

Failed dispatch and projection attempts back off for 60, 120, 240 and 480 seconds,

capped at 900 seconds, before becoming eligible for another claim.

At startup, the combined runtime holds the storage writer lock and returns

abandoned processing raw logs to DEFERRED, clearing their claims.

Operator replay also resets an extraction dead letter's dispatch attempts and

raw-log claim, making it eligible to run immediately.

These are pipeline states; client-facing mutation states are listed in

concepts and ingestion. COMMITTED is the success

state; receiving an admission response alone does not establish completion.

Projection prerequisites are enforced in the database:

text
SQL → vector → graph

The projector only claims a later lane when its predecessor is complete.

Every lane is idempotent by mutation and physical artifact identity.

Legal citation retrieval selects trusted typed identities through the scoped

compound assertion index before lane ranking. Tenant, dataset, agent,

jurisdiction, validity and committed-state predicates are applied in that SQL

selection. Typed identity refines candidate recall and lane order; it does not

bypass deterministic weighted RRF or replace canonical evidence text.

Artifact ownership and rollback#

artifact_registry represents physical SQL, vector and graph artifacts.

artifact_sources allows several mutations/source chunks to own the same

entity or vector.

Rollback removes only the requesting run's source ownership. A shared artifact

remains active while another source owns it. The last-owner path writes the

SQLite retrieval tombstone first and schedules idempotent vector/graph cleanup.

An unfinished cleanup leaves the run BLOCKED, never falsely committed.

A new document revision does not erase history. Prior assertions become

SUPERSEDED. A legal purge retracts the requested source and removes only

artifacts that have no remaining active owner.

Reconciliation and health#

Reconciliation is bidirectional:

  • ledger artifact missing from a store: requeue the missing lane;
  • physical artifact without registry/ownership: quarantine it in the registry

and enqueue cleanup.

Authenticated /health returns the aggregate fields

status, maintenance and rebuild_state. /health/init reports readiness

and fails with HTTP 503 when required stores, workers or backlog checks fail.

Projection backlog, dead letter, stuck leases, cleanup backlog, blocked cleanup,

ownerless registry rows and shared-owner counts are aggregate gauges at /metrics;

they are not individual fields in the public /health response.

Retrieval#

All SQL, vector and graph lanes receive the authorized dataset filter before

ranking. V4 combines rank positions using real reciprocal rank fusion. Jurisdiction and validity constrain eligibility before ranking; assertion

confidence affects assertion-lane ordering. The bounded post-RRF legal factor

uses query statute/article identity. See retrieval. Candidate assertion identity is the deterministic final tie-break.

Responses include document, revision, chunk, evidence, assertion, pipeline and

embedding provenance. PageRank is not a retrieval eligibility signal.

Bounded adaptive context retrieval#

The raw V4 search endpoint remains the deterministic single-query primitive.

The context endpoint and mesa_get_context default to retrieval_mode=single.

Callers may explicitly select adaptive, which runs the original query first

and invokes one semantic reformulation call only when the result is empty or

none of its top three candidates has support from at least two retrieval lanes.

The planner receives only the normalized user query and may return at most two

validated query strings. It cannot change tenant, agent, dataset,

jurisdiction, temporal or graph scope. New statute/article citations are

rejected. Every expansion reuses MemoryDAO.search_v4_memory; query rankings

are fused with deterministic RRF weights Q0=1.0, Q1=0.5 and Q2=0.5, with a

Q0-preserving tie break. Inner lane scores and provenance remain unchanged,

and outer query_origins plus query_fusion_score remain separately auditable.

An adaptive request performs at most one planner call and three total

retrievals. Planner or optional expansion failures preserve the original

retrieval. The deduplicated final candidate list enters the existing

ContextBuilder once, so the 2048-token default and atomic proof packing remain

the sole model-visible budget authority. This feature performs no answer

generation, recursive planning, decomposition or reranking.

Migration and release#

V3 and V4 use separate storage roots. Moving V3 content into V4 requires an

explicit source/catalog mapping and ingestion through the V4 contract; there

is no general automatic V3-to-V4 converter documented here. The offline

projection runner rebuilds derived stores from an existing V4 canonical ledger,

not a new ledger from raw V3 data.

For V4 upgrades use migration,

backup/restore and the

projection rebuild procedure. Keep the prior root and

matching code/config for rollback.

Production remains NO-GO until migration/restore, v4 Compose, crash,

saturation, cross-tenant concurrency, benchmark and 24-hour soak gates all

pass with production-like storage and the selected model provider.

Canonical V4 insertion requires the combined processing consumer. With models

off, it stores records with a SKIPPED VECTOR lane and finds them through BM25.

Enabling embedding later backfills skipped lanes without hiding committed records.