MESA Core · v0.9.3

Troubleshooting

v0.9.3 source: docs/RUNBOOK.md

This guide has equivalent English and Turkish versions. The source sections behind this page are at the bottom.

Purpose and boundaries#

Check initialization separately from authenticated readiness. Inspect capability flags before claiming vector or graph availability. Models-off combined operation supports ingestion and BM25 search. A 503 canonical_processing_unavailable error indicates the combined consumer is unavailable; 202 still needs mutation polling. Inspect pipeline/projection receipts for retryable failures. Do not delete locks, bypass tenant checks or replay semantic rejection as a generic recovery step.

Continue with the maintained contracts#

Use the pinned source reference below for complete tables and operator details. Related guides: architecture, API, operations and release limits.

Diagrams

UML · State machineOne mutation from QUEUED to a terminal state
QUEUEDRUNNINGEXTRACTEDVALIDATEDPROJECTINGCOMMITTEDDLQRETRY_PENDINGREJECTEDBLOCKEDROLLED_BACK worker claimsfacts or fallbackchecks passSQL writtenlanes complete transient error (any step) [validation rejects] retry retries exhausted replay rollback[cleanup done] rollback [cleanup unfinished] REJECTED keeps the audit source but creates no search data.

COMMITTED means required lanes completed; disabled vector/graph lanes may be SKIPPED. Retries are bounded; exhausted work parks in the dead-letter queue (DLQ) until someone replays it. A failure stays visible instead of being hidden by compensation.

All diagrams with their design rationale: Architecture →

Verified sources
Source reference for this page

The sections of the upstream file that back this page, translated to English where the original mixes in Turkish. The GitHub link shows the exact original.

Troubleshooting#

Capture version/commit, runtime profile, content-free health, mutation/operation

IDs and error class. Never include keys, claim tokens or full source/query text.

SymptomCheckSafe response
Missing runtime profile/rootExplicit environment and CLI helpUse a dedicated writable root, never repo/home/filesystem root
401Key registry, active principal, key rotationRe-provision/reload authorized credentials; do not disable auth
403/session conflictACL, session principal and dataset/agent bindingGrant intended scope offline; start a properly scoped session
Insert 422Required title/IDs, character bounds and contentCompare actual OpenAPI schema; keep exact source
Insert 409Frozen revision, identity/idempotency conflictReuse exact retry payload or create a new revision
Embedding 409/mismatchFull provider/model/revision/dimension/normalization identityFollow offline migration/rebuild; equal dimension is not equal space
503 vector_backend_unavailableProvider availability and operational identityRestore the selected provider; no silent model switch
503 graph_backend_unavailableHealth, timeout, Kùzu schema/parityDiagnose query/backend and bounded traversal; do not advertise complete results
503 maintenance_pendingActive/retryable rebuild statusFollow rebuild runbook; no manual lease/queue deletion
Mutation stalls or enters DLQProjection predecessor, fenced claim, provider errorsRepair cause, then authorized replay if state permits
Semantic rejectionRejection reason and validation policyTreat as terminal; generic replay returns NON_REPLAYABLE
Cleanup BLOCKEDStore availability and shared ownershipPreserve evidence, repair store, run supported recovery
Readiness 503Worker liveness, thresholds, canonical backlogResolve health; process alive does not mean ready
FTS/parity errorMatching Alembic head and checksummed backupStop writers and rehearse recovery into a new root
Old Kùzu timestamp/schema errorRelease code and offline schema migrationUpgrade/rehearse from backups; do not issue ad hoc ALTER or wipe
Growing RSS/diskLance fragments/versions, queues and retained generationsMeasure representative workload; apply supported maintenance, reserve capacity

Inspect /v4/capability before assuming semantic/graph availability. A model-disabled

profile intentionally lacks semantic generation. wait_until_committed() can

return failed terminal states; inspect state explicitly.

CLI --help, configuration, API,

operations and migration describe supported

interfaces. Avoid raw SQL edits, in-place restore, forced queue completion and

unbounded retries. Real provider or production-size evidence unavailable in an

audit must remain NOT EXECUTED, never converted to a simulated success.