Research record
- Status
- Planned protocol · no results
- MESA version
- To be pinned before execution
- Dataset
- Not selected; criteria must be frozen first
- Models
- To be declared and version-pinned
- Planned metrics
- Recall@k · nDCG@k · MRR · latency distribution
Known limitations
- No experiment has been executed.
- No dataset size, score, or performance claim is reported.
- The protocol may change before preregistration.
Research question#
Under a frozen legal retrieval corpus and independently judged query set, how do lexical, semantic, and hybrid retrieval differ in relevance and operational cost?
Why this matters#
Search approaches are often compared with broad claims that hide corpus, query, relevance, and configuration choices. A useful comparison must make those choices inspectable.
Dataset#
The corpus will not be described by an invented size. Source eligibility, licensing, temporal cut-off, deduplication, and version identity must be recorded before execution.
Experimental setup#
- Freeze the corpus, query set, judgments, and system configurations.
- Run lexical, semantic, and hybrid conditions against identical eligible documents.
- Keep query construction separate from relevance judgment.
- Record failures and zero-result queries, not only favorable examples.
Results#
No benchmark run has been completed. Tables and charts will be added only after evidence artifacts are available.
Reproducibility#
A publishable run should record dataset identity, query and judgment versions, model identifiers, configuration, code revision, seeds where applicable, environment, raw outputs, and analysis steps.
