Research record

Status
Planned protocol · no results
MESA version
To be pinned before execution
Dataset
Not selected; criteria must be frozen first
Models
To be declared and version-pinned
Planned metrics
Recall@k · nDCG@k · MRR · latency distribution

Known limitations

  • No experiment has been executed.
  • No dataset size, score, or performance claim is reported.
  • The protocol may change before preregistration.

Research question#

Under a frozen legal retrieval corpus and independently judged query set, how do lexical, semantic, and hybrid retrieval differ in relevance and operational cost?

Why this matters#

Search approaches are often compared with broad claims that hide corpus, query, relevance, and configuration choices. A useful comparison must make those choices inspectable.

Dataset#

Not selected

The corpus will not be described by an invented size. Source eligibility, licensing, temporal cut-off, deduplication, and version identity must be recorded before execution.

Experimental setup#

  • Freeze the corpus, query set, judgments, and system configurations.
  • Run lexical, semantic, and hybrid conditions against identical eligible documents.
  • Keep query construction separate from relevance judgment.
  • Record failures and zero-result queries, not only favorable examples.

Results#

No results yet

No benchmark run has been completed. Tables and charts will be added only after evidence artifacts are available.

Reproducibility#

A publishable run should record dataset identity, query and judgment versions, model identifiers, configuration, code revision, seeds where applicable, environment, raw outputs, and analysis steps.