Beyond Vector Similarity: Designing Domain-Aware Retrieval for Legal RAG

Vera Civic is an AI-powered civic information platform that helps people find and understand Nigerian law, regulation, case laws, and current affairs. To answer questions with relevant supporting sources, Vera retrieves material from a corpus containing legislation, regulations, judicial decisions, explanatory content, and news. The first version of the retrieval pipeline followed a conventional RAG architecture:

1. Embed the user's question.
2. Retrieve semantically similar passages.
3. Use those passages to ground the generated answer.

This architecture was effective at identifying passages about the same subject as a query. It was not sufficient for determining which sources were legally authoritative, currently applicable, or appropriate for the user's time frame.

The limitation became visible when the Nigeria Data Protection Regulation 2019 ranked above material from the framework that replaced it: the Nigeria Data Protection Act 2023 and its subsequent implementation guidance.

The result was semantically reasonable. These instruments regulate the same domain and contain closely related terminology. A vector model should consider them similar. The result was nevertheless unsuitable for a present-day legal question.

This exposed the central limitation in the original design:

Semantic similarity measures textual relevance. It does not establish legal authority.

Improving Vera's retrieval quality therefore required more than changing the embedding model or adjusting a similarity threshold. It required a domain-aware ranking system that could combine semantic similarity with legal status, temporal context, exact terminology, source authority, and document type.

Retrieval architecture

Vera stores statutes, regulations, cases, explanatory material, and news in one Qdrant collection. Development environments can use an embedded local database, while production connects to a hosted Qdrant cluster.

@lru_cache
def client() -> QdrantClient:
    settings = get_settings()

    if settings.qdrant_cluster:
        return QdrantClient(
            url=settings.qdrant_cluster,
            api_key=settings.qdrant_key or None,
        )

    return QdrantClient(path=settings.qdrant_path)

Documents are embedded with BAAI/bge-small-en-v1.5 through FastEmbed. The model produces 384-dimensional vectors and can run locally without depending on an external embedding API.

Local embeddings allow the corpus to be rebuilt and evaluated repeatedly without per-token charges. They also introduce operational constraints that affect the ingestion design.

Controlling memory during ingestion

FastEmbed's default batch behavior produced large temporary allocations during full-corpus ingestion. ONNX Runtime also retained some allocated memory after individual batches completed. On a small server, this could cause swapping or terminate ingestion with an out-of-memory error.

Vera therefore uses a configurable, smaller embedding batch size:

batch_size = get_settings().embed_batch_size  # default: 32

This reduces throughput but makes memory consumption more predictable. For an ingestion process that runs periodically rather than on every request, reliability is more important than maximum throughput.

Preserving query-document asymmetry

The BGE embedding model uses different representations for documents and search queries. FastEmbed exposes this distinction through separate embed() and query_embed() methods.

Using the document method for a query does not necessarily produce an exception or an obviously invalid vector. It instead causes a quiet reduction in retrieval quality. This kind of error can remain undetected unless retrieval recall is evaluated with representative queries.

Vera uses embed() when indexing source passages and query_embed() when processing user questions.

Repeatable and safe ingestion

Each source document is divided into sections, and every resulting section is stored as an individual Qdrant point. The ingestion layer must ensure that repeated ingestion produces a collection that accurately reflects the current source documents.

Deterministic point identifiers

Each point identifier is derived from the source and the section's position. Re-ingesting the same document therefore produces the same identifiers and replaces existing points instead of creating duplicates.

Deterministic identifiers alone do not handle documents that become shorter. If an old version produced 20 sections and a new version produces 17, upserting the new document replaces the first 17 points while leaving three obsolete points in the collection.

Before re-ingesting a document, Vera deletes every existing point associated with that source:

client().delete(
    collection_name=settings.collection,
    points_selector=Filter(
        must=[
            FieldCondition(
                key="source",
                match=MatchValue(value=doc.source),
            )
        ]
    ),
)

The deletion is scoped to one source, after which the current sections are inserted with deterministic identifiers. The effective ingestion operation is replacement rather than a blind upsert.

Chunking within the embedding window

Embedding models have finite input windows. When a statutory section exceeds that limit, the remaining content may be silently truncated.

This is especially risky in legal material. The omitted text may contain an exception, qualification, definition, enforcement condition, or operative subsection. A vector produced from only the beginning of a provision may no longer represent that provision accurately.

Vera therefore splits oversized sections before embedding them. Chunking is not merely a storage decision: it determines which parts of a legal rule can be retrieved and supplied to the answer-generation stage.

Embedding retrieval-oriented text

The passage displayed to a user is not always the best representation to embed.

Case law illustrates this problem. A stored passage may contain a prose summary while the user searches using a case name, citation, court, central holding, or specific legal principle. If only the narrative summary is embedded, important identifiers may have little influence on retrieval.

For case-law documents, Vera constructs a richer embedding input by prepending identifying information:

def _embed_text(doc, section) -> str:
    if doc.kind == "case_law":
        heading = " — ".join(
            part
            for part in (doc.source_title, section.section)
            if part
        )

        return (
            f"{heading}\n{section.content}"
            if heading
            else section.content
        )

    return section.content

This creates two related representations: a retrieval-oriented representation containing identifying context and a display-oriented representation containing the passage shown to the user.

Deciding what to embed is part of search-system design. It should not be treated as a mechanical conversion of stored text into vectors.

Why vector similarity is insufficient for legal retrieval

Dense embeddings identify semantic relationships. They do not inherently understand the legal lifecycle of an instrument.

A repealed regulation and the statute that replaced it may regulate the same subject, use the same vocabulary, describe similar obligations, and answer similar natural-language questions. The embedding model can correctly assign both documents high similarity scores without knowing which one currently governs.

The original corpus did not represent this distinction strongly enough. Legal documents had been modeled primarily as searchable passages rather than instruments whose authority changes over time.

The vector database was not producing an incorrect similarity calculation. The application was asking similarity to answer an authority question without supplying authority data.

Modeling legal status as structured metadata

Removing superseded material would prevent it from outranking current law, but it would also make historical research incomplete. A question such as "What data-protection rules applied in Nigeria in 2021?" legitimately requires older law.

The correct approach is to retain superseded documents while representing their status explicitly. Relevant metadata can include:

effective_from
superseded_on
superseded_by
jurisdiction
instrument_type
authority

These fields allow the ranking layer to distinguish between an instrument currently in force, a superseded instrument returned for a present-day question, and a superseded instrument relevant to an explicitly historical question.

For present-day questions, superseded material receives a substantial ranking penalty. For historical questions, the penalty is reduced or omitted when the instrument applied during the requested period.

if hit.superseded_on:
    if asks_about_past:
        score -= HISTORICAL_SUPERSEDED_PENALTY
    else:
        score -= SUPERSEDED_PENALTY

In one evaluation, an obsolete result led the relevant current result by 0.038 in raw vector similarity. That measurement established the size of the observed failure: a negligible adjustment would not reliably reorder closely related instruments.

The legal-status penalty was therefore selected using actual result distributions rather than an arbitrary theoretical value.

Separating similarity from final ranking

Vera preserves the raw vector score instead of overwriting it with the domain-adjusted result. A separate score controls the final order:

ranking score
    = vector similarity
    + lexical adjustment
    + temporal adjustment
    + source-trust adjustment
    + legal-status adjustment

The vector score answers, "How semantically similar is this passage to the query?" The ranking score answers, "How useful is this passage for this query under Vera's domain rules?"

Keeping both values makes it possible to determine whether a result ranked highly because of semantic similarity, an exact-term match, recency, source trust, or legal status. Without this separation, ranking behavior becomes difficult to explain and tune.

Adding lexical evidence to dense retrieval

Many legal and civic-information searches are short and specific:

• SARS
• INEC
• Section 24
• a statute name
• a case name
• a legal citation

These queries contain little semantic context. A broadly related passage may receive a strong vector score even when another passage contains the exact term requested.

Vera therefore adds a lexical signal to the ranking score:

if wanted_terms:
    searchable = f"{hit.citation} {hit.content}".lower()

    terms_present = sum(
        1 for term in wanted_terms
        if term in searchable
    )

    score += LEXICAL_WEIGHT * (
        terms_present / len(wanted_terms)
    )

Dense similarity supports concepts, paraphrases, and related language. Lexical matching supports citations, names, acronyms, and exact statutory terms. Combining them produces more reliable results for the language people use when searching legal material.

Treating time differently across source types

Vera's corpus contains both law and news, but these domains have different relationships with time.

News normally loses relevance as it ages. Legislation does not become less authoritative simply because it was enacted several years ago.

For non-historical news queries, Vera applies an approximate 14-day half-life and incorporates source reliability:

if hit.kind == "news":
    if requested_years and hit.year in requested_years:
        score += YEAR_BONUS
    elif not historical and hit.published:
        age_days = max(
            0.0,
            (now - hit.published) / 86400,
        )

        score += RECENCY_WEIGHT * (
            0.5 ** (age_days / 14.0)
        )

    score += TRUST_WEIGHT * weight_for(hit.trust_tier)

The resulting rules are domain-specific:

• News generally loses relevance with age.
• Legislation remains relevant while it is legally in force.
• Superseded law remains relevant to the periods in which it applied.
• An explicitly requested year can make older material preferable.
• Source reliability can affect the order of otherwise similar news results.

A generic recency boost would fail across this mixed corpus. Time must be interpreted according to document type and query intent.

Applying source-specific relevance thresholds

Different source types have different score distributions. News and general web content are comparatively noisy, while the case-law corpus is smaller and curated. Case summaries may also use different language from the user's question while still containing a relevant precedent.

Vera therefore applies different minimum similarity thresholds:

KIND_FLOORS = {
    "news": 0.35,
    "web": 0.35,
    "case_law": 0.25,
}

These values are deployment-specific and should be evaluated against the actual corpus. The design principle is that one global threshold assumes every document type has the same structure, noise level, and scoring distribution. They do not.

Using payload indexes for filtering and ordering

Metadata is useful only if the retrieval system can filter and order by it efficiently. Vera creates Qdrant payload indexes for fields commonly used in retrieval:

for field in (
    "source",
    "kind",
    "category",
    "scope",
    "country",
    "agencies",
    "topics",
):
    client.create_payload_index(
        collection_name=settings.collection,
        field_name=field,
        field_schema=PayloadSchemaType.KEYWORD,
    )

client.create_payload_index(
    collection_name=settings.collection,
    field_name="published",
    field_schema=PayloadSchemaType.INTEGER,
)

Indexing the publication timestamp also corrected a problem in the news feed. The original implementation retrieved a limited number of records and sorted them by date in application code. Once the collection grew beyond that limit, the newest item could fall outside the candidate window and never be displayed.

Sorting an arbitrary subset produces the correct order only within that subset. Moving publication-date ordering into Qdrant allowed the database to order the complete eligible result set before applying the limit.

The resulting retrieval model

Vera's retrieval pipeline now separates candidate discovery from domain-aware ranking:

User query
    ↓
Query analysis
    - source intent
    - historical intent
    - requested years
    - exact terms and citations
    ↓
Query embedding
    ↓
Vector candidate retrieval
    ↓
Metadata filtering
    ↓
Domain-aware reranking
    - legal status
    - temporal relevance
    - lexical evidence
    - source trust
    - document-type rules
    ↓
Source-specific thresholds
    ↓
Passages supplied to answer generation

Vector search remains responsible for finding semantically related candidates. It is not expected to encode every rule governing their final order. The ranking layer adds the domain information that embeddings cannot reliably infer.

Evaluation and tuning

Ranking weights should be based on observed retrieval behavior. Useful evaluation cases include:

• current law versus the superseded instrument it replaced;
• a historical question requiring the superseded instrument;
• an acronym whose exact match competes with a broader semantic result;
• a case-name query against narrative case summaries;
• recent and older news about the same event;
• legislation that is old but still in force;
• weak web matches competing with curated legal sources.

For each query, evaluation should retain the raw vector similarity, each ranking adjustment, the final ranking score, the document type, the legal status, and the expected result order.

This makes tuning empirical and explainable. Instead of selecting weights because they appear reasonable, the system can measure whether they separate relevant and irrelevant results across representative failure cases.

Engineering principles

Embeddings cannot replace domain modeling

Legal currency, jurisdiction, authority, and commencement dates are structured facts. They should be represented explicitly rather than inferred from prose.

Superseded material must be preserved deliberately

Removing old law destroys historical coverage. Retaining it without recording its status creates a risk for present-day answers. A reliable legal corpus needs both preservation and lifecycle metadata.

Similarity and relevance are not identical

The most semantically similar passage may not be the most authoritative or useful source. Vector similarity should remain one input to ranking rather than becoming the complete definition of relevance.

Hybrid retrieval is necessary for legal language

Embeddings handle concepts and paraphrases. Lexical signals handle case names, statutory provisions, acronyms, and citations. Legal search needs both.

Temporal relevance is domain-specific

Recency improves many news searches but should not reduce the authority of an older statute that remains in force. Time must be modeled according to source type.

Ingestion determines retrieval quality

Stale points, truncated sections, incomplete metadata, and weak embedding representations all become retrieval defects later. Search quality begins when the corpus is constructed, not when the query is submitted.

Conclusion

The initial retrieval system successfully found passages that were semantically similar to a user's question. Its failure was treating semantic similarity as a sufficient measure of legal relevance.

Legal retrieval requires additional knowledge: whether an instrument remains in force, when it became effective, what replaced it, which jurisdiction it belongs to, what authority issued it, whether the user is asking a current or historical question, and whether an exact citation or legal term appears in the source.

Vera now preserves vector similarity as the candidate-retrieval signal and applies a separate, domain-aware ranking layer for legal status, time, lexical evidence, source trust, and document type.

The broader lesson is that legal RAG is not only a nearest-neighbor search problem. It is a domain-modeling problem supported by vector search.