Keyword, Semantic, or Hybrid Search: What Actually Finds Better Papers?
Consider two searches that fail in opposite ways. A researcher enters the exact name of a clinical scale and receives papers that contain the words but do not use the scale as an outcome. Another researcher asks a natural-language question and receives conceptually related papers that omit the precise population, compound, or method needed. The first result set has literal overlap without enough contextual relevance. The second has contextual resemblance without enough exactness.
.png)
It is tempting to arrange the technologies behind those searches on a timeline: keyword search is old, semantic search is intelligent, and hybrid search is the inevitable upgrade. Information-retrieval research does not support that simple ranking. Keyword and semantic systems capture different evidence about relevance. Exact wording can be decisive for a gene variant, trial identifier, software package, legal citation, named instrument, author, or paper title. Conceptual similarity can be decisive when two disciplines use different terms for the same phenomenon or when a question paraphrases the language of the literature. A reliable scholarly-search system has to preserve both possibilities and test when each one fails.
Keyword search ranks documents largely through lexical evidence, meaning evidence found in the words that actually appear. Semantic search represents queries and documents as embeddings, which are lists of numbers learned so that texts with related meanings tend to occupy nearby positions in a multidimensional space. A query can then retrieve a passage even when it shares few words with the query. Hybrid search normally runs more than one retrieval method and combines their ranked results. Those definitions are straightforward. The important differences lie in how each score is produced, what information is lost, and how performance is measured on the intended research task.
Keyword retrieval preserves distinctions that meaning can blur
Modern keyword retrieval is more than checking whether a document contains a string. BM25, the best-known lexical ranking function, combines several signals. A term that is rare across the collection usually carries more distinguishing power than a term found almost everywhere. Repeated occurrences within a document raise the score, but the benefit saturates rather than growing without limit. Document length is normalized because a long document has more opportunities to contain a term by accident. The technical foundations are developed in The Probabilistic Relevance Framework: BM25 and Beyond, but the practical intuition is accessible: a rare exact term occurring prominently in a reasonably focused document is useful evidence(2).
That behavior is valuable in research. A query for “BRAF V600E” should not be replaced by a general collection of papers about oncogenic mutations. A known DOI should lead to one record, not to semantically adjacent work. A researcher looking for difference-in-differences analysis needs the named design, not every study that compares changes between groups. Lexical matching preserves these boundaries because it treats the observed term as evidence rather than asking only whether an embedding considers two texts close.
Lexical systems also expose their main weakness: the searcher must anticipate the language used in the documents. Authors can use synonyms, abbreviations, spelling variants, historical names, and discipline-specific expressions. Controlled vocabularies in scholarly databases help by assigning standardized subject labels, but new records may not yet be indexed and emerging concepts may not have a stable heading. Exact words can also be ambiguous. “Depression” might refer to a health condition, an economic contraction, or a geographic hollow. Lexical overlap preserves form; it does not automatically resolve context.
Even the category “keyword search” is no longer technically uniform. SPLADE v2 uses a neural model to learn sparse term weights and expansions while retaining a representation compatible with inverted indexes(8). A sparse representation has values for only a small share of all possible vocabulary dimensions; most dimensions are zero. An inverted index works like the index at the back of a book: for each term, it records the documents in which that term occurs. SPLADE can add learned vocabulary evidence without converting every document into a dense vector whose many dimensions usually contain values. This makes “keywords versus AI” the wrong boundary. More useful questions are whether exact terms remain recoverable, which expansions were learned, how large the index becomes, and whether the model transfers to the field being searched.
Exact matching is not merely a preference for familiar technology. Neural retrievers can struggle with rare terms and domain shift, which occurs when the documents and questions used in practice differ from the data on which a model was trained. Match Your Words! studied lexical behavior in neural information retrieval and found that the tested models generalized exact-term matching poorly on out-of-domain collections and on terms rarely observed in training(18). In scholarly search, those difficult terms may carry the most information: chemical codes, instrument abbreviations, protein variants, local policy names, and newly coined methods. Removing lexical retrieval can therefore remove the only component that treats a rare string as intrinsically important.
Semantic retrieval finds paraphrases but does not read the researcher's mind
Dense semantic retrieval usually begins with a bi-encoder. One encoder maps the query to a vector and another, often sharing the same architecture, maps each document or passage to a vector. Because document vectors can be calculated and indexed in advance, the system can quickly search for nearby vectors. A common numerical comparison is cosine similarity. It compares the angle between two vectors rather than their absolute size; a smaller angle suggests that their directions, and therefore their represented features, are more alike.
Sentence-BERT helped make this architecture practical for sentence similarity(6). Instead of running a large neural language model jointly for every possible pair, it creates reusable sentence embeddings. In the paper's setup, the estimated time to compare one sentence with 10,000 others fell from about 65 hours with pairwise BERT to about five seconds with SBERT. Those figures are not a current universal speed guarantee. They show why precomputed representations changed the scale at which semantic comparison could be used.
Dense Passage Retrieval applied separate question and passage encoders to open-domain question answering(5). On the tested datasets, it improved top-20 passage retrieval accuracy by 9 to 19 absolute percentage points over a strong Lucene BM25 baseline. The result established that learned vectors could recover answer-bearing passages without depending solely on shared wording. It did not establish that the same model would dominate known-item searches, rare identifiers, long scholarly abstracts, or a domain absent from its training data. Retrieval gains belong to a task, corpus, query distribution, judgments, and metric. They do not belong permanently to a model name.
The character of the task matters here. MS MARCO uses over one million anonymized Bing search queries with human-authored answers and a web-document collection comprising millions of passages; it defines answerability, answer generation, and passage-ranking tasks from those logs(3). Strong performance on that benchmark demonstrates competence at web question and passage ranking. It does not by itself show that a retriever will understand long scholarly abstracts or specialist terminology, especially when the corpus and judgments differ.
Scientific-document embeddings can use scholarly structure as a training signal. SPECTER represents papers from titles and abstracts and learns from citation links, treating cited and related papers as evidence about document proximity(14). SciNCL samples positive and difficult negative examples from neighborhoods in a citation-graph embedding(15). These models can be useful for paper recommendation, field mapping, and finding work adjacent to a seed paper. Citation proximity, however, is not identical to relevance for every question. It can reflect disciplinary communities, popularity, and established citation habits. A paper can be close in a citation graph while failing a user's population, method, or outcome requirement.
One vector is also unlikely to serve every scholarly task equally well. SciRepEval evaluates scientific representations across 24 tasks spanning classification, numerical prediction, similarity ranking, and direct query-to-document search(16). The benchmark found that prominent scientific-document embeddings did not transfer uniformly across task formats.
A method that used task-format controls and adapters to produce different representations improved aggregate results, reinforcing a broader lesson: “find similar papers,” “retrieve papers answering this query,” “predict a paper's category,” and “recommend likely citations” define different notions of similarity.
Semantic systems make another tradeoff when they compress an entire passage into one vector. A bi-encoder is efficient because the query and document are encoded separately, but it loses some fine-grained interaction between their words. A cross-encoder instead reads the query and one candidate document together and scores that pair. It can model richer relationships but is too expensive to run against every document in a large collection. This leads to a common two-stage architecture: a fast retriever proposes perhaps dozens or hundreds of candidates, then a slower cross-encoder reranks only those candidates. Passage Re-ranking with BERT is an early clear example of this pattern(9).
There are intermediate designs. ColBERT precomputes contextual token representations for documents, then applies late interaction between query and document tokens at search time(7). It preserves more detail than a single document vector while avoiding the full cost of a cross-encoder over the entire collection. The price is a larger index and more query-time work than simple dense retrieval. The phrase “semantic search” therefore hides several architectures with different storage, latency, and relevance behavior.
The broader survey of dense text retrieval reaches the same architectural distinction: bi-encoders make efficient first-stage retrieval possible, whereas cross-encoders model richer interactions at a much higher per-pair cost and are commonly used for reranking(13). It also identifies training data, query type, vocabulary overlap, negative sampling, approximate indexing, and domain shift as performance conditions. Its literature cutoff predates newer embedding and language-model rerankers, so it is better used to understand durable tradeoffs than to choose a current leaderboard winner.
Hybrid retrieval must be evaluated, not assumed
Hybrid retrieval commonly runs a lexical search and a semantic search in parallel, then merges their ranked lists. The rationale is complementarity. BM25 can protect exact entities and rare vocabulary; embeddings can recover paraphrases and conceptual neighbors. A preprint on hybrid scientific-document retrieval reported that a sparse method and SPECTER2 performed similarly on the tested cystic-fibrosis collection while their combination performed better(17). The result is suggestive, not universal: it comes from one small, old medical test collection. Its strongest lesson is methodological. Complementarity was measured on a target collection rather than declared from the labels “sparse” and “dense.”
.png)
Combining rankings is itself a design choice. BM25 scores and vector-similarity scores do not naturally share a scale. Reciprocal Rank Fusion, or RRF, avoids comparing raw scores. It gives each document a contribution based on its position in each ranked list and adds those contributions. The original short RRF paper found that this simple method outperformed the tested individual systems and another fusion approach on its collections(10). Simplicity is not the same as universal optimality. An Analysis of Fusion Functions for Hybrid Retrieval found RRF sensitive to its setting and reported that a tuned weighted average of rescaled lexical and semantic scores performed better in the studied in-domain and out-of-domain settings(11). A hybrid checkbox does not reveal which fusion rule, cutoff, or weighting is operating underneath.
Production systems add more choices: whether filters run before or after retrieval, how many candidates each method contributes, whether duplicates are removed before fusion, and whether a reranker follows the merged list. OpenSearch documentation describes hybrid subqueries followed by score normalization or rank fusion in a search pipeline(19). Azure AI Search runs BM25 and vector queries in parallel, merges them through RRF, and can add semantic reranking(20). Google Vertex AI Vector Search accepts dense and sparse representations in a hybrid query(21). These are authoritative descriptions of implementations, not independent evidence that any default is best for scholarly literature. They demonstrate that systems sharing the word “hybrid” can execute materially different pipelines.
The pipeline should be chosen with relevance judgments. A relevance judgment is a human assessment of whether, or how strongly, a result satisfies a defined information need. Testing only whether a few familiar papers appear can expose obvious failures but cannot show how many unfamiliar relevant papers were missed. Evaluation also has to match the workflow. A researcher exploring a new field may care about highly useful papers near the top. A systematic reviewer may care more about recovering nearly every eligible study even if screening takes longer.
Metrics express these priorities. Recall at k asks what proportion of all known relevant documents appears among the first k results. nDCG at 10, or normalized discounted cumulative gain, rewards systems that place more highly relevant documents closer to the top ten while allowing graded relevance rather than a simple yes or no. Precision measures how much of the retrieved set is relevant. Click-through rate records behavior, but clicks are affected by position, presentation, familiarity, and convenience; they are not a complete relevance standard. A search design cannot be called better without stating which outcome it improved.
The BEIR benchmark makes this point across 18 diverse retrieval datasets, including question answering, fact checking, biomedical search, and citation prediction(1). BM25 remained a robust zero-shot baseline, meaning it worked competitively on new datasets without task-specific retraining, while dense, learned sparse, late-interaction, and reranking systems varied substantially by dataset. The benchmark does not reproduce every commercial scholarly index or interface, but it decisively undermines the idea that one architecture wins simply because it is newer.
TREC Deep Learning further shows why reusable collections require careful human judgments and warns against repeatedly tuning choices on the final test set(4). Pyserini provides a common environment for sparse, dense, and hybrid experiments so that corpus, topics, judgments, and metric code can be held constant(12).
A defensible scholarly-search workflow starts with the query type. Use lexical retrieval aggressively when exact identity is the task: titles, authors, DOIs, variants, named methods, instruments, or uncommon technical expressions. Use semantic retrieval to expand discovery when terminology is uncertain, language differs across disciplines, or a question paraphrases likely evidence. For high-recall work, take candidates from both, preserve which method retrieved each record, remove duplicates, and rerank only when a representative judged query set shows an improvement worth the extra cost. Then inspect failures, not just average scores. If the system misses rare identifiers, strengthen lexical retrieval. If it misses paraphrases, improve semantic representations or training data. If both retrieve the right candidates but order them poorly, examine fusion and reranking.
Keyword search is not obsolete, semantic search is not comprehension, and hybrid search is not automatically the best of both. Each is a way of turning partial evidence about relevance into a ranking. Better paper discovery comes from keeping exact and conceptual signals visible, matching architecture to the research task, and evaluating results on questions that resemble the work researchers will actually do.
Sources
1. BEIR - https://proceedings.neurips.cc/paper/2021/hash/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract.html
2. The Probabilistic Relevance Framework: BM25 and Beyond - https://doi.org/10.1561/1500000019
3. MS MARCO - https://arxiv.org/abs/1611.09268
4. TREC Deep Learning Track - https://doi.org/10.1145/3404835.3463249
5. Dense Passage Retrieval - https://aclanthology.org/2020.emnlp-main.550/
6. Sentence-BERT - https://aclanthology.org/D19-1410/
7. ColBERT - https://doi.org/10.1145/3397271.3401075
8. SPLADE v2 - https://arxiv.org/abs/2109.10086
9. Passage Re-ranking with BERT - https://arxiv.org/abs/1901.04085
10. Reciprocal Rank Fusion - https://doi.org/10.1145/1571941.1572114
11. An Analysis of Fusion Functions for Hybrid Retrieval - https://doi.org/10.1145/3596512
12. Pyserini - https://doi.org/10.1145/3404835.3463238
13. Dense Text Retrieval Based on Pretrained Language Models: A Survey - https://doi.org/10.1145/3637870
14. SPECTER - https://aclanthology.org/2020.acl-main.207/
15. SciNCL - https://aclanthology.org/2022.emnlp-main.802/
16. SciRepEval - https://aclanthology.org/2023.emnlp-main.338/
17. Sparse Meets Dense - https://arxiv.org/abs/2401.04055
18. Match Your Words! - https://arxiv.org/abs/2112.05662
19. OpenSearch hybrid search - https://docs.opensearch.org/latest/vector-search/ai-search/hybrid-search/index/
20. Azure AI Search hybrid search - https://learn.microsoft.com/en-us/azure/search/hybrid-search-overview
21. Vertex AI Vector Search hybrid queries - https://cloud.google.com/vertex-ai/docs/vector-search/query-index-public-endpoint#hybrid_queries