User:Rbdota23/Information retrieval

Article Draft

Below is the article draft. I will add a section named SPLADE, and add to History and Timeline section of the Information Retrieval article. I understand that both History and Timeline are connected and similar, but it our responsibility to update it.

History

By the late 1990s, the rise of the World Wide Web fundamentally transformed information retrieval. While early search engines such as AltaVista (1995) and Yahoo! (1994) offered keyword-based retrieval, they were limited in scale and ranking sophistication. The breakthrough came in 1998 with the founding of Google, which introduced the PageRank algorithm, leveraging the web’s hyperlink structure to assess page importance and improve relevance ranking.

During the early 2000s, web search systems evolved rapidly with the integration of machine learning techniques. These systems began to incorporate user behavior data (e.g., click-through logs), query reformulation, and content-based signals to improve search accuracy and personalization. In 2009, Microsoft launched Bing, introducing features that would later incorporate semantic web technologies through the development of its Satori knowledge base. Academic analysis have highlighted Bing’s semantic capabilities, including structured data use and entity recognition, as part of a broader industry shift toward improving search relevance and understanding user intent through natural language processing.

A major leap occurred in 2018, when Google deployed BERT (Bidirectional Encoder Representations from Transformers) to better understand the contextual meaning of queries and documents. This marked the integration of deep neural language models into production-scale retrieval systems. BERT’s bidirectional training enabled a more nuanced comprehension of word relationships in context, improving the handling of natural language queries. Its success spurred adoption of transformer-based models across academic research and commercial search applications.

Simultaneously, the research community began exploring neural ranking models that outperformed traditional lexical-based methods. Long-standing benchmarks such as the Text REtrieval Conference (TREC), initiated in 1992, and more recent evaluation frameworks Microsoft MARCO(MAchine Reading COmprehension) (2019) became central to training and evaluating retrieval systems across multiple tasks and domains. MS MARCO has also been adopted in the TREC Deep Learning Tracks, where it serves as a core dataset for evaluating advances in neural ranking models within a standardized benchmarking environment.

As deep learning became integral to information retrieval systems, researchers began to categorize neural approaches into three broad classes: sparse, dense, and hybrid models. Sparse models, including traditional term-based methods and learned variants like SPLADE, rely on interpretable representations and inverted indexes to enable efficient exact term matching with added semantic signals. Dense models, such as dual-encoder architectures like ColBERT, use continuous vector embeddings to support semantic similarity beyond keyword overlap. Hybrid models aim to combine the advantages of both, balancing the lexical precision of sparse methods with the semantic depth of dense models. This classification reflects a broader shift in the 2020s toward retrieval techniques that scale effectively while optimizing for both relevance and computational efficiency.

As IR systems increasingly rely on deep learning, concerns around bias, fairness, and explainability have also come to the forefront. Research is now focused not just on relevance and efficiency, but on transparency, accountability, and user trust in retrieval algorithms.

Timeline

1998

• Google is founded by Larry Page and Sergey Brin. It introduces the PageRank algorithm, which evaluates the importance of web pages based on hyperlink structure.

2001

• Wikipedia launches as a free, collaborative online encyclopedia. It quickly becomes a major resource for information retrieval, particularly for natural language processing and semantic search benchmarks.

2009

• Microsoft launches Bing, introducing features such as related searches, semantic suggestions, and later incorporating deep learning techniques into its ranking algorithms.

2013

• Google’s Hummingbird algorithm goes live, marking a shift from keyword matching toward understanding query intent and semantic context in search queries.

2018

• Google AI researchers release BERT (Bidirectional Encoder Representations from Transformers), enabling deep bidirectional understanding of language and improving document ranking and query understanding in IR.

2019

• Microsoft introduces MS MARCO (Microsoft MAchine Reading COmprehension), a large-scale dataset designed for training and evaluating machine reading and passage ranking models.

2020

• ColBERT (Contextualized Late Interaction over BERT) is proposed and introduced at SIGIR 2020, enabling scalable neural retrieval by combining efficient indexing with contextual deep matching.

2021

• SPLADE is introduced at SIGIR 2021. It’s a sparse neural retrieval model that balances lexical and semantic features using masked language modeling and sparsity regularization.

2022

• The BEIR benchmark is released to evaluate zero-shot IR across 18 datasets covering diverse tasks. It standardizes comparisons between dense, sparse, and hybrid IR models.

Splade

SPLADE is a neural retrieval model designed to learn sparse vector representations for queries and documents, effectively merging the strengths of traditional lexical matching methods and modern neural embeddings. Unlike dense retrieval models that rely on continuous vector spaces, SPLADE produces sparse representations, allowing for efficient integration with inverted index structures commonly used in information retrieval systems.

SPLADE and its subsequent versions have brought advancement in the field of learned sparse retrieval. The original SPLADE model was presented at the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021), a trusted peer-reviewed conference in the field.

SPLADE v2 is an advancement of the original SPLADE model, designed to learn sparse vector representations for queries and documents, effectively merging the strengths of traditional lexical matching methods and modern neural embeddings. Unlike dense retrieval models that rely on continuous vector spaces, SPLADE v2 produces highly sparse representations, allowing for efficient integration with inverted index structures commonly used in information retrieval systems.

The model operates by using the masked language modeling (MLM) head of BERT, applying explicit sparsity regularization to generate term-weighted representations. This approach enables SPLADE v2 to perform exact term matching while capturing semantic nuances, addressing challenges like the vocabulary mismatch problem encountered in traditional sparse methods.

Key improvements in SPLADE v2 include modifications to the pooling mechanism, benchmarking models based solely on document expansion, and introducing models trained with knowledge distillation. These enhancements have led to significant gains in retrieval effectiveness, achieving more than a 9% improvement in NDCG@10 on the TREC DL 2019 dataset and leading to state-of-the-art results on the BEIR benchmark.

One of the notable advantages of SPLADE is its ability to maintain efficiency comparable to traditional lexical search methods like BM25, while offering the enriched semantic understanding characteristic of neural models. This balance makes it a compelling choice for real-world applications where both speed and accuracy are critical.

The official implementation and model weights for SPLADE are available under a Creative Commons NonCommercial license, with community-driven versions like SPLADE++ released under more permissive licenses.