Text Retrieval Conference

The Text REtrieval Conference (TREC) is an on-going series of workshops focusing on a list of different information retrieval (IR) research areas, or tracks. It is co-sponsored by the National Institute of Standards and Technology (NIST) and Advanced Research and Development Activity (ARDA) center of the U.S. Department of Defense, and began in 1992 as part of the TIPSTER Text program. Its purpose is to support and encourage research within the information retrieval community by providing the infrastructure necessary for large-scale evaluation of text retrieval methodologies and to increase the speed of lab-to-product transfer of technology.

Each track has a challenge wherein NIST provides participating groups with data sets and test problems. Depending on track, test problems might be questions, topics, or target extractible features. Uniform scoring is performed so the systems can be fairly evaluated. After evaluation of the results, a workshop provides a place for participants to collect together thoughts and ideas and present current and future research work.

Participation

The conference is made up a varied, international group of researchers and developers. In 2003, there were 93 groups from both academia and industry from 22 countries participating.

Conference Contributions

TREC claims that within the first six years of the workshops, the effectiveness of retrieval systems has approximately doubled. The conference was also the first to hold large-scale evaluations of non-English documents, speech, video and retrieval across languages. Additionally, the challenges have inspired a large body of [publications].

Tracks

New tracks are added as new research needs are identified, this list is current for 2005.

Enterprise Track - Goal: to study search over the data of an organization to complete some task.
Genomics Track - Goal: to study the retrieval of genomic data, not just gene sequences but also supporting documentation such as research papers, lab reports, etc.
HARD Track - Goal: to achieve High Accuracy Retrieval from Documents by leveraging additional information about the searcher and/or the search context.
Question Answering Track - Goal: to achieve more information retrieval than just document retrieval by answering factoid, list and definition-style questions.
Robust Retrieval Track - Goal: to focus on individual topic effectiveness.
SPAM Track - Goal: to provide a standard evaluation of current and proposed spam filtering approaches.
Terabyte Track - Goal: to investigate whether/how the IR community can scale traditional IR test-collection-based evaluation to significantly large collections.

Past tracks

Cross-Language Track - Goal: to investigate the ability of retrieval systems to find documents topically regardless of source language.
Filtering Track - Goal: to binarily decide retrieval of new incoming documents given a stable information need.
Interactive Track - Goal: to study user interaction with text retrieval systems
Novelty Track - Goal: to investigate systems' abilities to locate new (i.e., non-redundant) information.
Video Track - Goal: to research in automatic segmentation, indexing, and content-based retrieval of digital video.

In 2003, this track became it's own independent evaluation named TRECVID.

Web Track - Goal: to search on a document set that is a snapshot of the World Wide Web.

External links

Information extracted from the publicly available parts of the TREC homepage.