Distributed search engine
A distributed search engine is a search engine where there is no central server. Unlike traditional centralized search engines, work such as crawling, data mining, indexing, and query processing is distributed among several peers in a decentralized manner where there is no single point of control.
History
Rorur
The aim of the Rorur project is to create a distributed search engine that runs on a network of computers of common people. A competitive latency and the delivery of the requested rank can be achieved if the number of participating nodes is large enough and the fraction of malicious nodes does not exceed a calculable threshold https://rorur.com/whitepaper. The architecture builds on open-source algorithms that rely on public contribution for development and maintenance. To incentivize those who join and contribute, the revenue from advertising is distributed among node maintainers. [1]
Presearch
Presearch is a search engine powered by a distributed network of community operated nodes which aggregate results from a variety of sources. This powers the searches at https://engine.presearch.org/search This is planned to be a precursor where each node collaborates on a global decentralised index. [2]
YaCy
On December 15, 2003 Michael Christen announced development of a P2P-based search engine, eventually named YaCy, on the heise online forums.[3][4]
Dews
A theoretical design for adistributed search engine discussed in academic literature. [5]
Seeks was an open source websearch proxy and collaborative distributed tool for websearch. It ceased to have a usable release in 2016.
InfraSearch
In April 2000 several programmers (including Gene Kan, Steve Waterhouse) built a prototype P2P web search engine based on Gnutella called InfraSearch. The technology was later acquired by Sun Microsystems and incorporated into the JXTA project.[6] It was meant to run inside the participating websites' databases creating a P2P network that could be accessed through the InfraSearch website.[7][8][9]
Opencola
On May 31, 2000 Steelbridge Inc. announced development of OpenCOLA a collaborative distributive open source search engine.[10] It runs on the user's computer and crawls the web pages and links the user puts in their opencola folder and shares resulting index over its P2P network.[11]
FAROO
In February 2001 Wolf Garbe published an idea of a peer-to-peer search engine,[12] started the Faroo prototype in 2004,[13] and released it in 2005.[14][15]
See also
References
- ^ "Distributed Search Engine".
- ^ "Presearch is a Decentralized Search Engine".
- ^ "YaCy: News". Archived from the original on 2005-11-24.
- ^ Michael Christen. "Ich entwickle eine P2P-basierende Suchmaschine. Wer macht mit?". heise online.
- ^ "DEWS: A decentralized engine for Web search".
- ^ Justin Hibbard. "Can peer-to-peer grow up?". Red Herring.[permanent dead link]
- ^ Simon Foust. "Move Over Yahoo, Here Comes InfraSearch". Dmusic. Archived from the original on 2000-10-13.
- ^ Sean M. Dugan. "Peer-to-peer networking is poised to revolutionize the Internet once again". InfoWorld. Archived from the original on 2000-10-18.
- ^ John Borland. "Napster-like technology takes Web search to new level". Cnet.
- ^ David Akin. "Software launched with a little pop". Financial Post.[dead link]
- ^ Paul Heltzel. "OpenCola-Have Some Code and a Smile". Technology Review.
- ^
Wolf Garbe. "BINGOOO - Die Transformation des World Wide Web zur virtuellen Datenbank" (in German). Wirtschaftinformatik. Archived from the original on 2014-02-02. Retrieved 2010-12-21.
... Wir setzen dem das Konzept einer verteilten Peer-to-Peer-Suchmaschine entgegen [We counter with the concept of a distributed peer-to-peer search engine] ...
- ^
Bernard Lunn. "Technical Q&A With FAROO Founder". ReadWriteWeb. Archived from the original on 2011-02-14.
... When I started to work on the first prototype in 2004 ...
- ^ "FAROO: History". Archived from the original on 2008-03-22.
- ^ "Revisited: Deriving crawler start points from visited pages by monitoring HTTP traffic". Faroo.