Jump to content

Web directory

From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Proffviktor (talk | contribs) at 20:28, 21 April 2009. The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

A web directory or link directory is a directory on the World Wide Web. It specializes in linking to other web sites and categorizing those links.

A web directory is not a search engine and does not display lists of web pages based on keywords; instead, it lists web sites by category and subcategory. The categorization is usually based on the whole web site rather than one page or a set of keywords, and sites are often limited to inclusion in only a few categories. Web directories often allow site owners to directly submit their site for inclusion, and have editors review submissions for fitness.

RSS directories are similar to web directories, but contain collections of RSS feeds, instead of links to web sites.

Scope of listing

Most of the directories are very general in scope and list websites across a wide range of categories, regions and languages. But there are also some niche directories which focus on restricted regions, single languages, or specialist sectors. One type of niche directory with a large number of sites in existence, is the shopping directory for example. Shopping directories specialize in the listing of retail e-commerce sites.

Examples of well known, general, web directories are Yahoo! Directory and the Open Directory Project (ODP). ODP is significant due to its extensive categorization and large number of listings and its free availability for use by other directories and search engines.[1]

However, a debate over the quality of directories and databases still continues, as search engines use ODP's content without real integration, and some experiment using clustering. There have been many attempts to make directory development easier, such as using automated submission of related links by script, or any number of available PHP portals and programs. Recently, social software techniques have spawned new efforts of categorization, with Amazon.com adding tagging to their product pages.

Directories have various features in listing, often depend upon the price paid for inclusion:

  • Free submission – there is no charge for the review and listing of the site
  • Reciprocal link – a link back to the directory must be added somewhere on the submitted site in order to get listed in the directory
  • Paid submission – a one-time or recurring fee is charged for reviewing/listing the submitted link
  • No follow – there is a rel="nofollow" attribute associated with the link, meaning search engines will give no weight to the link.
  • Featured listing – the link is given a premium position in a category (or multiple categories) or other sections of the directory, such as the homepage
  • Bid for position – where sites are ordered based on bids
  • Affiliate links – where the directory earns commission for referred customers from the listed websites

Human-edited directories

A human-edited directory is created and maintained by editors who add links based on the policies particular to that directory.

Human-edited directories are often targeted by SEOs on the basis that links from reputable sources will improve rankings in the major search engines. Some directories may prevent search engines from rating a displayed link by using redirects, nofollow attributes, or other techniques. Many human-edited directories, including the Open Directory Project,World Wide Web Virtual Library and new-built AlexBid, are edited by volunteers, who are often experts in particular categories. These directories are sometimes criticized due to long delays in approving submissions, or for rigid organizational structures and disputes among volunteer editors.

In response to these criticisms, some volunteer-edited directories have adopted wiki technology, to allow broader community participation in editing the directory (at the risk of introducing lower-quality, less objective entries).

Another direction taken by some web directories is the paid for inclusion model. This method enables the directory to offer timely inclusion for submissions and generally fewer listings as a result of the paid model. They often offer additional listing options to further enhance listings, including features listings and additional links to inner pages of the listed web site. These options typically have an additional fee associated, but offer significant help and visibility to sites and/or their inside pages.

Today submission of websites to web directories is considered as a common SEO (search engine optimization) technique to get vital back-links for the submitted web site. One distinctive feature of 'directory submission' is that it cannot be fully automated like search engine submissions. Manual directory submission is a tedious and time consuming job and is often outsourced by the webmasters.

Bid for Position directories

Bid for Position directories or also known as bidding web directories, are paid-for-inclusion web directories where the listings of websites in the directory are ordered according to their bid amount. They are special in that the more a person pays, the higher up the list of websites in the directory they go. With the higher listing, the website becomes more visible and increases the chances that visitors who browse the directory will click on the listing. There are PHP scripts (free and paid versions) for the management of bid for position directories include phpLinkBid (paid), Link Bid Script (free) and a modified link bid version for phpLD (phpLinkDirectory).

Automated submission web directories

Creating an automated web directory is quite a challenging task because human editing is replaced with automated editing. Basically there are two major aspects in editing which should be automated:

1) assigning a right category to a website

2) filter out inappropriate websites

The first one requires a mechanism which classifies web site content and decides which category among a given set of categories in the directory this website belongs to. Two major features which constitute websites are textual content and images. Although a certain progress in image processing has been achieved in the last decades it is still far from the level when complex images can be successfully, reliably and promptly classified. Text or Document Classification is much more advanced. Since usually in web directories a set of categories is predefined supervised document classification may be applied here.

The filtering aspect (or imposing restrictions on submitted sites) is also very important. One example is: the same link is not allowed to be listed twice. Along with other simple tasks this one is solved using straightforward algorithms. Much more complicated task is to filter out sites with specific content, for example inappropriate content including legal and illegal pornography, racial hatred, violence, etc.

Using Machine Learning methods is one of solutions. An example of fully automated submission web directory which satisfies basic requirements is LogYourLink.com. One of important features is a reliable category recommender system which uses Machine Learning methods for content classification in both category suggestion and filtering.

Of course, automation is never perfect. However one possible advantage of using automated editing based on machine learning methods over human editing is better category suggestion in terms of search engine interpretation of the website textual content. The reasons is in certain flexibility in creating training sets of documents. They may be created in good relationship with search engine indexing and processing of textual components. This is very important for both search and web advertising.


See also

References

  1. ^ Paul Festa (December 27, 1999), Web search results still have human touch, CNET News.com, retrieved September 18, 2007