Content-based image retrieval

Content-based image retrieval (CBIR), also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR) is the application of computer vision to the image retrieval problem, that is, the problem of searching for digital images in large databases. "Content-based" means that the search makes use of the contents of the images themselves, rather than relying on human-inputted metadata such as captions or keywords. A content-based image retrieval system (CBIRS) is a piece of software that implements CBIR.

There is growing interest in CBIR because of the limitations inherent in metadata-based systems. Textual information about images can be easily searched using existing technology, but requires humans to personally describe every image in the database. This is impractical for very large databases, or for images that are generated automatically, e.g. from surveillance cameras.

The ideal CBIR system from a user perspective would involve what is referred to as semantic retrieval, where the user makes a request like "find pictures of dogs" or even "find pictures of Abraham Lincoln". This type of open-ended task is very difficult for computers to perform - pictures of chihuahuas and Great Danes look very different, and Lincoln may not always be facing the camera or in the same pose. Current CBIR systems therefore generally make use of lower-level features like texture, color, and shape.

Different implementations of CBIR make use of different types of user queries.

• With query by example, the user searches with a query image (supplied by the user or chosen from a random search), and the software finds images similar to it based on various low-level criteria. The search can be progressively refined by marking search results as "relevant" or "not relevant" to the search query.
• With query by sketch, the user draws a rough approximation of the image they are looking for, for example with blobs of color, and the software locates images whose layout matches the sketch.

One application of CBIR is to identify images with skin-tones and shapes that could indicate the presence of nudity, for filtering and for searching by law enforcement.

External links

• The GNU Image Finding Tool (GIFT) - an open source query by example CBIRS
• Viper Demo - an online demonstration of the GIFT
• LCPD: Leiden 19th-Century Portrait Database - an online database of 19th century studio portraits searchable via CBIR