WebSeer

Developer

Department of Computer Science, University of Chicago, Illinois, USA.

URL

http://infolab.cs.uchicago.edu/webseer/.

References

[SFA96].

Features

The images collected from the Web are submitted to a number of color tests in order to separate photographs from drawings. Some simple tests measure the number of different colors in the image, the fraction of pixels which have one of the N most frequent colors for a given threshold N, the fraction of pixels with transition value (the largest L₁ distance between a pixel and one of its neighbors in the RGB space) greater than a threshold T, the fraction of pixels with saturation level (the difference between the maximum and the minimum value of the RGB color bands) greater than a threshold T, and the ratio of the image dimensions. A more elaborate test creates first an average color histogram for graphics, H_g, and one for photographs, H_p, using two large sets of images. Defining the correlation between two normalized RGB histograms, A and B, as $C(A,B)=\sum_{i=0}^{15}\sum_{j=0}^{15}\sum_{k=0}^{15}A_{i,j,k}B_{i,j,k}$, an image with a color histogram H_i gains a score to the test equal to s=C(H_i,H_p)/(C(H_i,H_p)+C(H_i,H_g)). Two similar tests are also made using the farthest neighbour histograms and the saturation histograms instead of the color histograms. Images determined to be photographs are subjected to a face detector based on a neural network. Keywords are extracted from the image file name, captions, hyperlinks, alternate text and HTML titles.

Querying

The user gives keywords describing the contents of the desired images, and optionally specifies some image characteristics such as dimensions, file size or whether he is looking for photographs or drawings. In the case the user is looking for people, he must indicate the number of faces as well as the size of the portrait.

Matching

To classify an image as photograph or graphics, the color tests are combined using multiple decision trees constructed using a training set of hand-classified images. They are binary trees whose internal nodes contain the next test the image should be submitted to and a threshold to direct the search by comparing the test result with the threshold. In each leaf we find a probability estimate that the image is a photograph. Computing the mean of the results got from all decision trees and comparing this with a threshold, a decision is taken whether the image falls in one category or the other.

Result presentation

The thumbnails of the resulting images are displayed by decreasing size of the face in the case of searching for portraits. Otherwise, there is no explicit order of the retrieved images. The user has access to the Web page where the image was collected from, through a page icon aside the displayed thumbnail.