ImageRover

  

Figure 12: ImageRover query result.

Developer

Department of Computer Science, Boston University, MA.

URL

http://www.cs.bu.edu/groups/ivc/ImageRover/. For a demo of the system, look at http://www.cs.bu.edu/groups/ivc/ImageRover/demo.html.

References

[STC97], [TCS97].

Features

The features used for querying are color and texture orientation. The system computes distributions of color and orientation over 6 subimages (the whole image and 5 subregions: the central and corner regions). The result is an image index vector made of 2 $\times$ 6 subvectors. This dimension is subject to a reduction via a principal component analysis (PCA) for each of the subvector spaces. Image color histograms are computed in the CIE Luv color space and each histogram quantizes the color space into 64 bins. The texture direction distribution is calculated using steerable pyramids. At each of the four levels of the pyramid, texture direction and strength for each pixel is calculated resulting in an orientation histogram, quantized to 16 bins.

Querying

The user starts a query session by specifying a set of keywords related to the desired images. From the set of images displayed, the user finds and marks one or more images which are similar to what she/he is looking for. Apart from selecting relevant images, the user can deselect one or more of the query images before reiterating a new query. There is no limit to the number of iterations in providing relevance feedback, nor in the number of example images.

Matching

Based on relevance feedback from the user, the system selects the appropriate $\tilde{L}_m$ normalized Minkowski metric each time a query is made. The normalization factor $\mu^i_m$, employed by the normalized distance $\tilde{L}_m(x_i,y_i)=L_m(x_i,y_i)/ \mu^i_m$, with x_i and y_i being two subvectors of the image index vectors X and Y, is the expected value over the entire database: $\mu_m^i=E[\tilde{L}_m(x_i,y_i)]$. It is allowed to use metrics of different orders m_i for each of the image index subvectors. Thus, if S is a set of relevant images indicated by the user, the appropriate value for m of the i-th subvector is chosen so as to minimize the distance between relevant images: $m_i=arg \, \min_{m} E[ \tilde{L}_m(p_i,q_i)]$, over all $P,Q \in S$. Then the k-nearest neighbor search of the image index vector uses the following weighted distance metric: $d(X,Y)=(w_1,\ldots,w_n)(\tilde{L}_{m_1}(x_1,y_1),\ldots,$ $\tilde{L}_{m_n}(x_n,y_n))^T$ where X,Y are the image index vectors and w_i are relevance weights.

Indexing

An optimized kD tree is used and the search algorithm in this structure employs an approximation factor. The user can select through the search accuracy button one of the three values for this approximation factor.

Result presentation

Images similar to the query images are shown in decreasing similarity order. Their number is chosen by the user. Each displayed thumbnail image is a hypertext link to the original image, allowing the user to retrieve the desired image from its home WWW site.

Relevance feedback

The user indicates the more relevant images, after which the system selects the appropriate $\tilde{L}_m$, see above.

Applications

Image Rover is an image search engine for the WWW.