WebSEEk

Developer

Image and Advanced Television Lab, Columbia University, NY.

URL

http://www.ctr.columbia.edu/WebSEEk/.

References

[Smi97]

Features

WebSEEk makes text-based and color based queries through a catalogue of images and videos collected from the Web. Color is represented by means of a normalized 166-bin histogram in the HSV color space, see VisualSEEkVisualSEEk.

  

Figure 31: WebSEEk relevance feedback.

Querying

The user initiates a query by choosing a subject from the available catalogue or entering a topic. The results of the query may be used for a color query in the whole catalogue or for sorting the result list by decreasing color similarity to the selected item. Also, the user has the possibility of manually modifying an image/video color histogram before reiterating the search.

Matching

The distance between a query color histogram h_q and a target histogram h_t is given by $d(h_q,h_t)=(h_q-h_t)^tA(h_q-h_t) \approx \mu_q + \mu_t - 2 \sum_{k\, with\, h_q[k] \geq \tau} h_q[k] r_t[k]\:$, where A=(a[i,j]) is a color similarity matrix, $\mu_q=h_q^tAh_q$, $\mu_t=h_t^tAh_t$, r_t=Ah_t and $\tau$ defines a threshold for choosing the most significant colors in the approximated distance.

Indexing

For all the database images, $\mu_t$ and r_t[k], $k\in\{0,\ldots,165\}$ are computed and indexed individually.

Result presentation

Images retrieved are displayed page by page in a descending similarity order. A manipulation (union, intersection, subtraction) of the search result lists, in order to reiterate a query, is possible.

Relevance feedback

The user has the possibility of selecting positive and negative examples from the result of a query in order to reformulate the query, see figure 31. If the set of relevant images is denoted by I_r and the set of nonrelevant images is I_n then the new query histogram at the (k+1)th feedback iteration is computed by $h_q^{k+1}=\Vert\alpha h_q^k+ \beta\sum_{i\in I_r}h_i-\gamma\sum_{j\in I_n}h_j\Vert$.