Quicklook²

  

Figure 22: Quicklook².

Developer

CNR Institute of Multimedia Information Technologies, Milan, Italy.

URL

http://www.itim.mi.cnr.it/Linee/Linea1/Sottolinea3/relfeme.htm.

References

[CGSnt].

Features

Besides text descriptors, Quicklook² uses the following contest-based features. The color histogram of 64 bins. The color coherence vector in Lab color space, quantized into 64 colors, where pixels are coherent if they belong to a large similarly colored region; the image is first blurred by averaging a $3\times3$ neighborhood. The percentage of skin-colored pixels. The `spatial chromoticity histogram': bin i gives the ratio h(i) of pixels having color i, the relative coordinates of the baricenter $\bf {b}(i)$ of these pixels, and the standard deviation $\sigma(i)$ of these pixels' distance to the baricenter. The cooccurrence of colors in Lab color space, quantized into 11 colors: red, orange, yellow, green, blue, purple, pink, brown, black, gray, white. The moments of inertia (mean, variance, skewness, kurtosis) of the color distribution in Lab space. Contrast information at Canny edge pixels: the percentage of low, medium, and high contrast pixels, the thresholds on the gradient strenghts corresponding to medium and high constrast, the number of connected regions with high contrast contours, and the percentage of medium contrast edge pixels connected to high contrast edges. Texture features based on the `neighborhood gray tone difference matrix' [AK89]: coarseness, contrast, busyness, complexity, and strength. The mean and variance of the absolute values of the Daubechies wavelet transform coefficients of the luminance of the subimages at the first three levels of resolution. The histrogram of 72 bins of directions of high gradient pixels on Canny edges. The Hu moment invariants [Hu62]. After quantization into 11 colors: the number of color regions, the distribution of these regions around the center, x, and y axis.

Querying

The user initially selects a query image. The features of this image are taken for the first search. The user can than indicate which images are relevant. From these relevant image a new query feature vector is composed, see below.

Matching

The distance between two spatial chromoticity histograms of lenght k is defined as $\sum_{i=1}^k \min\{h_1(i),h_2(i)\} (1-d(\bf {b}_1(i),\bf {b}_1(i))\sqrt{2}/2) + \min\{\sigma_1(i),\sigma_2(i)\}/\max\{\sigma_1(i),\sigma_2(i)\}$. The distance measure for any of the other features is the L₁ distance. Each distances measure d_j is normalised by a factor $\mu_j+K\sigma_j$, with $\mu_j$ and $\sigma_j$ the average and the standard deviation over all images, and K, which affects the number of outliers, set to 3. The total distance between two images is a weighted sum of the feature distance measures d_j.

Indexing

In a preprocessing step, the distance of all database objects to m reference objects are computed. At query time, the distances d_i, $i=1,\ldots,m$ of the query to these reference objects is also computed. With an SQL query, those database objects are retrieved that lie within some range from each d_i. After this filtering, these database objects are compared to the query directly.

Result presentation

The results are listed in descreasing order of similarity. See figure 22 for the result after a few iterations of relevance feedback.

Relevance feedback

The innovative part of the system is the relevance feedback method. After the relevant images are selected, each of their features contributes to the new query feature vector if its distance to the average over the relevant images' feature is sufficiently large (three times the standard deviation). The new query feature vector is the average of the contributing featues.