NETRA

  

Figure 18: NETRA. Result of querying on shape with the complex description.

Developer

Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA.

URL

http://maya.ece.ucsb.edu/Netra/. A demo of the system is available at webaddress http://maya.ece.ucsb.edu/Netra/netra.html.

References

[Ma97], [MM99].

Features

Images in the database are segmented into regions of homogeneous color. Of those regions, the following features are extracted: color, texture, shape, and spatial location. On the basis of a training set of images, the RGB color space is quantized, and represented by a color codebook of 256 colors, the centroids of the quantization cells. The colors of an image region are also quantized, giving a color feature vector $f_c=(c_0,p_0,\ldots,c_n,p_n)$, with c_i the index into the color code book, and p_i the fraction of that color in the region, $p_0+\ldots+p_n=1$. The number n is the number of colors used to represent the region, which is different for each region. Texture is represented by a feature vector f_t containing the normalized mean and standard deviation of a series of Gabor wavelet transforms of the image: $f_t=(\mu_{0,0},\ldots,\mu_{s,k},\sigma_{0,0},\ldots,\sigma_{s,k})$, with s the number of scales, and k the number of directions. There are three feature vectors used to represent the shape of regions. The first, f_K, is based on the curvature function of the contour, giving the curvature at each point on the contour. The second, f_R is based on the centroid distance function, giving at each contour point the distance to the centroid of the region. The third, f_Z, is the complex coordinate function, representing each contour point as a complex number with real component equal to the x-coordinate, and the imaginary component equal to the y-coordinate. On 64 samples of each of these functions, the fast Fourier transform (FFT) is applied, of which the real (amplitude) component of the coefficients is used, the numbers $F_{-31},\ldots, F_{32}$. The feature vectors are as follows: $f_K=(\vert F_1\vert,\ldots,\vert F_{32}\vert)$, $f_R=(\vert F_1\vert,\ldots,\vert F_3{2}\vert)/\vert F_0\vert$, $f_Z=(\vert F_{-31}\vert,\ldots,\vert F_{-1}\vert,\vert F_2\vert,\ldots,\vert F_{32}\vert)/\vert F_1\vert$.

Querying

There are 2,500 images from the Corel photo collection, organized in 25 categories, with 100 images in each category. You can select any one of them as the query image. All images in the database have been segmented into homogeneous regions. You can click on one of the regions and select one of the four image attribute color, spatial location, texture, and shape. Instead of using an image example, you can also directly specify the color and spatial location. The spatial location querying tool utilizes two bounding boxes to define the area of interest. The inner box is used to define the preferred area, and the box outside is used to constrain the objects to be within this area. Thus, if the object has any its bodies exceeding this outside box, they will not be considered.

Matching

Consider two color feature vectors, f_c^A of region A, and f_c^B of region B. For each color c_i in f_c^A, the closest color c_k^B in f_c^B is found, and the distance d(c_i^A,f_c^B) is calculated as the weighted Euclidean distance in RGB space: d(c_i^A,f_c^B)=|p_i^A-p_k^B| d(c_i^A,c_k^B). The distance between the two color feature vectors is now $\sum_{i=0}^{n^A}d(c_i^A,f_c^B)$ + $\sum_{i=0}^{n^B}d(c_i^B,f_c^A)$. The distance between two texture feature vectors is the L₁-distance. The distance between two shape feature vectors is the Euclidean distance. Indexing is based on the SS-tree [WJ96]. Color, texture, and shape are indexed separately. The first feature the user specifies is used to retrieve about 100 candidates. Then this feature and the possible other features together are used to order the retrieval result.

Result presentation

The matched images are linearly ordered, see figure 18.

Applications

An initial prototype of NETRA is used in ADLADL (see above) to search on texture.