Query by Pictorial Example

Pavel Vácha, supervisor: Michal Haindl
pdf, summary, presentation, demos


Dept. of Pattern Recognition
Institute of Information Theory and Automation

Abstract:
retrieval screenshot Appearance of real scenes is highly dependent on actual conditions as illumination and viewpoint, which significantly complicates automatic analysis of images of such scenes. In this thesis, we introduce novel textural features, which are suitable for robust recognition of natural and artificial materials (textures) present in real scenes. These features are based on efficient modelling of spatial relations by a type of Markov Random Field (MRF) model and we proved that they are invariant to illumination colour, cast shadows, and texture rotation. Moreover, the features are robust to illumination direction and degradation by Gaussian noise, they are also related to human perception of textures.

The features were favourably tested on current textural databases (Outex, Bonn BTF, CUReT, ALOT, and KTH-TIPS2), where they outperformed state of the art methods (including opponent Gabor features, LBP, LBP-HF, and MR8-LINC) in almost all experiments, e.g. the results on ALOT dataset was improved by 20%. We applied our features in construction of a content-based tile retrieval system, optimisation of texture compression parameters in accordance with human perception, invariant segmentation of multimodal textures, and recognition of glaucomatous tissue in retina images.

The presented methods can improve existing CBIR systems or they can be utilised in domain specific CBIR systems focused on structural/textural similarity. Other possible applications include computer vision, since the analysis of real scenes often requires a recognition of textures in variable conditions.


Online demonstrations:
The input image is selected simply by clicking on it and the system finds the other most similar images. The images are considered to be similar if their structure is similar, regardless to variations of acquisition conditions (e.g. illumination colour - specified on the right). Comparison with alternative state-of-the-art methods can be explored by clicking on the "settings" button. The first demonstration explores classification performance (showing the nearest training images), while the others focus on retrieval of similar images from the database.

Rotation and illumination invariance
(ALOT dataset)
 
Illumination spectrum/colour invariance
(Outex dataset)
 
Illumination direction robustness
(CUReT dataset)
 
Content-Based Tile Retrieval System
(Sanita.cz dataset)


Selected publications:
P. Vacha, M. Haindl, and T. Suk. Colour and rotation invariant textural features based on Markov random fields, Pattern Recognition Letters, vol. 32, pp. 771-779, 2011. doi: 10.1016/j.patrec.2011.01.002,

P. Vacha and M. Haindl. Texture Recognition using Robust Markovian Features, in: LNCS proc. of MUSCLE International Workshop on Computational Intelligence for Multimedia Understanding 2011, pp. 1-12, in press. (oral presentation).

P. Vacha and M. Haindl. Natural Material Recognition with Illumination Invariant Textural Features, in: Proc. of International Conference on Pattern Recognition (ICPR'10), pp. 858-861, 2010. doi: 10.1109/ICPR.2010.216, (oral presentation).

R. Kolar and P. Vacha. Texture analysis of the retinal nerve fiber layer in fundus images via Markov Random Fields, in: Proc. of IFMBE World Congress on Medical Physics and Biomedical Engineering, vol. 25/XI, pp. 247-250, 2009. doi: 10.1007/978-3-642-03891-4_66, (oral presentation).

P. Vacha and M. Haindl. Illumination Invariants Based on Markov Random Fields, in: Proc. of International Conference on Pattern Recognition (ICPR'08), pp. 1-4, 2008. doi: 10.1109/ICPR.2008.4761375, (oral presentation).