Towards an interactive index structuring system for content-based image retrieval in large image databases

Alain Boucher; Hien Phuong Lai; Jean-Marc Ogier; Muriel Visani

摘要

In recent years, the expansion of acquisition devices such as digital cameras, the development of storage and transmission techniques and the success of tablet computers facilitate the development of many large image databases as well as the interactions with the users. This thesis [1] deals with the problem of Content-Based Image Retrieval (CBIR) on these huge masses of data. Traditional CBIR systems generally rely on three phases: feature extraction, feature space structuring and retrieval. In this thesis, we are particularly interested in the structuring phase (normally called indexing phase), which plays a very important role in finding information in large databases. This phase aims at organizing the visual feature descriptors of all images into an efficient data structure in order to facilitate, accelerate and improve further retrieval. We assume that the feature extraction phase is completed and the image feature descriptors which are usually low-level features describing the color, shape, texture, etc. of all images are available. Instead of traditional structuring methods, clustering methods which organize image descriptors into groups of similar objects (clusters), without any constraint on the cluster size, are studied. The aim is to obtain an indexed structure more adapted to the retrieval of high dimensional and unbalanced data. Clustering process can be done without prior knowledge (unsupervised clustering) or with a limited amount of prior knowledge (semi-supervised clustering). Due to the “semantic gap” between high-level semantic concepts expressed by the user via the query and the low-level features automatically extracted from the images, the clustering results and therefore the retrieval results are generally different from the wishes of the user. In this thesis, we proposed to involve the user in the clustering phase so that he/she can interact with the system so as to improve the clustering results, and thus improve the performance of the further retrieval. The idea is as follows. Firstly, images are organized into clusters by using an initial clustering. Then, the user visualizes the clustering result and provides feedback to the system in order to guide the re-clustering phase. The system then re-organizes the dataset by using not only the similarity between objects, but also the feedback given by the user in order to reduce the semantic gap. The interactive loop can be iterated until the clustering result satisfies the user. In the case of large database indexing, we assume that the user has no prior knowledge about the image database. Therefore, an unsupervised clustering method is suitable to be used for the initial clustering, when no supervised information is available yet. Then, after receiving the user feedback in each interactive iteration, a semi-supervised clustering can be used for the re-clustering process. Based on a deep study of the state of the art of different unsupervised clustering methods [4] as well as semi- supervised clustering approaches [2, 3], we propose in this thesis a new interactive semi-supervised clustering model [3] involving the user in the clustering phase in order to improve the clustering results. From the formal analysis of different unsupervised clustering methods [4], we chose to experiment some methods which appear to be the most suitable to be used in an incremental context involving the user in the clustering stage. The hierarchical BIRCH unsupervised clustering (Zhang et al., 1996) which gives the best performance from these experiments [4] is chosen to be used as the initial clustering in our model. Then, an interactive loop in which the user provides the feedback to the system and the system re-organizes the image database using the new semi-supervised clustering method proposed in this thesis is iterated until the clustering result satisfies the user. As the user has no prior knowledge about the image database, it is difficult for him/her to label the clusters or the images in the clusters using classes. Therefore, we provide to the user an interactive interface allowing him/her to easily visualize the clustering result and give feedback to the system. Based on the majority of the images displayed for each cluster, the user can specify, by some simple clicks, relevant or non-relevant images for each cluster. The user can also drag and drop images between clusters in order to change the cluster assignment of some images. Then, supervised information is deduced from the user feedback in order to be used for the re-clustering phase using the proposed semi-supervised clustering method. According to our study of the state of the art of different semi-supervised clustering methods, supervised information may consist of class labels for some objects or pairwise constraints (must-link or cannot-link) between objects. The experimental analysis of different semi-supervised clustering methods in the interactive context [2, 3] shows a high performa

机译：近年来，诸如数码相机之类的采集设备的扩展，存储和传输技术的发展以及平板电脑的成功推动了许多大型图像数据库的开发以及与用户的交互。本文[1]针对这些海量数据处理基于内容的图像检索（CBIR）问题。传统的CBIR系统通常依赖于三个阶段：特征提取，特征空间结构化和检索。在本文中，我们对结构化阶段（通常称为索引阶段）特别感兴趣，该阶段在大型数据库中查找信息方面起着非常重要的作用。此阶段旨在将所有图像的视觉特征描述符组织成有效的数据结构，以促进，加速和改善进一步的检索。我们假设特征提取阶段已完成，并且图像特征描述符通常是描述所有图像的颜色，形状，纹理等的低级特征，并且可用。代替传统的构造方法，研究了将图像描述符组织成相似对象（群集）组的群集方法，而对群集大小没有任何限制。目的是获得更适合于检索高维和不平衡数据的索引结构。可以在没有先验知识（无监督的聚类）或有限数量的先验知识（半监督的聚类）的情况下完成聚类过程。由于用户通过查询表达的高级语义概念与从图像中自动提取的低级特征之间存在“语义鸿沟”，因此聚类结果以及因此的检索结果通常与用户的意愿不同。在本文中，我们建议让用户参与聚类阶段，以便他/她可以与系统交互，从而改善聚类结果，从而提高进一步检索的性能。这个想法如下。首先，通过使用初始聚类将图像组织为聚类。然后，用户可视化聚类结果并向系统提供反馈，以指导重新聚类阶段。然后，系统不仅使用对象之间的相似性，还使用用户提供的反馈来重新组织数据集，以减少语义鸿沟。可以迭代交互式循环，直到聚类结果使用户满意为止。在大型数据库建立索引的情况下，我们假定用户没有图像数据库的先验知识。因此，当尚无监督信息可用时，无监督聚类方法适用于初始聚类。然后，在每个交互式迭代中收到用户反馈后，可以将半监督聚类用于重新聚类过程。在深入研究各种无监督聚类方法[4]和半监督聚类方法[2，3]的技术现状的基础上，本文提出了一种新的交互式半监督聚类模型[3]，该模型涉及用户处于聚类阶段，以改善聚类结果。通过对不同的无监督聚类方法的形式分析[4]，我们选择实验一些似乎最适合在聚类阶段涉及用户的增量上下文中使用的方法。从这些实验中获得最佳性能的分层BIRCH无监督聚类（Zhang等，1996）被选为模型中的初始聚类。然后，交互交互过程，在该交互过程中，用户将反馈提供给系统，然后系统使用本文提出的新型半监督聚类方法重新组织图像数据库，直到聚类结果满足用户为止。由于用户没有图像数据库的先验知识，因此他/她很难使用类来标记群集或群集中的图像。因此，我们为用户提供了一个交互式界面，使他/她可以轻松地可视化聚类结果并向系统提供反馈。基于为每个群集显示的大多数图像，用户可以通过一些简单的单击来指定每个群集的相关图像或不相关图像。用户还可以在群集之间拖放图像，以更改某些图像的群集分配。然后，使用建议的半监督聚类方法从用户反馈中推断出监督信息，以便将其用于重新聚类阶段。根据我们对不同的半监督聚类方法的研究现状，监督信息可能包含某些对象的类标签或对象之间的成对约束（必须链接或不能链接）。在交互上下文中对不同的半监督聚类方法进行的实验分析[2，3]显示了较高的性能

Towards an interactive index structuring system for content-based image retrieval in large image databases

摘要

著录项

相似文献

相关主题

期刊订阅