Semantic Face Index
With the increasing use of cheap cameras (e.g., available in cell phones) coupled with the ease of distribution, there is an exponential increase in the size of available face databases. An important key challenge is in the ability to search over these large databases in order to identify a face that may be of interest. This is significantly compounded in the case of face-in-the-wild, where the query face, the database faces, or both might have been captured in a non-constrained, non-ideal environment, with no control on the image quality, the illumination conditions, pose, partial occlusion, etc.
We are proposing a human-readable indexing scheme to reduce these problems, and thus support a rapid access in very large scale face databases. The index is based on descriptors that are understandable to humans, which makes the approach easy to use, and suitable for law enforcement applications.
WVU has data from several previous collection efforts, for both standard face images, and faces-in-the-wild. We also have access to publicly available large collections of face images, such as FRGC and FERET. Task 1: Facial Descriptors: We will first develop a framework for extracting novel facial descriptors suitable for human-readable indexing. Our semantic descriptors will be understandable for humans, and will build on key facial features, facial landmarks, and facial regions. To support matching under non-ideal conditions, we will perform normalization and feature post-processing using global and local contexts in the face.
This project consists of the following Tasks.
- to represent the descriptors for indexing, we will generate discrete representations of the descriptors, for instance, using a family of hash codes. A given face it can then be represented using a sequence of these hash codes. We then build an index structure on these discrete representations using a generalized suffix tree for rapid search and retrieval,
- test our proposed method using existing dataset of face images, both with WVU and other publicly available face images, example FERET, FRGC, and Audience dataset of faces-in-the-wild. Using these datasets, we will evaluate the performance of our approach using popular indexing and retrieval metrics of speed and penetration, precision, and recall,
- and evaluate performance of our approach in the presence of various forms of non-idealities, such as variations in illumination, pose, image quality, and partial occlusion.