Skip to main content

Projects

Non-cooperative Human Identification

Searching for a person in a large video archive, possibly made of videos acquired by a network of surveillance cameras, is a fundamental task because it allows tracing down when and where a person was present in the scene. For instance, it allows saving hours of manual video inspection to find and trace the presence of criminals, like those in the Boston Marathon bombings.

In such a scenario, quite often the human identification task can only rely on biometric traits acquired in unconstrained conditions from noncooperative subjects. More specifically, this means that both gallery and probe images of someone’s trait may be heavily corrupted by noise or other nuisance factors, such as the pose of an individual, or the illumination and occlusions. In such hostile conditions, human identification is typically attempted via face recognition (if enough image resolution is provided).

Although it has improved significantly even with corrupted probe and gallery images, there are still plenty of unfriendly scenarios where face recognition lack robustness. Hence, there is a critical need to reduce those cases in order to increase the effectiveness of human identification for searching unconstrained video archives.

The main goal of this project is to improve human identification by fusing the face modality with a pseudo-modality such as the clothing appearance of individuals. Using multiple biometric modalities is an effective approach for increasing identification robustness. Besides face, other viable options for the considered scenario include gait, and clothing appearance as shown in Fig.1. The extraction and characterization of human gait is not practical, unless specific conditions are met, which are typically too restrictive. The clothing appearance is not a biometric trait. However, its effectiveness in matching the identity of people that between sightings have not changed their clothes has been demonstrated, which is why it was chosen.

The objectives of this research include:

  1. developing a human identification approach jointly based on the face modality and the pseudo-modality of clothing appearance;
  2. improving our current single modality identification approaches based on face, and based on clothing appearance;
  3. demonstrating the performance of the joint face-clothing approach on our surveillance video archive.