Visible-Infrared Person Re-Identification via Homogeneous Augmented Tri-Modal Learning

Document Type

Article

Source of Publication

IEEE Transactions on Information Forensics and Security

Publication Date

1-1-2021

Abstract

© 2005-2012 IEEE. Matching person images between the daytime visible modality and night-time infrared modality (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing methods usually learn the multi-modality features in raw image, ignoring the image-level discrepancy. Some methods apply GAN technique to generate the cross-modality images, but it destroys the local structure and introduces unavoidable noise. In this paper, we propose a Homogeneous Augmented Tri-Modal (HAT) learning method for VI-ReID, where an auxiliary grayscale modality is generated from their homogeneous visible images, without additional training process. It preserves the structure information of visible images and approximates the image style of infrared modality. Learning with the grayscale visible images enforces the network to mine structure relations across multiple modalities, making it robust to color variations. Specifically, we solve the tri-modal feature learning from both multi-modal classification and multi-view retrieval perspectives. For multi-modal classification, we learn a multi-modality sharing identity classifier with a parameter-sharing network, trained with a homogeneous and heterogeneous identification loss. For multi-view retrieval, we develop a weighted tri-directional ranking loss to optimize the relative distance across multiple modalities. Incorporated with two invariant regularizers, HAT simultaneously minimizes multiple modality variations. In-depth analysis demonstrates the homogeneous grayscale augmentation significantly outperforms the current state-of-the-art by a large margin.

ISSN

1556-6013

Publisher

Institute of Electrical and Electronics Engineers (IEEE)

Volume

16

First Page

728

Last Page

739

Disciplines

Computer Sciences

Keywords

multi-modality, Person re-identification (Re-ID), ranking

Indexed in Scopus

no

Open Access

no

Link to Full Text

Share

COinS