Deep Learning-Based Hyperspectral Multimodal Biometric Authentication System Using Palmprint and Dorsal Hand Vein

INTRODUCTION

Biometric recognition system has been widely used in the construction of a smart society. Many types of biometric systems, including face, iris, palmprint, palm vein, dorsal hand vein, and fingerprint, currently exist in security authentication. Palmprint recognition system is a kind of reliable authentication technology, due to the fact that palmprint has stable and rich characteristics, such as textures, local orientation features, and lines. In addition, a palmprint is user-friendly and cannot be easily captured by a hidden camera device without cooperation from the users. However, palmprint images captured using a conventional camera cannot be used in liveness detection. Palm vein is a good remedy for the weakness of palmprint acquired using a near-infrared (NIR) camera. The vein pattern is the vessel network underneath human skin. It can successfully protect against spoofing attacks and impersonation. Similar to palm vein, dorsal hand vein also has stable vein structures that do not change with age. Besides vein networks, some related characteristics to palmprint such as textures and local direction features can also be acquired.

Up to now, palmprint and dorsal hand vein-based recognition methods have achieved competitive performances in the literature. Huang et al. [1] put forward a method for robust principal line detection from the palmprint image, even if the image contained long wrinkles. Guo et al. [2] presented a binary palmprint direction encoding schedule for multiple orientation representation. Sun et al. [3] presented a framework to achieve three orthogonal line ordinal codes. Zhao et al. [4] constructed a deep neural network for palmprint feature extraction, where a convolutional neural network (CNN)-stack was constructed for hyperspectral palmprint recognition. Jia et al. presented palmprint-oriented lines in [5]. Khan et al. [6] applied the principle component analysis (PCA) to achieve a low-dimensionality feature in dorsal hand vein recognition. Khan et al. [7] obtained a low-dimensionality feature representation with Cholesky decomposition in dorsal hand vein recognition. Lee et al. [8] encoded multiple orientations using an adaptive two-dimensional (2D) Gabor filter in dorsal hand vein feature extraction.

The palmprint and dorsal hand vein recognition is usually carried out by conventional and deep learning-based methods. The conventional methods need to design a filter to extract the corresponding feature, i.e., local direction, local line, principal line, and texture. These hand-crafted algorithms usually require rich prior knowledge based on the specific application scenario. PalmCode [9] encoded palmprint features on a fixed direction by using a Gabor filter. Competitive code [10] extracted the dominant direction feature by using six Gabor filters. Xu et al. [11] encoded a competitive code aiming to achieve the accurate palmprint dominant orientation. Fei et al. [12] detected the apparent direction from the palmprint image. In addition, Huang et al. [13] put forward a centroid-based circular key-point grid (CCKG) pattern in dorsal hand vein recognition, which extracts local features based on key- points detection. Deep learning-based algorithms require a mass of training data to train the parameters in the deep convolutional neural network (DCNN). Afterwards, the optimal DCNN can be utilised for classification or convolution feature extraction. However, a mass of training data is usually unavailable for a palmprint or dorsal hand vein recognition task. Especially, the transfer learning technology with DCNN supports an approach that a pretrained DCNN can be fine-tuned with a few training samples for classification in a specific application. Zhao et al. [14] proposed a deep discriminative representation method, which extracted palmprint features from deep discriminative convolutional networks (DDCNs). DDCNs contain a pretrained DCNN and a set of lightened CNNs corresponding to the global and local patches segmented from the palmprint image. Wan et al. [15] trained the VGG depth CNN to extract dorsal hand vein features and used the logistic regression for identification. Deep learning-based methods can be widely applied in generic application scenarios.

Increasing research studies have moved to the area of hyperspectral imagery technology in the past decades. Contrary to the traditional imagery technology, not only skin texture but also vascular networks are imaged using the designed hyperspectral imagery system with the specific spectrum setup. In the phase of imaging palmprint or dorsal hand combined hyperspectral technology, more discriminative information from the palmprint or dorsal hand image can be captured achieving a high recognition rate. With more than 60 bands covered in hyperspectral palmprint, the three- dimensional (3D) feature was extracted through 3D Gabor filters [16]. Due to the redundant data, hyperspectral palmprint authentication improved but not remarkably when every spectral data were considered in the feature extraction phase. Based on band combination, Shen et al. [17] clustered typical bands in hyperspectral palmprint images for authentication, which performed better compared with in Ref. [16], while Guo et al. [18] applied an approach of к means algorithm for representative band selection in hyperspectral palmprint database to improve performance. What’s more, the band clustering method can decrease computation and increase efficiency in hyperspectral biometrics. As is known, dorsal hand vein and palmprint are concentrated in one hand, which makes it more convenient to collect these two different modalities simultaneously. Based on this observation, the combination of hyperspectral palmprint and dorsal hand biometrics is developed to meet a higher security requirement and to guarantee an exceptional recognition performance. In addition, unimodal biometrics recognition based on a single trait easily suffers from spoofing and other attacks as stated in the literature [19,20]. Table 1.1 illustrates the survey of the current multimodal biometric recognition algorithms. First, it is observed from this table that palmprint and dorsal hand vein have been fused before [21]. However, Ref. [21] and the other methods in Table 1.1 used only two single-spectrum images (one for each modality) to improve the recognition performance.

Different from the literature in Table 1.1, this work will study and implement the merging hyperspectral palmprint feature into dorsal hand vein feature to develop a novel hyperspectral multimodal biometric authentication system, which is demonstrated by a flow diagram (refer to Figure 1.1). A hyperspectral acquisition device was utilised for collecting hyperspectral palmprint and dorsal hand images. Then, region of interest (ROI) is detected from hyperspectral palmprint, and dorsal hand images resulted in two corresponding ROI cubes. After ROI extraction, the optimal feature pattern, i.e., local binary pattern (LBP) [22], local derivative pattern (LDP) [9], 2D-Gabor [2], and deep convolutional feature (DCF) [23], is selected for the palmprint and dorsal hand vein, correspondingly. In the pattern selection procedure, each image in the ROI cube is extracted and its features are used in recognition. Thus, the pattern and band which can achieve the highest recognition are treated as the optimal pattern and band for hyperspectral palmprint and dorsal hand images. Afterwards, the feature corresponding to the optimal pattern from palmprint on the optimal band and the feature concerning to the optimal pattern from dorsal hand vein on the optimal band are merged as one feature vector. At last, this fused multimodal feature vector is directly used in matching with the 1-NN classifier.

TABLE 1.1

The Survey of Multimodal Biometric Recognition Algorithms

Literature

Algorithms

Modalities

Features

Year

[19]

Concatenation

Palmprint and hand-geometry

Line features; hand lengths and widths

2003

[20]

Combined face-plus-ear image

Face and ear

PCA

2003

[24]

Concatenation

Face and hand

PCA, linear discriminant analysis (LDA) and 9-byte features

2005

[25]

Concatenation

Face and palmprint

2D-Gabor PCA

2007

[26]

Concatenation

Fingerprint and face

Minutia features

2007

[27]

Concatenation

Side face and gait

PCA

2008

[28]

Fusion

Palmprint and fingerprint

Discrete cosine transforms

2012

[29]

Fusion

Profile face and ear

Speeded up robust features (SURF)

2013

[30]

Concatenation

Palmprint and fingerprint

Bank of 2D-Gabor

2014

[31]

Weighted concatenation

Face and ear

PCA

2015

[32]

Feature level

Iris, face and fingerprint

Group sparse representation- based classifier (GSRC)

2016

[21]

Score level

Palmprint and dorsal hand vein

Mean and average absolute deviation (AAD) features

2016

[33]

Bayesian decision fusion

Face and ear

CNN features

2017

[34]

Score level

Finger-vein and finger shape

CNN features

2018

[35]

Concatenation

Face and ear

CNN features

2017

The flowchart of the designed system-merged hyperspectral palmprint feature with dorsal hand feature

FIGURE 1.1 The flowchart of the designed system-merged hyperspectral palmprint feature with dorsal hand feature.

The major contributions in the chapter are briefly introduced as follows:

  • 1. A novel real-time hyperspectral multimodal biometric authentication system is conceived. It captures hyperspectral hand images by the proposed hyperspectral imaging acquisition device under 53 spectrums in the range of 520-1040 nm with intervals of 10 nm.
  • 2. We collected a big multimodal dataset containing hyperspectral palmprint and dorsal hand images using the designed device. More information about this dataset can be found in Section 1.4.1.

The remaining work is organised as follows. In Section 1.2, the designed capture device is introduced. Following this, the designed system is illustrated in Section 1.3, including ROI and feature extraction as well as multimodal fusion and matching. Extensive experiments and analysis are included in Section 1.4, while Section 1.5 concludes the proposed system.

 
Source
< Prev   CONTENTS   Source   Next >