Encrypted Sensing Based on Digital Holography for Fingerprint Images

We propose a novel biometric sensing technique for personal authentication in which fingerprint images are captured using an optical encryption method. This method can reduce the risk of data theft or leakage of personal information captured by biometric sensing. This method, termed encrypted sensing, is implemented using digital holography with double random phase encoding. We demonstrate experimentally that a fingerprint image can be captured as an optically encrypted image and can be restored correctly only when the correct cipher key is used. Moreover, we investigate experimentally the verification accuracy of the decrypted images.


Introduction
Recently, there has been a rise in the use of biometric authentication systems in a wide variety of secure interfaces, including computer logins, banking systems, automated teller machines, and building entrance management.Because biometric information represents important personal data, and because enrolled biometric information, unlike most personal authentication information such as passwords, cannot be altered, it is necessary to take great care to prevent leakage of these data.Correspondingly, template protection techniques for biometric authentication have been actively pursued through the development of cryptographic algorithms that protect raw biometric information obtained using a sensor while preserving high verification accuracy [1]- [4].However, particular threats, such as side-channel attacks [5] occurring while raw biometric information is being transformed into a protected template, are becoming increasingly realistic.It is believed that threats such as these can be avoided by encrypting biometric images prior to image sensing, i.e., optical encryption.
Optical encryption techniques apply optical calculation methods such as optical wave propagation oroptical phase modulation for image encryption and decryption.These techniques are advantageous not only for rapid calculations using parallel processing, but also security enhancement due to non-electronic computation methods, because there is less risk of optical information being read, as opposed to digital data in electronic devices, which are inevitably exposed to a number of security risks.Double random phase encoding (DRPE) [6] is a well-known optical encryption method that has been studied extensively for two decades.DRPE is an optical symmetric-key encryption technique in which an optical image is encrypted by multiplying two random phase masks in the spatial and Fourier or Fresnel planes.To obtain an optically-encrypted image, digital holography (DH) [7]- [10] is commonly employed.DH makes it possible to not only capture an optical wavefront as holographic data, but also to reconstruct various styles of optical wavefront computationally from the holographic data.DRPE has been implemented using DH [7] [11], whereby the random phase masks are typically inserted into the object beam path.Suchan optical system requires precise alignment; however, it is challenging to reconstruct the object image with high quality from the DH.Tajahuerce and Javidi proposed an optical encryption system for three-dimensional (3D) objects [12], whereby an encrypted image was obtained with simple alignment because only the reference beam was modulated using a random phase mask, and a non-modulated wave was used as the object beam.This optical arrangement could be implemented easily and can restore a clear object image from a DH.However, the encrypted image was not equivalent to DRPE because the object beam was not modulated by a random phase mask, which might decrease the security level of encryption.To obtain a fingerprint image for biometric authentication, both sufficient security to prevent data leakage and sufficient image quality for highly accurate recognition are required.Therefore, we propose an optical security system that is suitable for obtaining fingerprint images as optically encrypted holographic data.We term this system "encrypted sensing".In addition, we also propose an image reconstruction method through which we can obtain a clear fingerprint image from the encrypted hologram.To obtain the reconstructed image with high quality, we apply speckle reduction using multiple holograms, as well as reconstruction using a phase only mask, rather than a complex amplitude mask.We demonstrate that improved recognition accuracy is obtained by applying only a phase-only mask and speckle reduction.
The remainder of the paper is organized as follows.In the following section, a theoretical description of the proposed sensing system is given.In Section 3, experiments for obtaining encrypted fingerprint images are described, together with an investigation of the quality of the decrypted fingerprint images.Conclusions are drawn in Section 4.

User Authentication Scheme
Figure 1 shows a schematic diagram that illustrates the concept of biometric encrypted sensing proposed here.During enrollment, biometric information (i.e., a fingerprint pattern) is encrypted using the optical system within the biometric sensor, and the encrypted information is transmitted to the verification system for user authentication.This encrypted biometric information is then decrypted using a corresponding decryption key and the biometric information is stored in the biometric template database.During verification, encrypted biometric information is also captured, and it is then sent to the verification system.Subsequently, it is also decrypted in the verification system, and the biometric information and the enrolled data are verified by calculating their similarity.If both the biometric image and the encryption key are provided correctly, the user is authorized.This system makes it possible to perform two-factor authentication that will authorize users only when both biometric and the secret data are verified.The verification system, which is assumed to be a securely protected using, for example, a tamper-resistant device can be located in either a centralized server or a user device.Throughout this overall process, raw biometric information in digital form exists only in the protected verification system, and the risk of theft or leakage of biometric data is thus significantly reduced.

Encrypted Sensing Based on DRPE
In our system, optical encryption is performed via a combination of DH and DRPE.With DH, a wavefront encoding the optical information from an object wave is typically captured via interference with a reference plane wave, as shown in Figure 2. In ref. [12], random phase-modulated light was used as the reference wave, which is used as an encryption key.In our system, both the object wave and the reference wave are modulated using diffusers, as shown in Figure 3.To capture fingerprints stably, we place a total reflection prism in the middle of the object wavepath.The resulting DH is captured using a charged coupled device (CCD) image sensor, and can be expressed as follows: where , O x y represents the fingerprint image, ( ) is the phase modulation by the diffuser in the object wavepath, ( ) x y is the complex amplitude distribution of the phase-modulated reference wave on the CCD plane, ( ) x y and ( ) ( ) x y is derived as the Fresnel transform of the phase object assigned by the diffuser D2 (see Figure 3).
Figure 4 shows a flow chart of the Fresnel-type DRPE process.An encrypted image is obtained via two phase modulations-one in the space domain, and the other in the Fresnel domain.With the approaches described in Refs.[7] and [11], two phase masks were inserted in the object wave path for DRPE; however, with our system,    they are applied to the object and reference waves.The Fresnel-type DRPE uses three cipher keys: the phase distribution on the Fresnel domain, the distance of Fresnel diffraction, and the wavelength of laser; all of which that must be provided correctly to restore the plaintext image from the encrypted image.In this paper, the laser wavelength is assumed to be unique, whereas the other two components are employed as cipher keys.
Let us consider the third term of Equation ( 1), which is similar to DRPE.If the amplitude of ( ) x y can be approximated to be constant, the third term of Equation ( 1) corresponds to the encrypted image of Fresnel-type DRPE.In this case, ( ) x y can be expressed as where A E is a constant amplitude and , E K x y is the phase distribution of the encryption key.The encrypted image ( ) where the constant A E has been omitted for clarity.The encrypted image , C x y can be derived by recon- structing the first-order diffracted wave from the digital hologram captured by the CCD sensor.
For decryption, , C x y is multiplied by the phase term of the decryption key image , D K x y as fol- lows: After this calculation, the complex amplitude image , r C x y is Fresnel-diffracted at a distance ẑ , and a decrypted image ˆ, O x y is obtained by calculating its amplitude components as follows: ( ) ( ) where ( ) If ẑ z = − , this yields the following: From this equation, it follows that the original fingerprint image is obtained.
If the amplitude of ( ) x y cannot be approximated to be constant, the complex amplitude of ( ) x y must be obtained, which entails division process by its amplitude component in the decryption process, as described in Ref. [12].This process can lead to amplification of the error of a restored fingerprint image, because some of the amplitude component of ( ) x y may be very small.In our experimen- tal analysis below, we investigate both cases in which E K R has either a constant amplitude (our method) or a complex amplitude distribution (similar to Tajahuerce's method).
If the encryption and decryption keys differ significantly, the decrypted image becomes a random pattern.The diffraction distance z can be also used as a parameter to restore the correct plaintext image.

Obtaining the Decoding Key
To implement a recording system for encrypted DH, we employ off-axis holography, in contrast to the work described in Ref. [12], where they used in-line holography.To reconstruct a fingerprint image from an encrypted hologram, ( ) x y must be obtained in advance.To implement this with off-axis holography, a Fourier transform-based method [13] was employed.
In the optical system shown in Figure 5, a hologram , , x y and a plane wave, i.e., (

, exp 2π
p A x y j ax by = − + is a unit-amplitude plane wave, where a and b represent the angles between , x y and the plane wave.To eliminate the first and second terms of Equation ( 8), the intensity distribution of ( ) x y is recorded by shielding the plane wave, and , p A x y (which is constant) and ( ) , x y .Subsequently, the remaining terms are Fouri- er-transformed and we have where ( ) , u v are coordinates on the Fourier domain, ( ) , x y , and ⊗ denotes a convolution operator.If a and b are large enough to separate the two terms in Equation ( 9) in the spectral domain, the first and second terms do not overlap, and the first term can be extracted by shielding the second term.By calculating the inverse Fourier-transform, the complex amplitude distribution of ( ) x y can be determined.

Optical System
We carried out several experiments in which holograms of encrypted fingerprint images were captured and the original fingerprint images were then numerically restored.To do this, we constructed an optical system for acquiring holograms, as shown in Figure 6, which used a He-Ne laser with a wavelength of 633 nm as a light source and a CCD image sensor with 1024 × 768 pixels that was 4.65 μm × 4.65 μm.A fingerprint was placed on a total reflection prism and the reflected light was focused using two lenses (L3 and L4) to form an image on the CCD image sensor.

Image Reconstruction
As discussed in Section 2.2, the decrypted images were obtained so that ( ) x y was either treated as a complex amplitude wave (as with Tajahuerce's method) or with the amplitude component replaced with a constant value (our method).Examples of an encrypted hologram   component and the correct propagation distance.In this case, the fingerprint image is difficult to discern. Figure 7 (c) shows a decrypted image using the correct , E K R y with a constant amplitude and the correct propagation distance; the fingerprint pattern can be clearly seen in the decrypted image.These results indicate that better quality fingerprints can be obtained by using a constant amplitude for ( ) x y than by using a complex amplitude.However, the fingerprint pattern in Figure 7(c) contains significant speckle noise [14] [15], although it is sufficiently clear to recognize using the human eye.To reduce the amount of noise, we captured multiple holograms with differing speckle noise by changing the position of diffuser D1on the object beam path and summing the intensities of the decrypted images obtained from the holograms.Figure 7(d) shows a decrypted image obtained by summing four decrypted images, in which the restored fingerprint is more clearly visible than in Figure 7(c).
Figures 7(e) shows a decrypted image using an incorrect ( ) x y , and Figure 7(f) shows a decrypted image using an incorrect propagation distance.In both cases, random patterns similar to stationary white noise were obtained.

Verification Accuracy
We evaluated the verification accuracy of the experimental setup described above in terms of the ability to match restored fingerprint images.Using the optical system shown in Figure 6, we captured 100 encrypted holograms of fingerprint images from 10 subjects (10 encrypted DHs per fingerprint).In these trials, five of the fingerprint images for each subject were used for enrollment and the others were used for testing; 250 genuine fingerprint trials (25 trials per subject) and 450 imposter trials (45 trials per subject) were carried out.Nonlinear correlation [16] was employed in these trials for pattern matching.The nonlinear correlation n(x,y) between two images can be expressed as follows: F u v and k is the degree of the applied nonlinearity.We choose k = 0.3, which was found to provide optimal discrimination between genuine and imposter cases.Figure 8 shows the nonlinear correlation output; the correlation waveforms exhibited a sharp peak when verifying the correct fingerprint (a genuine case) with a constant amplitude ( ) x y , and with speckle noise reduction, the correlation peak became sharper.Figure 8(d) shows nonlinear correlation in case of verification between a correct and an incorrect decryption keys.Although both decrypted images were obtained from the same encrypted hologram, the nonlinear correlation did not exhibit a peak.Figure 8(e) shows the correlation waveform for imposter verification; here we also see a random pattern.
As a quantitative criterion for verification, the peak-to-side-lobe ratio (PSR) [17] was calculated as follows: peak mean , PSR σ − = (11) where σ denotes the standard deviation of ( ) , n x y and peak and mean denote the peak value and spatial aver- age of ( ) , n x y , respectively.
Using PSR as a verification score, the false rejection rate (FRR) of the genuine verification attempts and the false acceptance rate (FAR) of the imposter verification attempts were calculated.In addition, we determined a receiver operating characteristic (ROC) curve, in which each point is plotted according to the FRR (x-axis) and FAR (y-axis) for a given threshold value, as shown in Figure 9.The equal error rate (EER) of our method with speckle noise reduction was 12.4%, and that without speckle noise reduction was 35.0%.The EER obtained using a complex amplitude ( ) x y was 45.0%.These results indicate that the phase-only method proposed here provides better image verification, and that the speckle noise reduction contributes significantly to the improvement in the verification accuracy.
The accuracy of our system was not better than that of image verification using a conventional sensor (i.e., unencrypted sensing).However, there are a number of possible causes of image degradation: first, we did not use a guide to fix the fingerprints; and second, we did not ask the subjects to restrain the pressure that they applied.The latter issue might cause a variation in the degree of fingerprint distortion among the images.In such cases, speckle reduction via summation may not be effective.By reducing the influence of such factors, our method has potential for further improvements in verification accuracy.

Conclusions
We have described an encrypted sensing system for capturing fingerprint images using digital holography and  optical encryption.We showed experimentally that a fingerprint image could be restored correctly by decrypting an encrypted hologram with the correct key and propagation distance.The quality of the restored fingerprint image could be more clearly visible than that when using a conventional method by improving the calculation method for restoring a fingerprint image from an encrypted digital hologram.The verification performance of the restored fingerprint images was investigated, and we demonstrated that our method has favorable verification accuracy.
The principal advantage of the system described here is that it can enhance the security of biometric authentication by capturing optically encrypted images rather than raw fingerprints.Combining physically unclonable functions (PUFs) [18] with our system might enable the development of very secure user authentication methods.
For example, the use of an encoding key provided by a diffuser as a PUF may provide a secure key based on the physical features of an individual object.

1 1 ,
x y are coordinates of the object and the CCD planes, and * denotes the complex conjugate.The operator [ ] , FsT f z denotes a Fresnel transform in which f and z are the original wa- vefront and the propagation distance, respectively.

Figure 1 .
Figure 1.Concept of the encrypted biometric sensing system.The database (DB) in the verification system represents the template database for user biometric information.

Figure 2 .
Figure 2. Typical optical system used in digital holography.

Figure 3 .
Figure 3.The optical system used with our encrypted sensing scheme.Here D1 and D2 are diffusers.

2 2 ,
x y are coordinates on the new Fresnel transform domain.If the encryption key and the corresponding decrypted im- ages are shown in Figure 7. Figure 7(a) shows that the fingerprint image is not visible in the encrypted hologram ( ) , O I x y .Figure 7(b) shows adecrypted image using the correct ( )

Figure 5 .
Figure 5. Optical system for obtaining decoding key.
(m = 1, 2) denotes the Fourier transform of each image, IFT denotes inverse Fourier transform operator,