Applying multi-focus filtering plane in double-step Fresnel diffraction method for enhanced holographic projection region

Abstract

Computer-generated holography has been anticipated in augmented reality (AR) since it can fully provide multi-depth 3D information to users, but speckle noises are often accompanied as one of the most significant degradation factors. Since complex amplitude encoding can suppress speckle noises, applying double-phase encoding to a lensless holographic projection system using a double-step Fresnel diffraction (DSF) algorithm has been reported to reduce those noises. However, a single filtering focus at a virtual focal plane might significantly reduce the projection region compared to the maximum diffraction bandwidth of the spatial light modulator. In this paper, we applied multiple foci for the focal plane of the DSF method with off-axis phase encoding, which can expand the projection region and is applicable to display multi-depth images with DSF. Numerical analysis conducted for image quality compared with conventional single focus DSF algorithm shows that the speckle performance does not degrade so much as the number of foci increase. We expect the proposed method to be useful for holographic projection optical systems such as AR head-up display.

KEYWORDS:

• Computer-generated holography
• fresnel diffraction
• holographic display
• hologram encoding
• AR optical system

1. Introduction

Computer-generated holography has been expected to be one of the most promising technologies for expressing dynamic three-dimensional images due to its advantages of reproducing optical wavefront by controlling the amplitude and phase of light. While augmented reality (AR) systems utilized by conventional 2D micro-display make users feel uncomfortable and dizzy due to vergence-accommodation conflict (VAC), holographic displays can solve the problem by providing complete depth cues [1–3], so various studies have been conducted to employ the holography in the AR fields [4–7].

In most cases, current holographic displays generate object images in three-dimensional space by applying a computer-generated hologram (CGH) that contains phase-only information into a spatial light modulator (SLM) with coherent light source illumination. However, this process may cause serious speckle noise due to the random phase distribution and self-interference of the coherent light source [8,9]. To reduce these speckle noises, CGH can also be calculated based on the time-multiplexed methods or the iterative methods [10–12]. However, such iterative algorithms are quite difficult to be applied to systems that require real-time and high frame rate contents, such as AR-HUD. Using an incoherent light source, such as light emitting diode (LED), to generate a holographic image can also reduce the speckle noises. However, issues such as image blurring and weak brightness might limit the practical use of incoherent light-based holographic displays in AR-HUD systems [13,14].

To solve this problem, various CGH generation techniques for expressing complex amplitude information with phase-only SLM have been reported in recent years, which can be classified as encoding-based methods [15–18] and device-based methods [19–21]. For systems that simply use commercialized SLM without specific device-level modification, encoding-based methods might be preferred for the compactness of the overall system. One way to encode complex amplitude information with phase-only SLM is a double-phase hologram encoding method (DPHM), in which one complex amplitude information is decomposed into two pure phase values of constant amplitude [15]. To apply DPHM, focusing lens and iris for removing the unnecessary light are required, which make the entire system bulky.

Many studies on the propagation algorithms with two succeeded Fresnel diffraction, usually for obtaining holograms with desired magnification, have been conducted [18,22,23]. In a precedent study, a double-step Fresnel diffraction (DSF) algorithm with off-axis encoding was designed, which applied converging wavefront in SLM to generate a speckle-reduced projection-type hologram without a lens system [18]. This method can also achieve a zoomable image size by applying aliasing-reduced Fresnel diffraction with scale and shift operations (ARSS) algorithm [24]. The DSF method with off-axis encoding should be properly used for the holographic projection system such as AR-HUD in terms of speckle mitigation by reducing aliasing noise, adjustable sampling pitch between the image and hologram plane, and high-contrast by blocking non-diffractive DC-noise. However, since the position and shape of the focus on the filtering plane were fixed in DSF system, the holographic projection zone was significantly limited based on the triangular ratio between the SLM-to-filtering plane distance and the filtering-to-image plane distance. Therefore, the projection zone using the DSF algorithm is much smaller than the maximum diffraction region determined by the pixel pitch of SLM. In other words, using the maximum diffraction bandwidth of SLM with a fixed single focus filter was impossible when conventional DSF method was applied. In addition, since the entire area of SLM was not utilized, a problem of increasing ringing artifacts and intensity reduction was observed.

In this research, we propose the multi-focus DSF (MDSF) method that applies the multi-focus filtering plane to improve the spatial region of the holographic projection zone, which is also applicable for multi-focus image generation. By appropriately designing the locations of multi-focus of the filtering plane with off-axis phase encoding, the proposed method can use a much larger diffraction region of the SLM to reconstruct multi-depth images than the conventional DSF method. The feasibility of the proposed method is checked through numerical simulations and optical reconstructions. Furthermore, the numerical results show that the speckle performance measured by image quality assessment methods, such as peak signal-to-noise ratio (PSNR), mean structural similarity index measure (MSSIM), and speckle contrast (C), does not degrade so much as the number of focus increases. Hence, we expect that the proposed work can be useful in various systems that use the projection-type holographic display such as AR-HUD system [25–27].

2. Principles of the proposed method

Prior to describing the details of the CGH calculation process, we would like to briefly state the major differences between the conventional DSF and MDSF methods, which are illustrated in Figure 1(a) and (b). Since the MDSF method is based on conventional DSF, both methods have a similar reconstruction process that needs a virtual focus plane located at an intermediate depth between CGH and image planes. For calculating CGH, the virtual plane indicates the position where the wavefront from the image converges to a certain focus. This plane also indicates the position where physical iris is located for image reconstruction, to filter unnecessary light such as high-order diffraction, non-diffracted light from SLM, and conjugated fields. Here, we define the distance between CGH and the virtual plane as $d_f$ and between the virtual and image plane as $d_i$, where i indicates the image plane number. The reconstructed distance between CGH and the image plane is defined as $d_z=d_f+d_i$. Figure 1(a), shows the conventional DSF method with an off-axis encoding that uses single focus at the virtual plane must have a restricted projection region at the image plane compared to the maximum region that Fresnel diffraction from SLM can provide, which is indicated by the black solid square of the image plane. On the other hand, in an ideal case, the proposed method can utilize the maximum diffraction region as the same possible diffraction bandwidth of SLM by increasing the number of apertures at the virtual focus domain. Moreover, as shown in Figure 1(b), the image generated through each focus can provide various information at different depths even though only a single virtual plane exists. As depicted in Figure 1(b), sampling pitches of the image plane, virtual plane, and CGH plane are defined as $d{x}_i$, $d{x}_v$, $d{x}_h$, which have different values but tightly related to each other for calculation.

Figure 1 (a) Illustration of the conventional DSF method for image reconstruction, and (b) that of the proposed MDSF method, respectively. (c) Conceptual schematic of the overall CGH calculation process of the proposed method.

Figure 1(c) describes the CGH calculation process of the proposed MDSF method. At first, a spherical phase factor with a focal length of $d_i$: ${\phi }_i\left(x_i,y_i\right)$ = $-\pi ({x_i}^2+{y_i}^2)/\lambda d_i$ is multiplied by the amplitude of each image $A_i\left(x_i,y_i\right)$. This spherical phase factor converges each image at a point on the virtual plane. Therefore, the complex amplitude at each image plane is given as $U_i\left(x_i,y_i\right)=A_i\left(x_i,y_i\right)\cdot \exp \left[j{\phi }_i\left(x_i,y_i\right)\right]$.

After that, the CGH calculation of the proposed MDSF method is conducted in two steps. The first step applies the ARSS algorithm to convergence from the image to the virtual plane. If we use the complex amplitude in the image plane $U_i\left(x_i,y_i\right)$, the complex amplitude in the virtual plane is expressed by.

(1) $\begin{array}{rl}V{P}_i\left(x_v,y_v\right)& =\frac{\exp \left(jk{d}_i\right)}{j\lambda d_i}\int \int U_i\left(x_i,y_i\right)\exp \\ & \times \left[\frac{j\pi }{\lambda d_i}\left\{{\left(x_v-s_i{x}_i\right)}^2+{\left(y_v-s_i{y}_i\right)}^2\right\}\right]d{x}_{i}d{y}_i\\ & =C_{z}FF{T}^{-1}\left[FFT\right\{U_i\left(x_i,y_i\right)\exp \left(j{\phi }_u\right)\}FFT\\ & \times \left\{\exp \left(j{\phi }_f\right)Rect\left(\frac{x_f}{2x_m}\right)Rect\left(\frac{y_f}{2y_m}\right)\right\}].\end{array}$(1) Here, $s_i$ is the scaling parameter which means $d{x}_i/d{x}_v$, $FFT(\cdot )$ and $Rect(\cdot )$ indicate fast Fourier transform calculation and a rectangular-shaped function to reduce aliasing error, respectively. The detailed expressions of $C_z,{\phi }_u,{\phi }_f,x_m,y_m$ can be found in the reference list [24].

The second step applies the single FFT-based Fresnel diffraction to calculate the divergence from the virtual plane to the CGH plane. The complex amplitude of the CGH plane can be obtained by implementing the single Fresnel diffraction from $V{P}_i\left(x_v,y_v\right)$. It can be calculated as follows: (2) $\begin{array}{rl}H{P}_i\left(x_h,y_h\right)& =\int \int V{P}_i\left(x_v,y_v\right)\exp \\ & \times \left[\frac{j\pi }{\lambda d_i}\left\{{\left(x_h-x_v\right)}^2+{\left(y_h-y_v\right)}^2\right\}\right]d{x}_{v}d{y}_v\\ & =\exp \left\{\frac{j\pi \left({x_h}^2+{y_h}^2\right)}{\lambda d_f}\right\}FFT\\ & \times \left[V{P}_i\left(x_v,y_v\right)\exp \left\{\frac{j\pi \left({x_h}^2+{y_h}^2\right)}{\lambda d_f}\right\}\right].\end{array}$(2) The complex amplitude wavefront information on the CGH plane for each image can be obtained through the above algorithm. From the Fresnel diffraction formula, the sampling pitches of the virtual plane and CGH plane ($d{x}_h$ and $d{x}_v$) have a relationship of $d{x}_v=\lambda d_f/Nd{x}_h$ by Nyquist theorem, where $N$ is the sampling number. Here, the ARSS scaling parameter ‘$s_i$’ might provide a degree of freedom to rescale image projection zone size. Based on this degree of freedom, we define an MDSF scale parameter ‘$a_i$’, which represents the overall ratio of the image projection zone size relative to CGH size. Therefore, the sampling pitch of the image plane can be represented by $d{x}_i=a_{i}d{x}_h$, and the ARSS scaling parameter for each plane can be calculated as $s_i=a_i\frac{d{x_h}^{2}N}{\lambda d_f}$ or in other expression as (3) $a_i=s_i\frac{\lambda d_f}{d{x_h}^{2}N}.$(3) Equation (3) implies that $a_i$ of the image plane can be differently designed for each image plane number i, only with a single virtual plane (because $d_f$ can be fixed if we Tune $s_i$ in Eq. [3]) by appropriately tuning the ARSS scaling parameter of each plane number. The upper limit of $a_i$ is determined by a possible range of $s_i$ that does not produce an aliasing error, which can be found in the previous work [24].

If we want to design wavefronts generated from the objects that fully arrived at the region of the CGH plane without distortion, $a_i$ is recommended to be set as $a_{opt}$, which is an optimized MDSF scale parameter. The value of $a_{opt}$ is determined by triangular relations, expressed as $a_{opt}=d_i/d_f$. The size of an image is $L_i=L_h\frac{d_i}{d_f}$, where the $L_h$ is the size of CGH given as $L_h=Nd{x}_h$. If the parameter $a_i$ is smaller than $a_{opt}$, the wavefronts generated from the objects are recorded in a restricted region of the CGH, while the rest of the region remains unused. As a result, the optical intensity or quality of the reconstructed image can be reduced, but it can still reconstruct the image. On the other hand, if the parameter $a_i$ is larger than $a_{opt}$, an aliasing noise can occur because the wavefront is recorded incompletely [18].

To express the complex amplitude information with a phase-only SLM, double-phase encoding of $H{P}_i\left(x_h,y_h\right)$ is applied by decomposing the complex amplitude into two pure phase components; ${\vartheta }_{i,m}\left(x_h,y_h\right)$ and ${\vartheta }_{i,n}\left(x_h,y_h\right)$. If we define the amplitude and phase of $H{P}_i\left(x_h,y_h\right)$ as $|H{P}_i|$ and $\mathrm{\angle }H{P}_i$, (4) $\begin{array}{rl}{\vartheta }_{i,m}\left(x_h,y_h\right)& =\mathrm{\angle }H{P}_i+co{s}^{-1}\left[|H{P}_i|/{|H{P}_i|}_{max}\right]\end{array}$(4) (5) $\begin{array}{rl}{\vartheta }_{i,n}\left(x_h,y_h\right)& =\mathrm{\angle }H{P}_i-co{s}^{-1}\left[|H{P}_i|/{|H{P}_i|}_{max}\right]\end{array}$(5) where $|H{P}_i{|}_{max}$ is the maximum value of $|H{P}_i|$. Therefore, we can see $H{P}_i\left(x_h,y_h\right)=\frac{A_{max}}{2}\cdot \{\exp \left[j{\vartheta }_{i,m}\left(x_h,y_h\right)\right]+\exp \left[j{\vartheta }_{i,n}\left(x_h,y_h\right)\right]\}$, which means $H{P}_i\left(x_h,y_h\right)$ is the sum of the two waves $\exp \left[j{\vartheta }_{i,m}\left(x_h,y_h\right)\right]$ and $\exp \left[j{\vartheta }_{i,n}\right(x_h,y_h\left)\right]$. Using phase-only SLM, a superposition of two-phase information by spatial-multiplexing can lead to expressing complex amplitude information [28]. As a result, double-phase hologram $D{H}_i\left(x_h,y_h\right)$ can be expressed as (6) $\begin{array}{rl}D{H}_i\left(x_h,y_h\right)& =\exp \left[j\right\{{\vartheta }_{i,m}\left(x_h,y_h\right)M\left(x_h,y_h\right)\\ & +{\vartheta }_{i,n}\left(x_h,y_h\right)N\left(x_h,y_h\right)\left\}\right].\end{array}$(6) where $M\left(x_h,y_h\right)$ and $N\left(x_h,y_h\right)$ are the two-dimensional complementary binary checkerboard pattern.

Finally, an off-axis phase encoding can be applied on $D{H}_i\left(x_h,y_h\right)$ to place the image in the intended location within a Fresnel diffraction region of SLM and to split the image information from non-diffractive dc noise. For multiple images generation at different projection regions, phase information with different oblique incidence $O{B}_i\left(x_h,y_h\right)$ is multiplied by each double-phase encoded pattern, $D{H}_i\left(x_h,y_h\right)$. Here, the relation between the position of virtual focus and the off-axis encoding pattern is as follows: (7) $O{B}_i\left(x_h,y_h\right)=\exp \left[j2\pi \left\{x_h\text{sin}{\theta }_i\text{cos}{\varphi }_i+y_h\text{sin}{\theta }_i\text{sin}{\varphi }_i\right\}/\lambda \right]$(7) where the angles ${\theta }_i=ta{n}^{-1}\left[\left({X_i}^2+{Y_i}^2\right)/d_f\right]$ and ${\phi }_i=ta{n}^{-1}\left(Y_i/X_i\right)$ are the inclination and azimuthal angles, and the coordinates $(X_i,Y_i)$ indicate the location of virtual focus for i^th image plane. The final expression for the CGH can be expressed by (8) $CGH\left(x_h,y_h\right)=\sum _{i=1}^{n}D{H}_i\left(x_h,y_h\right)\cdot O{B}_i\left(x_h,y_h\right).$(8)

3. Results and discussions

3.1 Projection zone expansion using the proposed MDSF method

Throughout the paper, the wavelengths used for red, green, and blue (RGB) light sources are set to 638, 520, and 450 nm, respectively, in simulation and experiment. Figure 2(a) and (b) are the reconstructed image results show the major differences between the DSF and the MDSF methods. At first, to focus on the projection zone expansion characteristics of the proposed method, the distance from the virtual plane to the image plane ($d_i$) is fixed to 1.6 m for all images, and $d_f$ is set to 0.4 m. Considering the specification of the SLM that we used in experiments, the sampling pitch of the CGH is designed to $d{x}_h$ = 3.6 μm. The dotted-lined red rectangle represents the region where the experiment might have had non-diffractive reflected light from the SLM. Since the maximum diffraction angle from the given CGH is ${\theta }_{\text{m}}=\lambda /2d{x}_h$, the length of the full projection region with Fresnel approximation will be given as $L_{\text{m}}=\lambda d_{\text{z}}/d{x}_h$. The scale of Figure 2(a) and (b) are given as $L_{\text{m}}\times L_{\text{m}}$, so that these figures intuitively show how a large projection region can be used concerning the full Fresnel diffraction region of the given SLM.

Figure 2 Reconstructed amplitude of letter ‘IPOD’ by using (a) the conventional DSF method and that using (b) the proposed MDSF method that have four foci at the virtual plane. (c) Amplitude of light at the virtual plane and (d) the calculated double-phase encoded CGH that is used for reconstructing the result shown in (b). The overall size of (a) and (b) is set to full Fresnel diffraction region.

The original image of Figure 2(a) comprises a single image, ‘I P O D,’ with RGB and white colors, projected through a single focus positioned at $\left(X_0,Y_0\right)=$ (0, 7.7 mm). Figure 2(b) is composed of four images, ‘I’, ‘P’, ‘O’, and ‘D,’ which are projected through the different coordinate values of focus points: $\left(X_1,Y_1\right)=$ (−7.7, 7.7 mm), $\left(X_2,Y_2\right)=$ (7.7, 7.7 mm), $\left(X_3,Y_3\right)=$ (−7.7 mm, −7.7 mm), and $\left(X_4,Y_4\right)=$ (7.7 mm, −7.7 mm), respectively. The MDSF scale parameter of every image is set as $a$ = 3, sampled with 4096 $\times$ 4096 pixels with pixel pitch $d{x}_i$ = 10.8 μm. Figure 2(c) illustrates the light at the virtual plane which clearly shows that the proposed MDSF method is applied. The initially multiplied spherical phase and $O{B}_i\left(x_h,y_h\right)$ make the multi-focused light on the virtual plane at the intended location. The white circles of Figure 2(c) indicate the filter function multiplied by the result of the virtual plane. Figure 3(d) illustrates the relevant double-phase encoded CGH design for reconstructing Figure 3(b), calculated by the initial resolution of 4096 × 4096 and cropped into 3840 × 2160, considering the practical resolution of the SLM device that will be used in experiments.

Figure 3 (a) Experimental setup for optical reconstruction. Simulation results at a virtual plane and relevant experimental results at an image plane with various iris arrangements of the rectangle (b) and (c); horizontal line (d) and (e); and zigzag (f) and (g).

From the above simulation results, we verified that by applying the multi-focus filtering plane at the virtual plane, multiple images can be reconstructed by the double-phase encoded CGH recorded with the MDSF method, with each image imposed with a different oblique incidence angle. By comparing Figure 2(a) and (b), while maintaining the high-quality reconstruction achieved by the DSF algorithm, we can use a much wider area within the Fresnel diffraction region of the SLM in the same image information expression than the conventional DSF method. Furthermore, when different depths are given to each image, more abundant application is possible. As proof of this concept, both numerical and experimental results for such multi-depth images using theh MDSF method will be shown in the following section.

3.2 The iris layout optimization for avoiding conjugate field and high-order diffraction

Since the results in the previous section are based on quantitative simulation, double-phase encoding is ideally conducted so conjugate fields do not appear and higher-order diffractions are very weakly observed. However, in practical experiments, these conjugated images and higher-order diffracted fields are much stronger than the numerical simulation; therefore, these unwanted light portions can pass through the iris and generate ghost images at the projection plane. In the case of the single-focus filter, these noises are almost negligible, but for the proposed MDSF method, these noises cannot be ignored as the number of foci increases. Therefore, the iris layout optimization is needed to practically use the proposed method, as shown in Figure 3.

First, the experimental setup that we used is shown in Figure 3(a). The red line indicates a light path in the reconstruction process. The phase-only CGH loaded on the SLM is illuminated by laser diode sources of RGB (638, 520, and 450 nm) after beam collimation. The built-in type hologram generation kit (Holographic kit PEONY_62a, May) contains LCoS type SLM with 3840 × 2160 resolution and pixel size 3.6 μm × 3.6 μm is used, which can be operated with the time-multiplexed feature of RGB information with 360 Hz (120 Hz at each color) synchronized with laser diode sources. A multi-focus filter is placed at the designed distance of virtual plane $d_f$. An experimental image is observed by placing the diffuser screen at the designed reconstruction distance $d_z$ from the SLM and the scattered lights are captured by CMOS camera (Canon EOS 6D) placed right above the SLM.

In Figure 3(b)–(g), our strategy for designing the iris location for reducing the noise signal is shown step by step. If the rectangular layout, such as in Figure 3(b), is applied, the projection zone can be used as wide as possible. However, the blurred conjugate image of each object will be located at the odd-symmetry position. For example, the conjugate image of the image focused on the upper-left side may pass through the lower-right side iris and vice versa, which can make unwanted noise, as shown in Figure 3(c).

To avoid such odd-symmetric noises, the iris configuration of off-axis focal arrangement along the same side, as shown in Figure 3(d), can be applied to avoid image distortion caused by the conjugate field. However, if the apertures of virtual plane are positioned in-line, the convergent light of each image on the virtual Fourier plane generally affects teach other since higher-order diffraction field diffracts the horizontal and vertical sides due to the square-type pixel arrangement of the SLM as shown in Figure 3(e). Therefore, it would be advantageous to use zigzag-type arrangements, as shown in Figure 3(f), which cause fewer interactions between each image at the virtual plane, and finally, provide a reconstructed field without an unwanted image, as shown in Figure 3(g). Here, we would like to note that there can be various arrangement types, and zigzag-type arrangement is only one of these examples. In general, we recommend a structure in which adjacent filtering foci are not located on the same line along the horizontal and vertical directions as much as possible.

Figure 4 shows the numerical and experimental results for reconstructed holographic projection images using the MDSF method designed for multi-depth images. Figure 4(a)–(d) present the simulation results and 4(e) – 4(h) demonstrate the optical experiments results. The projected beam from the SLM transmits through a multi-focus spatial filter that consists of a zigzag-type iris arrangement whose diameters are 3.0 mm and foci locations of $\left(X_1,Y_1\right)=$ (−16.5, 11 mm), $\left(X_2,Y_2\right)=$ (−5.5, 7 mm), $\left(X_3,Y_3\right)=$ (5.5, 11 mm), and $\left(X_4,Y_4\right)=$ (16.5, 7 mm), respectively. Four original images of Mars, Earth, Venus, and Mercury are used with the different depth parameters, $d_1$ = 0.25 m, $d_2$ = 0.50 m, $d_3$ = 0.75 m and $d_4$ = 1 m, respectively. The distance from the SLM to the virtual plane is given as $d_f$ = 0.50 m, and the MDSF scale parameter of each image is $a_1$ = 0.5, $a_2$ = 1, $a_3$ = 1.5 and $a_4$ = 2. Therefore, $L_i$ of each projected images are $L_1=$6.35 mm, $L_2=$12.7 mm, $L_3=$19.05 mm, and $L_4=$25.4 mm (image size of the first image ‘Mars’ is 6.35 mm). In Figure 4, the reconstruction distance is $d_{z,i}=d_f+d_i\left(i=1,2,3,4\right)$, so the images Mars, Earth, Venus, and Mercury are focused respectively in order of figures from left to right.

Figure 4 Simulation ((a)–(d)) and experimental ((e)–(h)) results for reconstructed hologram using zigzag-type layout with multi-depth image conditions. Focused on Mars, $d_1$ = 0.25 m ((a) and (e)); Earth, $d_2$ = 0.5 m ((b) and (f)); Venus, $d_3$ = 0.75 m ((c) and (g)); and Mercury, $d_4$ = 1 m ((d) and (h)), which are indicated by yellow markers, respectively.

It can be observed that multi-depth full-color images through the MDSF method are well-reconstructed with less speckle noise. A small spot-like signal around the ‘Earth’ image, shown in the optical reconstruction results, originates from a leakage of non-diffractive light that passed through a nearby iris $\left(X_2,Y_2\right)$ on the virtual plane. To avoid such noise, it is helpful to set a sufficiently far distance between the filtering focus and non-diffractive light on the virtual plane, as shown in the case of the Venus image.

3.3 Image quality assessment with various metrics

As stated in the introduction, compared to the single-step Fresnel diffraction method with random phase (SSF), the proposed MDSF method may provide better image quality since it uses complex amplitude information to reconstruct. On the other hand, the degradation of image quality between conventional DSF and MDSF is quite small, even though we can use a much larger projection region in the MDSF method. To verify this, Figure 5 shows the reconstruction images for image quality assessment of the proposed method to previous approaches, which are numerically calculated with monochromatic light of 520 nm wavelength. Here, we compared the reconstruction images by using SSF (Figure 5(a)), conventional DSF method (Figure 5(b)), and the proposed MDSF method with three (Figure 5(c)) and five (Figure 5(d)) foci, respectively. Similar to previous sections, coordinate values of the multiple foci for the case of Figure 5(d) are given as $\left(X_1,Y_1\right)=$ (−13.8, 7 mm), $\left(X_2,Y_2\right)=$ (−6.9, 11 mm), $\left(X_3,Y_3\right)=$ (0 mm, 7 mm), $\left(X_4,Y_4\right)=$ (6.9, 11 mm), and $\left(X_5,Y_5\right)=$ (13.8, 7 mm). Among them, a middle focus and three middle foci are only used for the cases of Figure 5(b) and (c), respectively. For all cases, the distance parameters are set to $d_f$ = 0.25 m, $d_i$ = 0.75 m, and the reconstructed image length with $L_i=$22.1 mm, using the image ‘Baboons’ composed with the resolution of 2048 × 2048. From the red box region of the whole image, magnified images of the blue box are shown for visual comparison.

Figure 5 Comparison of the image quality using simulation results. (a) Single-step Fresnel diffraction method with random phase. (b) The conventional DSF method. (c) The MDSF method with three foci. (d) The MDSF method with five foci.

Before the comparison using quantitative data, we can also directly observe the differences k image qualities. Due to the random phase multiplication in the Fresnel diffraction method, Figure 5(a) shows significant speckle noise and unclear boundaries in the reconstructed image compared to other figures. In addition, if we compare Figure 5(b)–(d), although the small amount of degradation in reconstructed image quality exists as the number of foci increases, it still maintained sufficiently high image uniformity and low speckle noise compared to the SSF case.

To assess the quality of the reconstructed images quantitatively, we adopt three representative measurement methods—PSNR, MSSIM, and the speckle contrast [29–31]. The PSNR, which is based on structural mean squared error (MSE) between two images, is defined as (9) $PSNR\left(x,y\right)=10{\log }_{10}{L}^2-{\log }_{10}\frac{1}{MN}\sum _i^M\sum _j^N(x_{ij}-y_{ij}{)}^2,$(9) where $M$ and $N$ are the number of pixels vertically and horizontally for the sample image, and $L$ is the dynamic range of allowable image pixel intensities, which is $L=2^8-1=255$ in 8-bits/pixel gray-level image.

The structural similarity index measure (SSIM) can be the method of image quality assessment with better reliability, which is used for measuring similarity between the original and reconstructed images based on three parameters: luminance ($l$), contrast ($c$), and structure ($s$). The formula of SSIM is (10) $SSIM\left(x,y\right)=l\left(x,y\right)c\left(x,y\right)s\left(x,y\right)$(10) Here, the detailed expression of each component can be found in reference [29]. The MSSIM index to evaluate the overall image quality is represented as follows: (11) $MSSIM\left(x,y\right)=\frac{1}{MN}\sum _i^M\sum _j^{N}SSI{M}_{ij}$(11) Finally, the speckle contrast ($C$) is the quantitative image quality method that does not need the reference image, often used to measure the spatial noise for laser-projected images. The speckle contrast is represented as (12) $C\left(x,y\right)=\frac{{\sigma }_{x,y}}{{\mu }_{x,y}}$(12) where $\sigma$ is the standard deviation of the pixel intensity, and $\mu$ is the mean value of the pixel intensity [31].

According to the above equations, the higher value of PSNR and MSSIM indicates a better image quality, whereas the lower value of $C$ parameter suggests the better case. The assessment results of Figure 5, including other sample images (‘Baboon,’ ‘Lena,’ ‘Cameraman,’ ‘Earth,’ and ‘Parrots’), using three metrics are summarized in Table 1. Figure 6 outlines the normalized data after averaging all the cases of sample images.

Figure 6 Normalized average image quality assessment results of 5 sample images.

Table 1 Image quality assessment results of each sample image.

		PSNR				MSSIM				C
Sample Images		SSF	DSF	MDSF (3 foci)	MDSF (5 foci)	SSF	DSF	MDSF (3 foci)	MDSF (5 foci)	SSF	DSF	MDSF (3 foci)	MDSF (5 foci)
	Baboon	14.319	25.4175	20.8508	20.468	0.1197	0.5895	0.5656	0.5454	0.5734	0.4577	0.4582	0.4664
	Lenna	13.3413	19.9713	17.6943	17.1995	0.0553	0.6472	0.6333	0.6283	0.4138	0.2212	0.2218	0.2625
	Cameraman	15.107	22.0416	21.9108	21.4696	0.1755	0.725	0.7141	0.7061	0.7293	0.6544	0.6538	0.6577
	Earth	13.6582	20.2445	21.198	21.1073	0.1094	0.4782	0.4754	0.4725	0.4878	0.4082	0.4087	0.4471
	Parrots	16.5612	23.2973	22.2843	20.0345	0.1477	0.6825	0.6759	0.672	0.5937	0.4842	0.4846	0.4838

Through these data, regardless of the image type, it is shown that the proposed MDSF method has higher performance regarding all quality assessment metrics than the SSF. On the other hand, as the number of foci increases, there is only small degradation of the evaluation values, especially in MSSIM. Therefore, the quantitative assessment results match well with the observation results. We expect that this small image degradation occurs due to the crosstalk of a higher-order diffraction field into the misallocated iris. However, it is verified that complex amplitude encoding with a multi-focus filtering plane is more effective on speckle noise suppression than the random phase distribution in aspects of numerical results.

4. Conclusion

In this paper, we proposed the MDSF method for enlarging the projection region with a multi-focus filtering plane, which can display the different depths of images at different locations. In addition, we studied the practical iris layout based on the diffraction principle by avoiding the conjugate field and crosstalk of higher-order diffraction field on the virtual plane. Experiment results of four different images located at multi-depth condition were demonstrated. We also verified the image quality using numerical results compared with SSF with random phase and conventional DSF method. As a result, we found that increasing the number of foci has only a small degradation effect in image quality, while it can use much larger projection region. We hope this study can be useful for the various projection type holographic display or AR-HUD in automobiles.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Ministry of Science and ICT, South Korea: [Grant Number (IITP)2019-0-00001, (NRF)2022R1F1A1062278]; Ministry of Trade, Industry and Energy, South Korea: [Grant Number P20010672].

Notes on contributors

Young-Jun Ann

Young-Jun Ann received his B.S. degree from the School of Electronics Engineering, Kyungpook National University, Daegu, Republic of Korea, in 2022. He is currently taking his M.S. degree at Kyungpook National University. His research focuses on three-dimensional (3D) displays, AR/VR systems, and holographic display applications.

Seung-Yeol Lee

Seung-Yeol Lee received Ph.D. in 2014 from the School of Electrical Engineering, Seoul National University, Republic of Korea. In 2016, he was a researcher at Electronics and Telecommunications Research Institute (ETRI). He is currently an associate professor at the School of Electronic and Electrical Engineering, Kyungpook National University, Daegu, Republic of Korea.