Optics of AR/VR using liquid crystals

Abstract

Head-mounted displays (HMDs) for augmented reality (AR) and virtual reality (VR) are expected to have a wide field of view, high resolution, no artifacts, vergence accommodation conflict (VAC) management, and compactness. To meet these requirements, liquid crystal technologies have attracted considerable attention as displays and optical elements. In this paper, liquid crystal technologies for AR/VR HMDs are reviewed. Short-range lurch control in-plane switching (SLC-IPS) LCDs with fast response properties decrease the motion blur artifact. A liquid crystal lens with varifocal properties can solve the VAC problem. Pancharatnam–Berry phase optical elements contribute to high resolution, the management of VAC, and compactness.

Keywords:

• AR
• liquid crystal lens
• Pancharatnam–Berry phase optical elements
• SLC-IPS
• VR

1. Introduction

Head-mounted displays (HMDs) or near-eye displays for augmented reality (AR) and virtual reality (VR) play an important role in the metaverse. In AR, digital content and information are superimposed on a real-world scene. HMDs for AR should achieve a see-through representation. In VR, digital images cover the entire field of view (FOV), and 360-degree viewing of an image is made possible by head tracking. HMDs for VR should replicate the real world.

In VR, a display should show a motion picture while the head is moving, even if a still image is being viewed. Therefore, motion picture blur artifacts are important. HMDs can generate a 3D representation by displaying parallax images for each eye. However, such a stereoscopic display has a vergence-accommodation conflict (VAC) problem. VAC has been reported to cause visual discomfort and fatigue [1,2].

HMDs for AR require very small displays. Therefore, liquid crystal on silicon (LCOS) and micro-OLED (organic light-emitting diode) are generally used. In HMDs for VR, in-plane switching liquid crystal displays (IPS-LCDs) [3] or OLED displays with high image quality are used. In terms of pixel densities, LCDs are superior to OLEDs because the red, green, and blue (RGB) sub-pixels in LCDs are divided by precisely patterned color filters using photolithography, whereas they are generally fabricated by masked vapor deposition, which is less precise, in OLEDs. Recently, liquid crystal technologies for AR/VR have attracted considerable attention not only as displays but also as optical elements.

In this paper, several challenges in AR/VR HMDs are described. Then, the liquid crystal technologies that solve these challenges, i.e., fast response LCDs, liquid crystal lenses (LC lenses), and Pancharatnam–Berry phase optical elements (PBOEs), are reviewed.

2. Challenges in AR/VR HMDs

AR/VR HMDs are expected to have a wide FOV and a high resolution to replicate the real world. They should also have no artifacts, manage VAC for comfortable use, and be compact for all day use. For AR display, the see-through function is also required. In the following, motion blur artifacts, resolutions, and VACs are described in detail.

2.1. Motion blur artifact

In VR displays, motion picture blur artifacts are more significant than those in flat panel displays because a display shows motion pictures while the head is moving, even if a still image is being viewed. The impact of motion blur on VR has been discussed [4,5]. In hold-type displays, such as conventional LCDs and OLED displays, motion blur is a problem.

Consider that a person watches an image that is moving with a constant velocity on a display. In the case of a hold-type display, the displayed image is periodically shifted once in a frame period and is sustained during a frame period. However, the eyes continuously follow the image with a constant velocity. This mismatch causes motion picture blur [6].

On the other hand, an impulse-type display, such as CRT (cathode ray tube) and film, does not have this problem because the shifted image is displayed for a moment once in a frame. This is equivalent to a quick flashing image that is continuously moving at a constant velocity and is seen without blur by the eyes when they are following at the same velocity.

To improve motion picture quality, an impulse-type LCD has been proposed [6]. It is commonly used as a VR display. The backlight emits for a moment once in a frame, so the LCD functions as an impulse-type display. In impulse-type LCDs, the LC should respond within a time span that is much shorter than the frame period. Therefore, it requires a very-fast-response LCD mode.

For example, a typical timing chart of an LCD for VR with a 90 Hz frame frequency and 1700 × 1700 pixels (1700 lines) is shown in Fig. 1 [4]. The horizontal and vertical axes denote the time and panel position, respectively. The frame period is 11.1 (=1/90) ms. The voltage that corresponds to the display data is sequentially applied to the pixels line by line. It takes 5.5 ms to address 1700 lines. After the LC responds, the backlight emits for 1 ms. Therefore, the LC should respond within 4.6 ms, as shown in Fig. 1. In the case of conventional LCDs, it is sufficient to respond within one frame period, 11.1 ms in this case. However, HMDs for VR require that the LC responds approximately twice as fast as in conventional LCDs.

Figure 1. Typical timing chart of the LCD for VR with 1700 lines driven at 90 Hz.

2.2. Resolution

In VR, a high-resolution display is required to replicate the real world. Figure 2 shows angular pixel density as a function of FOV, as given by [5] (1) $$D\mathrm{ }\left[\mathit{\text{ppd}}\right]=N\mathrm{ }\left[\mathit{\text{pixel}}\right]/\mathrm{\Phi }\left[\mathrm{\text{deg}}\right]$$(1) where $D$ is angular pixel density in pixels per degree (ppd), $N$ is resolution, and $\mathrm{\Phi }$ is FOV. The acuity of a normal person with 20/20 vision is 1 arcmin, namely 60 ppd. In the case of an FOV of 100 degrees, 6000 (= 60 ppd $\times$ 100 degree) pixels are necessary.

Figure 2. FOV vs. ppd.

Typically, the width of an LCD is approximately 2 inches. Therefore, a high pixel density of 3000 pixels per inch (ppi) is needed. Achieving a high pixel density of 3000 ppi is challenging, although a VR HMD with an LCD of over 1000 ppi has recently been commercialized.

2.3. VAC

The VAC should be managed for a comfortable AR/VR experience. Figure 3 explains the conflict in visual representation between accommodation and convergence [7]. Different images with parallax are displayed on the virtual image plane according to HMD optics. A 3D image is perceived at the convergence plane due to the parallax. The accommodation distance is defined as the distance between the virtual image plane and the eyes, whereas the convergence distance is defined as the distance between the convergence plane and the eyes. In a real scene, both the accommodation distance and the convergence distance are matched. However, in conventional HMDs, these distances are not matched. VAC has been reported to cause visual discomfort and fatigue [1,2].

Figure 3. Conflict in visual representation between accommodation and convergence.

3. Fast-response LCD mode

To reduce motion blur artifacts, a fast-response LCD mode is needed. In the HMDs for VR, IPS-LCDs with high image quality are used. According to the one-dimensional equation derived from the continuum theory [8–11], the response properties of IPS-LCDs are described by the following equation [12]: (2) $${\tau }_{d-\mathit{\text{IPS}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{d^2}{K_{22}} {\tau }_{r-\mathit{\text{IPS}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{d^2}{K_{22}}\left(\frac{1}{E^2/{E_{C-\mathit{\text{IPS}}}}^2-1}\right),$$(2) where ${\tau }_{d-\mathit{\text{IPS}}}$ and ${\tau }_{r-\mathit{\text{IPS}}}$ are the time constants of decay and rise, respectively, with rotational viscosity ${\gamma }_1,$ twist elastic constant $K_{22},$ and thickness of the LC layer $d.$ $E_{C-\mathit{\text{IPS}}}$ is the threshold electric field given by (3) $$E_{C-\mathit{\text{IPS}}}=\frac{\pi }{d}\sqrt{\frac{K_{22}}{{\epsilon }_0\mathrm{\Delta }\epsilon }}$$(3) where $\mathrm{\Delta }\epsilon$ is the dielectric anisotropy of the LC and ${\epsilon }_0$ is the dielectric constant of free space.

The response time decreases with a decrease in LC cell thickness. However, the retardation decreases with a decrease in the thickness and the transmittance of the LCD decreases. Furthermore, it is difficult to fabricate LCDs with a thin LC layer in mass production.

To improve the response properties of IPS-LCDs, several technologies have been proposed [13–15]. Matsushima et al. proposed short-range lurch control in-plane switching (SLC-IPS) LCDs using the two-dimensional confinement of LCs [13].

Figure 4 shows cross-sectional and top views of SLC-IPS LCDs and conventional IPS-LCDs. Both SLC and conventional IPS are driven by a lateral fringe electric field at the edges of the upper electrodes. The lateral fringe electric field is induced by the applied voltage between the upper electrodes and the lower electrode.

Figure 4. Cross-sectional and top views of SLC-IPS LCD and conventional IPS-LCD. Red dotted lines denote the imaginary boundary.

In the conventional IPS-LCD, the directors of the LC initially align in a slightly slanted direction from the upper electrode, so that all the directors rotate in the same direction when the voltage is applied, as schematically shown in Fig. 4b.

In contrast, the directors in the SLC-IPS LCDs initially align parallel to the upper electrode, so that they rotate in the opposite direction, and the directors at the center of the upper electrodes and at the center of their gaps do not rotate, as is schematically shown in Fig. 4c. In this way, the regions at the center of the upper electrodes and at the center of their gaps behave as an imaginary fixed boundary during the response, and they accelerate the response.

According to the two-dimensional equation derived from the continuum theory, where $l$ is defined as the distance between the imaginary boundaries or half of the upper electrode pitch, the response properties of SLC-IPS LCDs are described by the following equation [4]: (4) $${\tau }_{d-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{K}{l^2}\right)},{\tau }_{r-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{K}{l^2}\right)}\left(\frac{1}{E^2/{E_{C-\mathit{\text{SLC}}}}^2-1}\right),$$(4) where ${\tau }_{d-\mathit{\text{SLC}}}$ and ${\tau }_{r-\mathit{\text{SLC}}}$ are the time constants of decay and rise, respectively. Here, the splay and bend elastic constants, $K_{11}$ and $K_{33},$ are assumed to be the same $\left(K=K_{11}=K_{33}\right).$ $E_{C-\mathit{\text{SLC}}}$ is the threshold electric field given by (5) $$E_{C-\mathit{\text{SLC}}}=\pi \sqrt{\frac{\frac{K_{22}}{d^2}+\frac{K}{l^2}}{{\epsilon }_0\mathrm{\Delta }\epsilon }}.$$(5)

The response time decreases with a decrease in the LC cell thickness $d$ and in the distance $l.$ Therefore, the response time can be reduced by decreasing the upper electrode pitch while maintaining the LC cell thickness.

Generally, $K_{11}$ and $K_{33}$ are approximately twice the value of $K_{22}.$ Assuming that distance $l$ is equal to the LC cell thickness $d,$ ${\tau }_{d-\mathit{\text{SLC}}}$ in Eq. (4)(4) $${\tau }_{d-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{K}{l^2}\right)},{\tau }_{r-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{K}{l^2}\right)}\left(\frac{1}{E^2/{E_{C-\mathit{\text{SLC}}}}^2-1}\right),$$(4) is expressed as (6) $${\tau }_{d-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{2K_{22}}{d^2}\right)}=\frac{1}{3}{\tau }_{d-\mathit{\text{IPS}}}.$$(6)

SLC-IPS is expected to be three times faster than conventional IPS.

Figure 5 shows the optical response properties measured for SLC-IPS and conventional IPS fabricated using the same LC material under the conditions of a LC cell thickness $d$ of 2.5 µm and a distance $l$ of 2.5 µm. The optical rise time (10% to 90%) of the SLC-IPS is 2.7 ms, whereas that of the conventional IPS is 7.1 ms. The optical decay time (90% to 10%) of SLC-IPS is 1.8 ms, whereas that of the conventional IPS is 5.9 ms. The response time of the SLC-IPS is approximately one-third that of the conventional IPS, as was predicted by Eq. (6)(6) $${\tau }_{d-\mathit{\text{SLC}}}=\frac{{\gamma }_1}{{\pi }^2}\frac{1}{\left(\frac{K_{22}}{d^2}+\frac{2K_{22}}{d^2}\right)}=\frac{1}{3}{\tau }_{d-\mathit{\text{IPS}}}.$$(6) . Figure 6 shows the measured optical decay response time (90% to 10%) of an SLC-IPS as a function of distance $l.$ The results confirm that the response time can be reduced by the distance $l.$

Figure 5. Optical response properties of SLC-IPS and conventional IPS.

Figure 6. Optical decay response time (90% to 10%) of SLC-IPS as a function of distance $l.$

Matsushima et al. developed an SLC-IPS LCD for VR HMDs [4]. The gray-to-gray (GTG) response time of the LCD is shown in Fig. 7. The optical rise time (10% to 90%) and the optical decay time (90% to 10%) of the SLC-IPS LCD are 1.9 and 2.3 ms, respectively. The slowest GTG response time is 4.5 ms, which meets the requirement for VR HMDs described in Section 2.1. The specifications of this LCD are summarized in Table 1. A motion-blur-free LCD with a 90 Hz frame frequency and 1700 lines was developed.

Figure 7. GTG response times of the developed LCD using SLC-IPS.

Table 1. Specifications for the developed LCD using SLC-IPS.

Parameter		Value
Scan line number		1700 lines
Driving conditions	Frame frequency	90 Hz
	Emission time	1 ms
Operating temperature range		−30 to 85 °C
Response time of LC	Tr (0–255)	1.9 ms
	Tf (255–0)	2.3 ms
	Slowest GTG (0–96)	4.5 ms

4. Liquid crystal lens

Several technologies are used to solve the VAC problem, such as the light field display with microlens arrays [16], the volumetric display with a stack of LC scattering shutters [17], the digital holographic display [18], the Maxwellian view with direct projection on the human retina [19], and the varifocal LC lens [20]. The combinations of various optical designs with eye tracking are expected to alleviate the VAC [21]. The varifocal LC lens coupled with a vergence eye-tracking system has the great potential for solving VAC. The eye tracking system can determine the convergence distance and the LC lens is adjusted to accommodate the convergence plane.

LC lenses have been developed since they were invented in 1979 [22]. Figure 8 shows the optics of the LC lens. A wavefront is a spherical surface centered on the focal point when the rays converge at the focal point. Considering a spherical surface having a radius $f,$ the phases on that spherical surface are identical. Therefore, the phase at A on the LC lens is delayed relative to the phase at O by $(2\pi /\lambda )\mathrm{\Delta }{l}_{\mathit{\text{OP}}}$ with the wavelength $\lambda$ and the optical path difference (OPD) $\mathrm{\Delta }{l}_{\mathit{\text{OP}}},$ which is given by (7) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\left(\sqrt{r^2+f^2}-f\right),$$(7)

Figure 8. Optics of the liquid crystal lens.

as shown in Fig. 8. When the focal length is much longer than the radius ($f\gg r$), i.e., for a low-power lens, Eq. (7)(7) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\left(\sqrt{r^2+f^2}-f\right),$$(7) is approximated by Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) , which includes up to the second-order term of the Taylor expansion [23]: (8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) where $D$ is diopter defined by $D=1/f$ in m units.

The phase generated when the ray passes through the LC depends on the director configurations of the LC. Figure 9 shows the director configurations of the LC lens. The director configuration of the LC layer is homogeneously aligned parallel to the x-axis when no voltage is applied. When a voltage is applied, the directors align according to the voltage, as shown in Fig. 9. The effective refractive index at $(r,z)$ with $r=\sqrt{x^2+y^2}$ for the linear polarization along the x-axis is given by [23]: (9) $$n_{\mathit{\text{eff}}}(r,z)=\frac{n_o{n}_e}{\sqrt{{n_o}^2{\mathrm{\text{cos}}}^2\theta (r,z)+{n_e}^2{\mathrm{\text{sin}}}^2\theta (r,z)}},$$(9) where $n_o$ and $n_e$ are the ordinary and extraordinary refractive indices, $\theta$ is the angle between the director and the LC layer plane, which depends on $r$ and $z.$ The optical path length generated at $r$ when the ray passes through the LC layer along the z-axis is expressed as follows: (10) $$l_{\mathit{\text{OP}}}\left(r\right)={\int }_0^d{n}_{\mathit{\text{eff}}}\left(r,z\right)\mathit{\text{dz}},$$(10) where $d$ is the thickness of the LC layer. The OPD is defined by (11) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=l_{\mathit{\text{OP}}}\left(0\right)-l_{\mathit{\text{OP}}}\left(r\right).$$(11)

Figure 9. Director configuration of the LC lens.

If Eq. (11)(11) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=l_{\mathit{\text{OP}}}\left(0\right)-l_{\mathit{\text{OP}}}\left(r\right).$$(11) is equal to Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) , the LC layer functions as a lens with focal length $f$ or the diopter $D.$ As the angle $\theta$ is larger with an applied voltage than without a voltage, and $n_e>n_o,$ the value of the optical path length $l_{\mathit{\text{OP}}}\left(r\right)$ is smaller with an applied voltage than without a voltage.

Figure 10 shows the OPD profiles of the LC layer. When no voltage is applied at the center ($r=0$) and a high voltage is applied at the periphery, the profile shown in Fig. 10a is generated. This profile serves as a converging lens with a focal length of $f.$ Conversely, when a high voltage is applied at the center, profile (b) is generated and the LC layer functions as a diverging lens with a focal length of $-f,$ as derived from Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) . The OPD profile varies continuously as $n_{\mathit{\text{eff}}}$ is continuously adjusted between $n_o$ and $n_e$ by changing the applied voltage. Therefore, the focal length of the LC lens can continuously vary between $-f$ and $f,$ i.e., it achieves a $2D$ continuous diopter swing.

Figure 10. OPD profiles of the LC lens: (a) focal length of $f$ with high applied voltage at the periphery, (b) focal length of $-f$ with high applied voltage at the center.

To satisfy Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) , the OPD profile across the diameter of the lens is controlled by applying an electric field to the concentric ring electrode structure, as shown in Fig. 11 [23]. The LC lens comprises an LC layer between two transparent substrates. A common transparent electrode is fabricated on one substrate. Many discrete ring electrodes with inter-ring resistors are fabricated on the other substrate. The gap between the ring electrodes should be small relative to the thickness of the LC layer to align the LC director even in the gap due to the elasticity.

Figure 11. Concentric ring electrode structure.

To realize the desired OPD profile, n of the discrete ring electrodes are individually connected to external bus lines. For example, ring electrodes “a” and “e” are addressed with external bus lines in Fig. 11. The lens has (n-1) regions of linear voltage drop because of the inter-ring resistors between any two adjacent electrodes. The LC director aligns depending on the applied voltage between the common and ring electrodes. The OPD profile given by Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) can be achieved by designing the structure of the ring electrodes and applying the proper voltages to the bus lines.

The response time of the LC lens that is used for VR is estimated as follows. It has been reported that VAC can be solved completely with a 2 diopters (D) swing lens and solved minimally with a 1 D swing lens, assuming the objective distance between 0.5 m and infinity and an allowable discrepancy of 1/2 D [20]. To obtain an optimal FOV in VR, the LC lens should be large. Typically, the distance between the eyes is approximately 65 mm. Therefore, the radius of the LC lens is limited to less than 32.5 (= 65/2) mm.

Considering an LC lens with a 25 mm radius and 1 D swing ($\pm 0.5$ D), Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) gives an OPD of 156 µm. Considering Eqs. (9)(9) $$n_{\mathit{\text{eff}}}(r,z)=\frac{n_o{n}_e}{\sqrt{{n_o}^2{\mathrm{\text{cos}}}^2\theta (r,z)+{n_e}^2{\mathrm{\text{sin}}}^2\theta (r,z)}},$$(9) and (10)(10) $$l_{\mathit{\text{OP}}}\left(r\right)={\int }_0^d{n}_{\mathit{\text{eff}}}\left(r,z\right)\mathit{\text{dz}},$$(10) , the maximum OPD for an LC layer with $\mathrm{\Delta }n$ ($=n_e-n_o$) and thickness $d$ is equal to $\mathrm{\Delta }\mathit{\text{nd}}.$ Using $\mathrm{\Delta }n$ = 0.27 (which is a relatively high value for LC materials) and an OPD of 156 µm gives a thickness $d$ = 578 µm, which is approximately 100 times thicker than that used in conventional LCDs. (The thickness of conventional LCDs is several microns.) The response time of the LC is proportional to the square of the thickness of the LC layer. Therefore, the response time is 100² times longer than that of conventional LCDs.

Using a stack of lenses is effective for improving the response properties of the LC lens. The diopter of the stack of LC lenses is the sum of the diopters of each LC lens. By stacking LC lenses, the thickness of each LC lens can be reduced. However, using many LC lenses increases the thickness of HMDs.

Fresnel facets are also effective for improving the response properties of the LC lens [20]. The Fresnel facets are similar to the Fresnel lens, which reduces the thickness of an optical lens. Figure 12 explains the principle of the Fresnel LC lens with three facets. The OPD with the Fresnel lens style shown in Fig. 12a has the same lens properties as that shown in Fig. 10a. The optics are explained in Fig. 12b. If the change in OPD at the facet is equal to an integer multiple of the wavelength ($n\lambda$) as shown in Fig. 12, the LC layer functions as the LC lens in Fig. 8. For example, to obtain the OPD profile shown in Fig. 12a, the same voltages are applied to the ring electrodes at $r_1,r_2$ and $r_3.$

Figure 12. Fresnel LC lens.

Bhowmick et al. proposed an LC lens with a 5 cm diameter and an 0.8 D optical swing for VR HMDs using a stack of lenses and Fresnel facets [24]. Two LC lenses were stacked. Each lens was capable of continuous change in the diopter from −0.20 D to +0.20 D, i.e., a 0.40 D diopter swing. An LC material with $\mathrm{\Delta }n$ = 0.27 was used. The thickness was reduced to approximately 12 µm with 28 Fresnel facets on the LC lens, which drastically reduced the switching time of the LC lens.

The image quality of the fabricated LC lens was tested using a negative USAF 1951 test target with fluorescent light illumination on the back. A linear polarizer was placed in front of the LC lens so that the linear polarization was parallel to the homogeneous LC director. Pictures were taken using a camera with the LC lens on the camera lens. An aperture stop having a diameter of 5 mm, which was chosen considering the average diameter of the human pupil, was attached to the LC lens surface. The position of the aperture stop was shifted from the center of the lens to the edge in 5 mm increments. MTF (Modulation Transfer Function) was measured from the pictures taken of the USAF 1951 test target corresponding to a spatial frequency of 10 cyc/deg (equivalent to 20 ppd).

The MTF value was 0.73 when no voltage was applied (lens off). When a voltage was applied, the MTF values were 0.70, 0.43, 0.56, 0.36, and 0.23 with the aperture stop at 0, 5, 10, 15, and 20 mm from the center. It has been reported that human eye MTF drops to 0.40 at 10 cyc/deg [25]. A large-diameter LC lens with good MTF values was developed, although a drop in the MTF value was observed in the outer area of the lens. However, as it is expected that the users will move their head to generally maintain a gaze angle of less than approximately 25 degrees, the degradation in the MTF at larger angles may not be problematic [24].

5. PBOEs

Recently, several LC technologies with PBOEs have been proposed to solve the challenges for AR/VR HMDs. In PBOEs, a phase is induced by the geometric configuration. The Pancharatnam–Berry phase, or geometric phase results from a change in polarization. LCs are used for PBOEs because the geometric phase can be easily obtained using photo-alignment technology [26].

In 1956, Pancharatnam pointed out that if a polarization is converted to a common polarization state along two different paths on a Poincaré Sphere, a relative phase emerges between the two [27]. In 1984, Berry more generally pointed out that during the diabatic cyclic evolution of a quantum system, a phase is added to the wave function, and this phase depends on the geometric properties of the path taken in the parameter space [28]. It turns out that the Pancharatnam phase can be interpreted as the Berry phase.

5.1. Pancharatnam–Berry phase lens and deflector

The Pancharatnam–Berry phase lens (PBL) and Pancharatnam–Berry phase deflector (PBD) are described in this section. The principle of PBOEs can be described by Jones matrices, assuming an input circular polarization as (12) $$J_{\pm }=\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \pm i\end{array}\right),$$(12) where $J_{+}$ and $J_{-}$ correspond to left- and right-handed circular polarization (LCP and RCP), respectively [29]. When the input light passes through a half-wave plate with its slow axis defined by the angle $\varphi ,$ the output light is given by (13) $$\begin{array}{c}{J^{\prime }}_{\pm }=R\left(-\varphi \right)W\left(\pi \right)R\left(\varphi \right)J_{\pm }\\ =\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & -\mathrm{\text{sin}}\hspace{0.17em}\varphi \\ \mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& e^{-i\pi }\end{array}\right)\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & \mathrm{\text{sin}}\hspace{0.17em}\varphi \\ -\mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \pm i\end{array}\right)\\ \begin{array}{c}=\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & -\mathrm{\text{sin}}\hspace{0.17em}\varphi \\ \mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & \mathrm{\text{sin}}\hspace{0.17em}\varphi \\ -\mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \pm i\end{array}\right)\\ =e^{\pm i2\varphi }\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \mp i\end{array}\right)=e^{\pm i2\varphi }{J}_{\mp }\end{array}\end{array}$$(13) where $R\left(\varphi \right)$ and $W\left(\pi \right)$ with phase retardation $\pi$ are the Jones matrices of the rotation and phase retardation, respectively. Equation (13)(13) $$\begin{array}{c}{J^{\prime }}_{\pm }=R\left(-\varphi \right)W\left(\pi \right)R\left(\varphi \right)J_{\pm }\\ =\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & -\mathrm{\text{sin}}\hspace{0.17em}\varphi \\ \mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& e^{-i\pi }\end{array}\right)\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & \mathrm{\text{sin}}\hspace{0.17em}\varphi \\ -\mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \pm i\end{array}\right)\\ \begin{array}{c}=\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & -\mathrm{\text{sin}}\hspace{0.17em}\varphi \\ \mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)\left(\begin{array}{cc}\mathrm{\text{cos}}\hspace{0.17em}\varphi & \mathrm{\text{sin}}\hspace{0.17em}\varphi \\ -\mathrm{\text{sin}}\hspace{0.17em}\varphi & \mathrm{\text{cos}}\hspace{0.17em}\varphi \end{array}\right)\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \pm i\end{array}\right)\\ =e^{\pm i2\varphi }\frac{1}{\sqrt{2}}\left(\begin{array}{c}1\\ \mp i\end{array}\right)=e^{\pm i2\varphi }{J}_{\mp }\end{array}\end{array}$$(13) indicates that when circular polarization passes through the half-wave plate, a phase difference of $\mathrm{\Delta }\mathrm{\Phi }=2\varphi$ occurs according to the angle $\varphi$ that defines the slow axis of the half-wave plate, and the handedness of the circular polarization becomes reversed.

Circular polarization with opposite handedness induces a phase with the same absolute value but the opposite sign. When a circular polarization is converted to a common opposite circular polarization by a half-wave plate with a different slow axis, a relative phase emerges, as Pancharatnam pointed out.

The optics of the PBLs are the same as those of the LC lens shown in Fig. 8. If the phase difference $\mathrm{\Delta }\mathrm{\Phi }\left(r\right)$ satisfies Eq. (14)(14) $$\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\frac{\pi }{\lambda }\frac{r^2}{f}.$$(14) regardless of the distance $r,$ the LC layer functions as a lens with a focal length $f$ [29]. (14) $$\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\frac{\pi }{\lambda }\frac{r^2}{f}.$$(14) where $\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)$ is the OPD given by Eq. (8)(8) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)\cong -\frac{r^2}{2f}=-\frac{r^2}{2}D,$$(8) . Equation (14)(14) $$\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\frac{\pi }{\lambda }\frac{r^2}{f}.$$(14) indicates that the phase on the output plane of the LC lens at $r$ is delayed relative to that at $r=0,$ which is at the center. Substituting the phase difference that occurs in PBL, which is given by $\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=2\varphi (r),$ into Eq. (14)(14) $$\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\frac{\pi }{\lambda }\frac{r^2}{f}.$$(14) gives (15) $$\varphi \left(r\right)\cong -\frac{\pi }{\lambda }\frac{r^2}{2f}.$$(15)

Equation (15)(15) $$\varphi \left(r\right)\cong -\frac{\pi }{\lambda }\frac{r^2}{2f}.$$(15) indicates that the LC aligns in a plane with its director rotating continuously along a radial axis as a quadratic function of $r$ with a second-order coefficient inversely proportional to the wavelength and focal length, as shown in Fig. 13a [29]. A PBL with a focal length of $f$ can be realized by designing LC director configurations that satisfy Eq. (15)(15) $$\varphi \left(r\right)\cong -\frac{\pi }{\lambda }\frac{r^2}{2f}.$$(15) with phase retardation $\pi .$

Figure 13. Pancharatnam–Berry lens (PBL).

In a PBL, the opposite phase profile is induced for RCP and LCP input, as shown in Fig. 13b. Considering Eq. (14)(14) $$\mathrm{\Delta }\mathrm{\Phi }\left(r\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(r\right)=-\frac{\pi }{\lambda }\frac{r^2}{f}.$$(14) , the opposite phase profile means the focal length has the opposite sign. The PBL functions as a converging lens for RCP and as a diverging lens for LCP, as shown in Fig. 13c. Although a PBL has a strong chromatic effect, i.e., the focal length is inversely proportional to the wavelength, as shown in Eq. (15)(15) $$\varphi \left(r\right)\cong -\frac{\pi }{\lambda }\frac{r^2}{2f}.$$(15) , it has been reported that acceptable performance can be achieved for applications where the lens has low power and small aperture [30].

Figure 14 shows the optics of the PBD [5]. When the rays deflect at an angle $\theta ,$ the phase at A on the LC lens is advanced to the phase at O by $(2\pi /\lambda )\mathrm{\Delta }{l}_{\mathit{\text{OP}}}$ with the wavelength $\lambda$ and the OPD $\mathrm{\Delta }{l}_{\mathit{\text{OP}}},$ which is given by (16) $$\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(x\right)=x\mathrm{\text{sin}}\hspace{0.17em}\theta ,$$(16) as shown in Fig. 14. If the phase difference $\mathrm{\Delta }\mathrm{\Phi }\left(r\right)$ satisfies Eq. (17)(17) $$\mathrm{\Delta }\mathrm{\Phi }\left(x\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(x\right)=\frac{2\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(17) whatever the distance $x,$ the LC layer functions as a deflector with a deflection angle of $\theta .$ (17) $$\mathrm{\Delta }\mathrm{\Phi }\left(x\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(x\right)=\frac{2\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(17)

Figure 14. Optics of the Pancharatnam–Berry phase deflector (PBD).

Substituting the phase difference that occurs in PBD given by $\mathrm{\Delta }\mathrm{\Phi }\left(x\right)=2\varphi (x)$ into Eq. (17)(17) $$\mathrm{\Delta }\mathrm{\Phi }\left(x\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(x\right)=\frac{2\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(17) gives (18) $$\varphi \left(x\right)=\frac{\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(18)

Equation (18)(18) $$\varphi \left(x\right)=\frac{\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(18) indicates that the LC aligns in a plane with its director continuously rotating along the x-axis as a linear function of $x,$ as shown in Fig. 15a [5]. A PBD with a deflection angle of $\theta$ can be realized by designing LC director configurations that satisfy Eq. (18)(18) $$\varphi \left(x\right)=\frac{\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(18) with phase retardation $\pi .$ Using the pitch $P$ of the rotation of the director configuration, $\varphi \left(x\right)$ is defined by (19) $$\varphi \left(x\right)=\frac{2\pi }{P}x.$$(19)

Figure 15. Pancharatnam–Berry deflector (PBD).

From Eqs. (18)(18) $$\varphi \left(x\right)=\frac{\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(18) and (19)(19) $$\varphi \left(x\right)=\frac{2\pi }{P}x.$$(19) , the deflection angle $\theta$ is derived as follows: (20) $$\theta ={\mathrm{\text{sin}}}^{-1}\left(\frac{2\lambda }{P}\right).$$(20)

In the PBD, the opposite phase profile is induced for RCP and LCP input in a way similar to the PBL, as shown in Fig. 15b. Considering Eq. (17)(17) $$\mathrm{\Delta }\mathrm{\Phi }\left(x\right)=\frac{2\pi }{\lambda }\mathrm{\Delta }{l}_{\mathit{\text{OP}}}\left(x\right)=\frac{2\pi }{\lambda }x\mathrm{\text{sin}}\hspace{0.17em}\theta .$$(17) , the opposite phase profile means that the deflection occurs in the opposite direction. The PBD functions as a deflector for RCP and as a deflector in the opposite direction for LCP, as shown in Fig. 15c. The PBD has a strong chromatic effect as well as the PBL, i.e., the deflection angle depends on the wavelength as shown in Eq. (20)(20) $$\theta ={\mathrm{\text{sin}}}^{-1}\left(\frac{2\lambda }{P}\right).$$(20) .

Two methods for controlling PBOEs have been proposed: active driving and passive driving [31]. Figure 16a,b shows active driving and passive driving for the PBL, respectively. Although PBL is given as an example, active driving and passive driving are also possible for PBD.

Figure 16. Active driving and passive driving of PBL.

In active driving, a voltage is applied to the LC layer. With no voltage, the LC layer functions as a PBL. On the other hand, with an applied voltage, the LC directors uniformly align in the normal direction to the PBL plane that generates no phase difference. Therefore, the PBL functions as a transparent window. The switching speed of the active driving is fast because the phase retardation of $\pi$ is achieved using a thin LC layer. For example, considering the wavelength of 550 nm and a $\mathrm{\Delta }n$ of 0.1, the condition generating the phase retardation of $\pi ,$ i.e., $\mathrm{\Delta }\mathit{\text{nd}}=\lambda /2,$ gives an LC layer thickness $d$ equal to 2.75 µm, which is thinner than that used in LCDs.

In passive driving, a polarization rotator is added to the PBL. The polarization rotator switches the input polarization to the PBL between RCP and LCP. For example, the polarization rotator consists of a twisted nematic (TN) LC layer sandwiched between quarter-wave plates (QWPs), as shown in Fig. 16b. The input RCP is converted to a linear polarization by the first QWP. Then, it is converted to an orthogonal linear polarization by the TN LC layer when no voltage is applied. The orthogonal linear polarization is converted to LCP by the second QWP. The PBL functions as a diverging lens for LCP.

On the other hand, when sufficient voltage is applied to the TN LC layer, it maintains the input linear polarization. The linear polarization is converted to RCP again by the second QWP. The PBL functions as a converging lens for the RCP. The passive driving for the PBL switches between converging and diverging lenses. The switching speed of the TN LC layer is fast enough that the passive driving also switches quickly.

To vary the focal length, a stack of PBLs is needed because the active and passive driving of PBLs switch between only two states. Stacking N pieces of PBLs enables the focal length to have 2^N switchable states, i.e., a 2^N multifocal length. Figures 17 and 18 show the stacking of the active PBLs and passive PBLs, respectively. Three stacked PBLs are illustrated as examples.

Figure 17. Stack of active PBLs.

Figure 18. Stack of passive PBLs.

A stack of active PBLs consists of three active PBLs and a circular polarizer that consists of a linear polarizer and a QWP, as shown in Fig. 17. PBL-1, −2, and −3 switch their focal length between $f_1/\infty ,$ $f_2/\infty ,$ and $f_3/\infty ,$ respectively. Combining the states of three active PBLs enables eight focal lengths, as shown in Fig. 17.

A stack of passive PBLs consists of three passive PBLs, three polarization rotators (PRs), and a linear polarizer, as shown in Fig. 18. The PRs comprise a TN LC layer sandwiched between QWPs. The input QWP can be omitted in the first polarization rotator. PBL-1, −2, and −3 switch their focal length between $\pm f_1,$ $\pm f_2,$ and $\pm f_3,$ respectively, according to the voltage applied to TN-1, −2, and −3. Combining the states of the three active PBLs gives eight focal lengths, as shown in Fig. 18b.

Using the polarization selectivity, a geometric phase doublet lens, or a doublet PBL has been proposed. It functions as either a lens or a transparent window depending on the input polarization. Figure 19 shows the principle of a doublet PBL [32]. The proposed doublet lens is composed of two PBLs having the same focal length. A right-handed circular polarizer is placed between the two PBLs. The right-handed circular polarizer changes any input polarization to RCP. It consists of a linear polarizer and a QWP, as shown in Fig. 19. Both PBLs function as a converging lens for RCP.

Figure 19. Principle of geometric phase doublet lens, or doublet PBL.

When an RCP input impinges on PBL-1 as shown in Fig. 19a, it is converged ($+f$) and converted to LCP after passing though PBL-1. It is then converted to RCP by the right-handed circular polarizer. This RCP is converged ($+f$) and converted to LCP after passing through PBL-2. Because PBL-1 and PBL-2 have the same focal length, the focal length of the doublet PBL is half of that of each PBL ($+f/2$).

On the other hand, when an LCP input impinges on PB-1 as shown in Fig. 19b, it is diverged ($-f$) and converted to RCP after passing though PB-1. It is then converted to RCP by a right-handed circular polarizer (RCP is regenerated from RCP). This RCP is converged ($+f$) and converted to LCP after passing through PBL-2, so that the power of two PBLs are canceled, i.e., the focal length of the doublet PBL is infinity ($\infty$). Thus, a doublet PBL functions as a lens for RCP and as a window for LCP.

To fabricate PBOEs, photo-alignment technology and interferometry are used. Figures 20 and 21 show the setups of the interferometry for PBD and PBL, respectively [33]. Polarization interference with orthogonally circular polarizations is required, and it results in linear polarization fringes with rotating polarization axes [34,35]. A linearly polarized laser beam is expanded and split into recording and reference beams using a nonpolarizing beam splitter (NPBS). The beams are orthogonally and circularly polarized by QWPs before they recombine and hit the photo-alignment layer. The exposed photo-alignment layer orthogonally aligns the LC director to the exposed linear polarization.

Figure 20. Optical setup of interferometry for PBL.

Figure 21. Optical setup of interferometry for PBD.

In the setup for the PBL, the recording beam passes through a focusing lens, as shown in Fig. 20 [31]. The interference concentric fringes of linear polarization with rotating polarization axes are exposed in the photo-alignment layer. According to the exposed linear polarization fringes, the LCs in the photo-alignment layer align with the rotating directors along the radial axis. Thus, LC director configurations that satisfy Eq. (15)(15) $$\varphi \left(r\right)\cong -\frac{\pi }{\lambda }\frac{r^2}{2f}.$$(15) are generated, and the LC layer functions as a lens.

In the setup for PBD, the recording beam is tilted with respect to the reference beam, as shown in Fig. 21 [33]. The interference linear polarization fringes along one dimension with rotating polarization axes are exposed in the photo-alignment layer. According to the exposed linear polarization fringes, the LCs in the photo-alignment layer aligns with the rotating directors along one dimension. Thus, the LC director configurations that satisfy Eq. (19)(19) $$\varphi \left(x\right)=\frac{2\pi }{P}x.$$(19) are generated, and the LC layer functions as a deflector.

Interferometry is useful for recording arbitrary wavefronts and generating corresponding patterns for PBOEs. However, it is susceptible to environmental disturbances and requires a coherent light source. Single-exposure fabrication where linear polarizations are patterned using a spatial light modulator (SLM) has also been proposed, but the resolution is limited by the pixel density of the SLM [36].

5.2. Reflective polarization volume gratings

Kobashi et al. proposed PBOEs using cholesteric LCs (CLCs) or reflective polarization volume gratings (PVGs) [37]. PVGs can be regarded as polarization-selective holographic gratings. They are based on patterned CLCs. CLCs have a spontaneous twist with a helical pitch $P.$ In a PVG, another periodic structure is induced in the horizontal plane with the in-plane periodicity ${\Lambda }_x,$ as illustrated in Fig. 22 [38].

Figure 22. Director configurations of PVG.

Bragg diffraction can be established when the medium is thick enough to generate adequate periodical refractive index planes. The Bragg pitch ${\Lambda }_B$ is given by (21) $$\frac{1}{{{\mathrm{\Lambda }}_B}^2}=\frac{1}{{{\mathrm{\Lambda }}_x}^2}+\frac{1}{{{\mathrm{\Lambda }}_z}^2}$$(21) where ${\Lambda }_z=P/2$ and $P$ is the helical pitch of CLCs due to the head-tail symmetry of the director. PVG produces Bragg diffraction for a circular polarization having the same handedness as the helix.

The PVG allows for semitransparent devices that function as a reflective deflector for the circular polarization of one handedness and as a transparent window for the circular polarization of the opposite handedness. If the Bragg surface is curved, the PVG functions as a concave mirror, and is also called a CLC lens.

5.3. Proposed technologies that use PBOEs

5.3.1. Compact near-eye display with depth adaption using PVGs and passive PBLs

Lee et al. proposed a compact see-through near-eye display having depth adaption with PVGs and a stack of passive PBLs [33]. Fast depth adaption of an image can be achieved using PBLs. The combined use of PVGs and PBLs makes possible a compact optical design with depth control of the display content to counter the VAC.

Figure 23 shows the optical design of the proposed compact, see-through near-eye display. The light source generates uniform light, which is reflected by a polarizing beam splitter (PBS) toward an LCOS panel. The displayed content is then reflected toward a QWP to convert the linear polarization to circular polarization. The PBL stack consists of multiple passive PBLs with PRs that switch between multiple focusing powers. The light is then passed through a PVG due to the selectivity of the polarization. After the light is reflected by the concave mirror, the handedness of the polarization is reversed and then deflected by the PVG into the waveguide. The second PVG deflects and guides light into the viewing region.

Figure 23. Compact HMD for AR with PBLs and PVGs.

The combined use of the PVGs and PBLs allows for a compactly designed see-through near-eye display with depth adaption. Fast depth adaption of the image was achieved using PBLs with appropriate additive light field rendering. The light field was reconstructed by assigning computationally generated 2D images to corresponding sub-frames with different two depths. The device was set up to generate a switching of 1 D at a 60 Hz frame frequency. The two subframes reproduced a depth adaption of 1 D. It was confirmed that the user could choose to focus on a near or far object with the device.

5.3.2. Doubling the pixel density of near-eye displays using an active PBD

Zhan et al. proposed optics that used fast-switching active PBD to improve the display resolution of VR HMDs [39]. To double the resolution, a display panel was synchronized with the PBD. Figure 24 illustrates the optics that doubled the pixel density. The optics comprise a PBD, a QWP, and a magnifying lens. By applying a voltage to the PBD, the image on the display can be shifted diagonally by half a pixel, as shown in Fig. 24b.

Figure 24. Optics that double pixel density.

The original panel with square pixels is plotted in solid blue lines, and the shifted panel is plotted in dotted red lines. Pixels A through D are labeled for the original display, while A’ through D’ are labeled for the shifted display. The combined fine grid effectively forms a double-resolution display with the pixels labeled from 1 to 9. An enhanced resolution was demonstrated by switching the PBD on and off, synchronizing with computationally generated sub-frame images.

5.3.3. Wide FOV near-eye display for AR using doublet PBL

Lee et al. proposed an optical combiner for AR by implementing a geometric phase doublet lens, also called a doublet PBL [40]. Depending on the polarization state of light, the real-world scene passes directly through the lens, and virtual images are generated as a function of the positive power of the lens.

When a lens is set in front of an eye in a see-through display, it should function as a lens for the light from a digital display and as a transparent window for the real-world scene. The doublet PBL described in Section 5.1 realizes both of these functions, depending on the input circular polarization.

Figure 25 shows a schematic of the proposed near-eye display, which consists of the doublet PBL, a QWP, a PBS, and a display panel. The real-world scene and the digital image are combined using the PBS. After passing through the PBS, both components are aligned to P-polarized and S-polarized light. The QWP in front of the doublet PBL converts linearly polarized light to circularly polarized light. The QWP axis assigns circular polarization states to each component. Then, the real-world scene is set to LCP and the image from the display is set to RCP. Since the configuration of the entire system is optically equivalent to that of a VR system, it is possible to widen the FOV in the AR HMDs.

Figure 25. Optics of the PB doublet lens for AR.

5.3.4. Foveated display system using doublet PBL

Yoo et al. proposed a foveated display having a single display module based on a doublet PBL. A multi-resolution and wide FOV was simultaneously achieved with temporal polarization multiplexing [41].

The human eye contains the majority of its cone photoreceptors at the center of the macula of the retina, which is called the fovea. [42]. The fovea has high resolving power. The discriminability of the eye is lower in the peripheral region of the retina. Accordingly, high-resolution images should be projected to the fovea. A foveated display system can realize both high resolution and a wide FOV by projecting high-resolution images to the fovea and low-resolution images to the peripheral region.

The doublet PBL functions as a lens or a transparent window, depending on the polarization, as previously described, and it can provide two operating modes: fovea mode for a high-resolution image, and peripheral mode for a wide FOV. The proposed system supports a foveated and wide FOV image simultaneously by superimposing both fovea mode and peripheral mode images.

Figure 26 shows the optical configuration of the proposed foveated display for VR. The display consists of a single display module, two polarization controllers, diverging and converging doublet PBLs, and an eyepiece lens. The display module is based on parallel-beam projection with a collimated beam. The first polarization controller converts an input beam to the RCP or LCP state. The second polarization controller converts the polarization to RCP when LCP light impinges and maintains the polarization when RCP light impinges. These controllers can be implemented using a switchable active-type half-wave plate. Two polarization controllers are synchronized to combine two images in dual operating mode in real time. The diverging doublet PBL functions as a window for RCP and as a diverging lens for LCP. The converging doublet PBL functions as a window for LCP and as a converging lens for RCP. The eyepiece lens focuses a beam on the pupil of the eye.

Figure 26. Foveated display system based on doublet PBLs.

In the fovea mode, as shown in Fig. 26a, the first polarization controller changes the beam to RCP. The diverging doublet PBL functions as a window for RCP and maintains the polarization. The second polarization controller converts the polarization to LCP. The converging doublet PBL functions as a window for LCP. Only the eyepiece lens functions as a lens.

In the peripheral mode, as shown in Fig. 26b, the first polarization controller changes the beam to LCP. The diverging doublet PBL functions as a diverging lens for LCP and converts the polarization to RCP. The second polarization controller maintains the polarization. The converging doublet PBL functions as a converging lens for RCP. Both the diverging and converging doublet PBLs and the eyepiece lens function as lenses. If the focal length of the converging doublet PBL is identical to the negative focal length of the diverging doublet PBL, the diverging beam from the diverging doublet PBL is converted to a magnified parallel beam by the converging doublet PBL. The expanded beam in the peripheral mode enables a wider FOV than in the fovea mode, but the angular resolution is compromised.

If the two modes are temporally multiplexed at high speed, foveated and peripheral images can be simultaneously observed by the human eye.

5.3.5. Pancake optics for VR using PVGs

Polarization-based optical folding, or “pancake” optics, reduces the thickness of VR HMDs [43]. In conventional HMDs, a bulky lens is used to generate a magnified virtual image of a display, such as an LCD, as shown in Fig. 27a. The thick lens is set apart from the display to create a large virtual image. Thus, conventional HMDs are thick.

Figure 27. Pancake optics.

Figure 27b shows the concept of basic pancake optics. A gap is created between the concave half-mirror and the reflective polarizer. The concave half-mirror reflects 50% and transmits 50% of the incident light. The reflective polarizer reflects light of the linear polarization while transmitting that of the orthogonal polarization. QWPs are placed on the LCD and inside the gap.

In pancake optics, light travels as follows: The first QWP on the LCD converts parallel linear polarized light from the LCD to LCP. The second QWP converts LCP back to parallel linear polarized light. The reflective polarizer reflects this linear polarized light, which is then converted to LCP by the second QWP. Then, the concave half-mirror reflects half of the LCP light. The polarization of the reflected light is converted to RCP. Thus, the second QWP converts it to orthogonal linear polarization and then the light passes through the reflective polarizer.

In pancake optics, light traverses the length of the gap three times while occupying only the physical space of one gap length. This enables the LCD to be placed closer to the optics than in conventional optics. However, since the light transmits the concave half-mirror twice and half of the light is lost each time, the overall light efficiency is 25%.

Li et al. proposed a CLC lens for pancake optics [44]. If the Bragg surface is curved, the PVG functions as a concave mirror. Figure 28 shows the proposed pancake optics with a CLC lens. Commercial pancake optics have a flat reflective polarizer and an optical lens, as shown in Fig. 28a, although pancake optics with a curved reflective polarizer and no lens are shown in Fig. 27b for simplicity.

Figure 28. Proposed pancake optics with CLC lens.

Compared with the conventional design shown in Fig. 28a, the proposed pancake optics comprises only three components, including a half-mirror, an optical lens, and a reflective CLC lens, which substitutes for the reflective polarizer and the QWP in the conventional pancake lens, as shown in Fig. 28b. Due to the unique chromatic dispersion of diffractive optics, the CLC lens also assists in correcting the chromatic aberration of the system.

The outgoing light from the display panel passing through a right-handed circular polarizer becomes the RCP light. Next, it passes through the half-mirror and is reflected by the broadband CLC lens with the same handedness. Then, the half-mirror reflects the RCP light and converts it to LCP light, which can directly pass through the CLC lens after traveling through the cavity three times. A CLC lens with a diameter of 1 inch and a focal length of 770 mm was fabricated.

To further reduce the thickness of HMDs, Maimone et al. proposed an HMD for VR that combined pancake optics with holographic optics, directional backlighting, and laser illumination [45,46]. Pancake optics reduce the thickness. However, the concave half-mirror still occupies some room. As shown in Fig. 29, thickness due to bulk optics can be reduced by holographic optics. Focusing is performed by holographic optical elements (HOEs) rather than bulk optics. The concave half-mirror is replaced by an HOE. An HOE consists of thin and flat films of negligible thickness and weight. Therefore, thin and lightweight HMDs are ultimately achievable.

Figure 29. Pancake optics combined with holographic optics.

Holographic optics provide an arbitrary deflection of light from a thin and flat surface, but they require wavelength and angle sensitivities. A spectral bandwidth of 0.1 nm to 1 nm is required for optimal performance. This spectrum is much narrower than that of conventional displays, such as LED backlight LCDs or OLED displays. This can be achieved using a laser backlight LCD. The laser backlight LCD can also achieve a wide color gamut, such as the B.T. 2020 color gamut [47].

The application of PBOEs has also been proposed for these ultimately thin and lightweight HMDs [45]. A PVG or a CLC lens was used to add focusing power to the reflective polarizer surface in addition to the HOE. The PVG comprises a reflective polarizer surface and has an aspherical phase profile. The PVG element was designed to correct some optical aberrations of the HOE.

5.3.6. Other technologies for AR/VR HMDs using PBOEs

Many technologies other than those described in this paper have been proposed, such as passive PBLs for chromatic aberration correction [48], Alvarez passive PBLs for a VAC countermeasure [49], and eyebox expansion [50] and gaze-matching [51] for a Maxwellian view.

6. Conclusions

AR/VR HMDs are expected to play an important role in the metaverse. In this review, several challenges were described, such as achieving a wide FOV and high resolution to replicate the real world, freedom from artifacts and the management of VAC for comfortable use, and compactness for all-day use. LC technologies for AR/VR have attracted considerable attention for solving challenges, not only as displays but also as optical elements. Among these technologies, fast response LCDs, called SLC-IPS LCDs, LC lenses, and PBOEs were reviewed in detail. Optics using LCs are quite promising for achieving the HMDs expected for AR/VR.

Disclosure statement

No potential conflict of interest was reported by the author(s).