Open Access
Add to BibSonomy
Abstract
Augmented reality head-mounted displays (AR-HMDs) utilizing diffractive waveguides have emerged as a popular research focus. However, the illuminance uniformity over the fields of view (FOV) is often unsatisfactory in volume holographic grating (VHG) based waveguide displays. This paper proposes a high uniformity AR waveguide display system. Firstly, the angular uniformity of the VHG-based waveguide displays is analyzed. Subsequently, diffractive optical elements (DOEs) are seamlessly integrated onto the outer coupling surface of the waveguide substrate to improve the angular uniformity through phase compensation. To design the DOE phase, the multi-objective stochastic gradient descent (MO-SGD) algorithm is proposed. A single DOE is used to compensating various images form the image source. A hybrid loss, which includes the learned perceptual image patch similarity (LPIPS) metric, is applied to enhance the algorithm performance. Simulation results show that the proposed method effectively suppresses illumination degradation at the edge FOV in exit pupil images of the waveguide display system. In the results, the peak signal-to-noise ratio (PSNR) is improved by 5.54 dB. Optical experiments validate the effectiveness of the proposed method. The measured nonuniformity (NU) against FOVs is improved by 53.05% from 0.3749 to 0.1760.
© 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Augmented reality (AR) technology seamlessly integrates virtual world images with real-world visuals, creating an immersive experience [1]. Head-mounted displays (HMDs) serve as the primary hardware for AR technology and find wide applications in military, education, navigation, entertainment, and other fields [2]. Several optical solutions exist for AR-HMDs, such as geometric optical elements utilizing flat or freeform beam splitters [3], hybrid reflective-refractive systems [4], Maxwellian displays [5], computational holography [6], and optical waveguides [7–9]. Among these options, optical waveguides represent the most promising approach to enhance the portability and wearability of AR-HMDs due to their lightweight, compact design, and glass-like form.
Optical waveguides are generally categorized into two types: geometric waveguides and diffractive waveguides based on their light coupling mechanisms. Geometric waveguides often use refractive or reflective optics for light coupling, employing devices like the partial reflector mirror array (PRMA) [10]. Nevertheless, this method may present some challenges, including heavier stray light and a smaller exit pupil. Moreover, the complex coating and bonding processes may lead to lower processing yields. For instance, Lumus [11] used PRMA as the out-coupler and bonded a multilayer beam-splitter film, which required necessitating high alignment accuracy. On the other hand, diffractive waveguides commonly use periodic grating structures for light coupling. These are classified into surface relief grating (SRG) [12] and volume holographic grating (VHG) waveguides [13] based on different periodic structure types. SRG-based waveguides can be produced using methods like reactive-ion etching [14] or electron-beam lithography [15]. However, their high cost is due to the involved complex process flow and expensive equipment. VHGs are produced through laser interference exposure and are seen as a promising solution for AR-HMD due to their controllable manufacturing processes and lower cost [16].
Illumination uniformity and fields of view (FOV) are critical parameters in both SRG-based and VHG-based waveguide systems. The FOV determines the angular range of the image that the user can observe. Illuminance uniformities include the uniformity over the expanded exit pupil (exit-pupil uniformity) and the uniformity over the FOV (angular uniformity) [17]. Exit pupil uniformity is particularly crucial in waveguide systems with exit pupil expansion. Conventional methods improve exit pupil uniformity by adjusting the diffraction efficiency distribution of the out-coupling grating [8]. However, achieving ideal angular uniformity becomes challenging when the image illumination of an HMD covers a wide FOV. There is a trade-off between the FOV and angular uniformity. Illumination non-uniformity may occur at the FOV edges, significantly affecting display quality.
In recent years, some meaningful researches have been carried out on the angular uniformity of waveguide display systems. Some researchers posit that the principal cause of angular uniformity issues is the varying diffraction efficiencies of the grating concerning light incident at diverse angles. To address this, they utilized particle swarm optimization algorithms and rigorous coupled wave analysis (RCWA) to optimize grating structural parameters [18]. This optimization approach improves the uniformity of diffraction efficiency across different FOVs, achieving an exit pupil uniformity of 0.91 and an angular uniformity of 0.64. Despite significantly improving exit pupil uniformity, this method exhibits limited enhancements in angular uniformity. Alternatively, other researchers attribute angular uniformity issues to inherent natural vignetting in waveguide display systems. The differential evolution algorithms were used to address the impact of natural vignetting [17]. This method achieves a relatively uniform distribution of diffraction efficiency across different FOVs, resulting in 0.89 angular uniformity across the entire 45° horizontal FOV. However, this method is exclusively applicable to optimizing the grating parameters for SRGs. Concurrently, some researchers underscore the role of back-coupling loss in influencing angular uniformity in diffractive waveguides. They reduced the back-coupling loss by introducing a thin film sandwich with a lower or higher refractive index between the in-coupling grating and the waveguide substrate [19]. This method amplifies diffraction efficiency for certain incident angle, leading to a reduction in back-coupling losses and an improvement in uniformity from 0.2483 to 0.3502. However, the addition of a thin-film sandwich imposes higher process demands. In conclusion, the aforementioned improvement methods exclusively address specific factors impacting angular uniformity, and their applicability is confined to SRG-based waveguide systems. Limited approaches have been reported regarding angular uniformity issues in VHG-based waveguides.
This paper introduces, to the best of our knowledge, a pioneering system employing a compensated diffractive optical element (DOE) to achieve heightened angular uniformity in VHG-based waveguide displays. Firstly, we present a comprehensive analysis of the angular uniformity of VHG-based waveguide display systems. On this basis, a DOE for phase compensation to the exit pupil beam is introduced for the first time. Compensating DOE enhances angular uniformity with little increase in system size and weight. The multi-objective stochastic gradient descent (MO-SGD) algorithm based learned perceptual image patch similarity (LPIPS) hybrid loss is proposed to optimize a single DOE used for compensating different image sources. Through simulations, the proposed method effectively mitigates edge FOV illumination degradation in exit pupil images of the waveguide display system. This results in a 5.54 dB improvement in the peak signal-to-noise ratio (PSNR) of the reconstructed images and a 69.94% reduction in nonuniformity (NU). The effectiveness of this method has been validated through optical experiments.
2. Improved waveguide system to enhance angular uniformity
2.1 Classical waveguide system
The classical diffractive waveguide display system is depicted in Fig. 1. The system consists of a micro-display, a collimation module, a waveguide substrate, and two VHGs. These VHGs serve as the in-coupling VHG and the out-coupling VHG, respectively. The light originating from the micro-display is collimated by the collimation module, transforming each point source into an incident plane wave. These plane waves, bearing image information, subsequently impinge upon the in-coupling VHG. Subsequently, the plane wave is diffracted by the in-coupling VHG into the waveguide, where it is confined by total internal reflection (TIR) and guided towards the out-coupling VHG. The out-coupling VHG extends the light in one direction and directs it outward from the substrate towards the observer.
As shown in Fig. 1, waveguide displays are required to offer the observer with a relatively uniform virtual image at infinity, a large field of view, and a large exit pupil. Therefore, the design of waveguide displays should take into account both exit pupil uniformity and angular uniformity. Exit pupil uniformity refers to the consistency among various exit pupil positions within the same FOV. It signifies the energy distribution of light of the same color at different positions, as illustrated in Fig. 1. Angular uniformity, on the other hand, pertains to consistency across different FOVs at the same exit pupil position [19]. It illustrates the energy distribution of light of the different color at the same pupil position in Fig. 1. The primary focus of this paper is to address the issue of angular uniformity. Therefore, the latter part of this paper assumes that the pupil exit uniformity is ideal.
Achieving satisfactory angular uniformity in waveguide displays is a notable challenge due to inherent system characteristics. The illumination of the image typically diminishes as it extends outward from the center-FOV area, resulting in notable darkening towards the edges and corners of the displayed image. This edge FOV attenuation, particularly evident in volume holographic grating (VHG)-based waveguide systems, exhibits asymmetric properties.
As described in Supplement 1, four factors contributing to the degradation of angular uniformity include the angular dependence of the VHGs, the discontinuity of waveguide propagation, the cosine-fourth power law, and the back-coupling loss [17,19–22]. For conciseness, the one-dimensional analysis is used here.
2.2 Waveguide display system with compensated DOE
To suppress the illumination decrease at the edge FOV described in Supplement 1, we propose a novel waveguide display system comprising a micro-display, a collimation module, in-coupling and out-coupling VHGs, and a compensated DOE, detailed in Fig. 2. The compensated DOE is affixed to the top surface of the waveguide substrate at the exit pupil position. The light emitted from the micro-display is transmitted through the waveguide system and then reaches the compensating DOE on the upper surface of the waveguide substrate. The compensated DOE modulates the wavefront phase, rectifying angular uniformity errors in the transmitted light, to ensure uniformity as it exits towards the observer. Notably, the compensated DOE adds little to the size and weight of the system.
The principle of compensated DOE is shown in Fig. 3. An incident angle beam coupled from the waveguide is diffracted as it passes through the DOE, and the maximum diffraction angle θdiff of the DOE is related to the wavelength λ and the sampling interval p,
To provide the desired modulation capability, the maximum diffraction angle θdiff of the DOE needs to be close to or greater than half of the maximum FOV of the waveguide. We perform simulation verification in the later discussion. The intensity of the full edge FOV beams is enhanced by phase modulation, while the intensity of the full center FOV beams is suppressed. This improves the uniformity of beams with different incident angles. It is noted that the compensation effect is only for systematic errors and is not related to the specific source content.
However, the effectiveness of the proposed system in compensating for angular uniformity depends on the generation of the compensated DOE phase. There are two challenges in solving the DOE phase in the waveguide system. One is that the DOE is generated in a multi-plane propagation model. The second is the multi-objective optimization solution for a single DOE.
Firstly, the optical propagation process of the proposed system imaging belongs to the multi-plane propagation model between three planes, as illustrated in Fig. 4. The model includes a exit pupil plane, a lens plane, and a retinal plane. The upper surface of the waveguide substrate serves as the exit pupil plane. A compensated DOE is overlaid on the exit pupil plane. Meanwhile, the lens plane and the retinal plane simulate the imaging system of the human eye. The distance between the exit pupil plane and the lens plane, denoted as z1, also represents the eye relief of the waveguide system. The distance between the lens plane and the retinal plane, identified as f, corresponds to the focal length of the human eye. The DOE modulates the phase of the wavefront at the exit pupil plane. The modulated wavefront propagates to the lens plane and then is focused by the lens. The compensated image is obtained on the retinal plane.
Traditional DOE phase generation algorithms, including the classical Gerchberg-Saxton algorithm (GS) and its enhancements [23,24], are mainly used for solving between two planes. The solving capability is limited in multi-plane models. Recently, stochastic gradient descent (SGD) and related methods have been proposed to optimize the phase-only DOE [25,26]. The DOE phase distribution can be better solved in a multi-plane propagation model using the SGD method.
The second point to consider is the multi-objective optimization problem. In a waveguide display system, the image supplied by the image source is indeterminate. The single DOE be used to compensate for the systematic error common by all image sources during waveguide propagation, such as the non-uniform error. This means that the non-uniform input image and the target image used for the optimization algorithm are not only a pair of images. Two datasets consisting of two sets of one-to-one images is required.
Although SGD method has demonstrated its efficiency in phase recovery, prior studies have predominantly concentrated on single objects rather than multiple objects. The robustness of these methods is susceptible to compromise due to the absence of comprehensive optimization for distinct target images. These methods cannot achieve the optimization goal of compensating DOE in waveguide display systems.
3. MO-SGD algorithm for compensated DOE with high robustness
To improve the robustness of the compensated DOE in waveguide display systems, a MO-SGD algorithm is proposed as shown in Fig. 5. By considering the specific characteristics of each image in the datasets, the DOE calculated by SGD is made effective for each image within the datasets.
The solution process of the proposed algorithm can be divided into three steps. These three steps are represented in Fig. 5 using cyan, black and blue arrows respectively.
3.1 Nonuniform error acquisition
In the first step, we need to obtain the nonuniform error. The propagation of the image from the micro-display through the waveguide system introduces non-uniform errors, which manifest in the image formed within the human eye. By utilizing white field images, transmitted through the waveguide, non-uniform errors can be observed in the images formed at the retinal plane. This process can be achieved by theoretical derivation, ray-tracing software simulation, or actual optical experimental measurements.
To ensure the validity of the single DOE phase being trained on all images of the image source, two image sets are set up. Namely, the target image datasets and the non-uniform image datasets. The target image dataset is the original image displayed on the micro-display from the ILSVRC2012 dataset [27]. The images are preprocessed as gray scale images with a resolution of 2160 × 2160. Using the non-uniform errors, the target image datasets is processed to obtain a non-uniform image datasets whose content corresponds one-to-one with the target image datasets. In the MO-SGD algorithm, these two datasets are used as the input image Xi and the target image Ti, respectively, as shown by the cyan arrows in Fig. 5.
3.2 Multi-plane propagation by the angular spectrum
In the second step, multi-plane propagation is performed using the angular spectrum method (ASM), as shown by the black arrows in Fig. 5. The input non-uniform image Xi is propagated twice in the backward and twice in the forward direction to obtain the reconstructed amplitude distribution E2’ in the retinal plane after DOE modulation. The schematic diagram of the multi-plane propagation model is shown in Fig. 4.
Firstly, the non-uniform amplitude distribution E2 is obtained from the input image Xi. Next, the amplitude distribution E2(x2,y2) on the retinal plane is combined with a random phase to form the complex amplitude distribution U2(x2,y2). Then, the complex amplitude distribution U1(x1,y1) on the lens plane is obtained through inverse diffraction calculation. The ASM is used as a method of diffraction computation, and the expression is given by Eq. (2):
Next, the lens phase distribution in the lens plane is removed. Here, the lens distribution ϕlens is the ideal lens phase distribution and given by Eq. (4):
Then the complex amplitude distribution U0(x0, y0) on the exit pupil plane is obtained from the lens plane by inverse diffraction propagation,
Restricting the amplitude distribution E2 on the retinal plane and the amplitude distribution on the exit pupil plane, the complex amplitude U0(x0, y0) can be found more accurately by some iterative methods.
Subsequently, DOE modulation is performed. The complex amplitude distribution U0(x0,y0) on the exit pupil plane is multiplied by the DOE phase distribution ϕDOE to obtain the new input complex amplitude distribution U0’(x0,y0).
The new forward propagation process is the opposite of the back propagation process. The new complex amplitude U0’(x0,y0) on the exit pupil plane through two steps forward propagation to obtain the reconstruction complex amplitude U2’(x2,y2) on the retinal plane,
The reconstruction amplitude distribution E2’ is derived by taking absolute values of the complex amplitude distribution U2’.
Finally, the undiffracted light, also known as the zero order beam or DC component, is considered. It is inevitable whether the DOE is implemented using microstructured patterns or phase-only spatial light modulators (SLMs). Fortunately, in a waveguide display system, the DC component constitutes signal light rather than an error and can also be imaged on the retinal plane. Consequently, the final reconstruction image formation on the retinal plane includes both the phase modulation component and the DC component:
3.3 Multi-objective SGD iteration
In the third step, a special form of iterative loop paradigm is performed using the MO-SGD algorithm, as shown in Fig. 5. The number of target image datasets and non-uniform image datasets used is M, the number of training cycles of the MO-SGD is K, and the number of iteration steps of the SGD algorithm for each image is L.
Phase optimization for DOE can be formulated as solving an optimization problem of the form:
Solving Eq. (8) using the SGD algorithm by K steps leads to a simplified expression,
The proposed MO-SGD algorithm can be expressed as
First, a pair of images is selected from the target image datasets and the non-uniform image datasets. Then the DOE phase $\phi _{DOE}^{(i )}$ is optimized by L steps using SGD algorithm. Then select the next pair of corresponding images from the two datasets and continue to optimize the DOE phase $\phi _{\textrm{DOE}}^{({\textrm{L} + \textrm{i}} )}$ by L steps. This is repeated until M pairs of images in both datasets are optimized. Then the next loop is carried out and the optimization is iterated again for both datasets until the K epochs is completed. In this way, after K*M*L optimization iterations, the training of the DOE phase $\phi _{DOE}^{({K\ast M\ast L} )}$ is completed.
For the constants related to the number of iterations, the appropriate values need to be carefully chosen. The empirical settings K = 40, M = 1000, and L = 3.
3.4 Loss function
In addition, the DOE phase is generated by the MO-SGD algorithm, and the optimizer converges to different optimal solutions depending on different loss functions. Typically, SGD algorithms use mean square error (MSE) as the loss function [28]. But only using MSE loss function, the reconstructed image may exhibit a meshing effect [29]. More complex loss functions also exist in some deep learning tasks [30–31], but these loss functions do not work well in our system.
Waveguide displays are used as displays for direct imaging of the human eye. To generate reconstructed images by the DOE that are more compatible with the human visual system by the MO-SGD algorithm, we use a combination of MSE, MS-SSIM, and LPIPS as the loss function. LPIPS is closer to human perception and performs better in visual similarity judgments [32]. The hybrid loss function is expressed as:
The smaller the MSE and the larger the MS-SSIM indicators, the better the image quality. The smaller the LPIPS value, the closer the image is to the original. LPIPS is closer to human perception in terms of visual similarity judgments and performs better than traditional methods.
4. Results
In order to quantitatively analyze the quality of the reconstructed results, several evaluation parameters are introduced, including the PSNR, the structure similarity index measure (SSIM), and the LPIPS. In addition, to evaluate the uniformity distribution of full FOV rays imaged in the image plane, we have introduced a uniformity evaluation metric. However, the reconstructed results we evaluated included near-zero amplitude data at the very edges of the FOV, rather than intercepting the relatively uniform central FOV region. Therefore, illuminance uniformity metrics based on minimum and maximum values are not applicable. Instead, a standard deviation based NU metric is used to assess the uniformity of the reconstruction results. The NU is defined as follows:
4.1 Simulation results
To verify the effectiveness of the proposed algorithm, numerical simulations are performed. The wavelength of the light source is 532 nm. The reference distance z0 is set to 40 mm, and the focal length of the lens f is set to 20 mm. The sampling interval of the DOE is 3.6 µm. The training datasets has 1000 pairs of images, including original and non-uniform images. The validation datasets has 100 images including gray images and binary images. The images are preprocessed to a region of 2160 × 2160 pixels. The learning rate is 0.002. The MO-SGD method is implemented using Python 3.6.12 and Pytorch 1.7. The MO-SGD was trained for 40 epochs using Adam optimizer and the loss gradually reached a steady state. All algorithms shown in this paper are implemented on an i9-12900KF @ 3.2 GHz CPU, 128GB RAM, and NVIDIA RTX 3090 GPU with 24 GB. The CUDA version employed is 12.1.
The single DOE generated using the MO-SGD algorithm is numerically reconstructed and the results are shown in Fig. 6. Table 1 displays the evaluation parameters of the test images.
Table 1. The evaluation parameters
Comparing the non-uniform input images shown in Fig. 6, it is obvious that the MO-SGD algorithm is better in suppressing the illumination degradation at the edges of the image. The data in Table 1 show that the reconstruction results of the proposed algorithm are improved by 41.79%, 11.9% and 21.08% in terms of PSNR, SSIM and LPIPS, respectively, compared with the non-uniform input.
Due to the multiplexing of a single DOE, the modulation is only for the degradation of edge illumination and not for specific image details. There is an unavoidable loss of detail in the edges of the reconstruction result compared to the original image. In addition, the overall increase in edge illumination results in a loss of contrast.
4.2 Loss function comparison
In addition, we performed a simulation to highlight the contributions of each loss function to the MO-SGD algorithm. The DOE phase profile calculated by different loss functions are shown in Fig. 7(a). The reconstruction results are shown in Fig. 7(b). The PSNR, SSIM and LPIPS of the test images are shown in Table 2.
Table 2. The PSNR, SSIM, and LPIPS of the test images
The combined module of each loss function was evaluated. Only the MSE loss function is used and the reconstructed image has serious grid effect. MS-SSIM is not sensitive enough to consistency bias and the reconstructed image is dark overall. LPIPS has a better visual perception. However, there are distortions in the reconstructed image at the FOV edge part. Based on the evaluation metrics in Table 2 and the results shown in Fig. 7, our hybrid loss function appears to be the most effective overall. Although it may not perform the best on some specific metrics.
4.3 Uniformity verification in simulation
To quantitatively assess the uniformity of the full FOV, the white field is used as a test image for the numerical simulation, and the reconstruction result of the MO-SGD algorithm is shown in Fig. 8(d). For comparison, total energy in the reconstructed image and the non-uniform input image is controlled based on the energy conservation principle. The NU is shown in red letters in the upper left corner of the corresponding image. Figure 8(b), (e) shows the intensity distributions of the non-uniform input and the results obtained by the MO-SGD algorithm. Figure 8(c) (f) shows the profile curves of the intensity distributions in the horizontal incidence direction (fixed vertical angle of 0°). The reconstruction results show that the NU of the proposed method is improved by 69.94% from 0.4394 to 0.1321.
4.4 Optical experiment
In order to verify the effectiveness of the compensated DOE, the AR waveguide display system is established. The schematic diagram is shown in Fig. 9(a), and the experimental setup is shown in Fig. 9(b). The AR waveguide display system consists of a micro-display, a collimation module, a waveguide substrate, a polarizing beam-splitting prism (PBS) and a phase-only SLM. The micro-display utilizes a 0.32'‘ green Micro-OLED display with a resolution of 800 × 600 and a peak brightness of 40,000nit. The VHG-based waveguide is a one-dimensional EPE diffractive waveguide. The dimensions of the waveguide substrate are 55mm*25mm*2 mm. Two VHGs are used as the in-coupler and out-coupler respectively. The size of the in-coupler is 7mm*12 mm with high diffraction efficiency (∼90%). The out-coupler is 20mm*12 mm. The out-coupled VHG consists of three out-of-pupil extensions, with the diffraction efficiency of the three out-of-pupil regions increasing sequentially. The K vectors of the two VHGs are designed with mirror symmetry. The SLM is a phase-type LCOS with a resolution of 3840 × 2160 and a pixel size of 3.6µm × 3.6µm. The DOE phase distribution with 2160 × 2160 points is loaded on the SLM. The dimensions of the PBS are 15mm*15 mm. The waveguide system has a horizontal FOV of 10°, an eye box of 8 mm at an eye relief of 40 mm. The size of the camera lens is 6 mm, so the motion range of the lens in the eye box is limited.
The light emitted from the micro-display is collimated by a collimation module, coupled into a waveguide, and after transmission through the waveguide, coupled from the PBS into the SLM, and the phase-modulated light beam by the SLM passes through the PBS again and exits the pupil. A camera is used to simulate the human eye to receive the actual observed image. The actual effect of the AR waveguide display system is shown in Fig. 10(a), (d) shows the optical results of full FOV with and without DOE compensation. Figure 10(b), (e), (c), (f) show the left and right edge FOV optical results. Comparing to the outside area within the red lines, it can be seen that the experiments verify the compensation effect of DOE on the non-uniform region at the edge of the exit pupil image.
Fig. 10. Optical reconstructed results, (a) (d) full FOV, (b) (e) left edge FOV, (c) (f) right edge FOV.
The uniformity over the all FOV of the proposed waveguide display system is further verified using a white field as the image source in optical experiments, as depicted in Fig. 11. Figure 11(a), (d) present the white field image without and with DOE compensation, respectively. The white field intensity distributions data with and without DOE compensation in the red dashed box in Fig. 11(a), (d) are calculated respectively. Figure 11(b), (e) show the intensity distribution of the white field without and with DOE compensation, respectively. Figure 11(c), (f) shows the intensity profiles of the white field without and with DOE compensation in the horizontal incidence direction. The NU is shown in red letters in the upper left corner of the corresponding image. The calculated results show that the NU is improved by 53.05% from 0.3749 to 0.1760 after the inclusion of the compensated DOE.
Furthermore, the additional energy loss caused by the proposed method is analyzed. The intensity distribution data in the red box in Fig. 11 were used for the calculation. The peak intensity of our proposed method is reduced by 11.3% compared to without DOE. Due to the diffraction efficiency of the DOE, the total energy of our proposed method is reduced by 7.81%.
5. Discussions
Numerical simulations and optical experiments validate the effectiveness of our proposed method. However, there is still a gap between the experimental results of the proposed method and the simulation. In particular, the edges of the reconstructed images suffer from a more severe loss of detail, loss of contrast, and diffraction speckles compared to the image center. Some of the factors affecting the reconstruction results in the experiment are discussed.
Firstly, the reconstruction effect of the proposed method is affected by the maximum diffraction angle of the DOE, as shown in Fig. 3. The maximum diffraction angle of the DOE is determined by its sampling interval, as seen by Eq. (1). The effect of DOE sampling interval on the reconstruction results is discussed through simulation. The DOE size is set to 3 mm, z = 20 mm, and lens focal length f = 20 mm. The FOV in this case is about 8.6°. Varying the sampling interval, the proposed algorithm is solved and numerically simulated, and the results are shown in Fig. 12(a), (c).
Fig. 12. (a) PSNR and SSIM of reconstruction results for different DOE sampling intervals. (b) PSNR and SSIM of reconstruction results for different wavelength bandwidths. (c) Reconstruction results for different sampling intervals. (d) Reconstruction results for different wavelength bandwidths.
As seen in Fig. 12(a), the PSNR and SSIM of the reconstructed results are drastically reduced after the DOE sampling interval is larger than 4um. As can be seen from Fig. 12(c), the compensation effect of the edge region is gradually reduced as the DOE sampling interval increases. It is easily obtained from Eq. (1) that the maximum diffraction angle is about 3.8° at a sampling interval of 4 um. So it is necessary to make the maximum diffraction angle larger than half of the FOV by selecting the DOE sampling interval to ensure the DOE modulation effect. However, in the experiment, our SLM pixel pitch is 3.6um and the maximum diffraction angle is about 4.3°. It is close to but does not satisfy half of the FOV, so the experimental result is not very satisfactory.
Second, the reconstruction effect of the proposed method is affected by the wavelength bandwidth of the light source. DOE is based on a single wavelength design. However, wavelength bandwidth exists in actual light sources such as LED, OLED. The light source used in our experiments is an OLED with a wavelength bandwidth of about 30 nm. The effect of wavelength bandwidth on the reconstruction effect is observed by simulation using the parameters in 4.1, as shown in Fig. 12(b), (d). It can be seen that as the wavelength bandwidth increases, the appearance of non-uniformity in the edge region. When wavelength bandwidth is 30 nm, the PSNR is reduced by 0.21 dB from 18.65 dB to 18.44 dB. Reconstruction results in less deterioration. The non-uniform diffraction spots at the edge region in the reconstruction result may be caused by the broadband light source.
In addition, the reconstruction results are affected by the diffraction distance. An inappropriate diffraction distance may cause the SGD optimization to converge to a local minimum. The distance constraint can be explained in terms of a variant of the maximum diffraction angle of the DOE, given by the following equation [33]:
In AR waveguide display systems, the eye relief is usually short. For the optical experiments in this paper, the DOE phase was loaded through the SLM, which required a PBS for coupling. This results in a large eye relief, and the sum of z1 and f is 60 mm. The z0 = 52 mm from Eq. (13). Thus, z1 + f > z0, which satisfies the distance limitation.
In commercial applications, the eye relief of AR waveguide display system usually needs to be 20 mm. Using a microstructured patterns to realize DOE phase delay can meet the system eye relief requirement. Using photolithography and other means to process DOEs, a sampling interval of 1µm can be realized. In this case, Eq. (13) shows that the z0 = 3.9 mm, which is much smaller than the 20 mm exit distance and satisfies the distance limitation.
Furthermore, our proposed method assumes that the parameters of the VGH are fixed and only the DOE is optimized. In the future, we may try to optimize both VHG and DOE in a pipeline using e.g. convolutional neural network means to expect better results.
6. Conclusion
In this paper, we propose a novel system to achieve high uniformity in VHG-based waveguide displays. We present a more comprehensive analysis of the angular uniformity of VHG-based waveguide display systems. The compensated DOE is designed to enhance the angular uniformity through phase compensation. Notably, the compensated DOE adds little to the size and weight of the system. The MO-SGD algorithm is introduced to optimize a single DOE used for compensating different image sources. The optimization capability of the proposed algorithm is further strengthened by incorporating a novel subjective evaluation, the LPIPS loss, which aligns more closely with human vision. This is combined with the MSE loss and MS-SSIM loss. Simulation results show that the proposed algorithm can effectively suppress the edge illumination degradation of the waveguide display system, and improve the PSNR of the reconstruction images by 5.54 dB. The effectiveness of the proposed method is verified by optical experiments. The measured nonuniformity (NU) against FOVs is improved by 53.05% from 0.3749 to 0.1760. This paper presents a new potential solution for error correction in commercial diffractive optical waveguides.
Funding
Major Science and Technology Projects in Anhui Province (202304a05020020).
Disclosures
The authors declare no conflicts of interest.
Data Availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
Supplemental document
See Supplement 1 for supporting content.
References
1. J. Xiong, E. L. Hsiang, Z. He, et al., “Augmented reality and virtual reality displays: emerging technologies and future perspectives,” Light: Sci. Appl. 10(1), 216 (2021). [CrossRef]
2. J. Carmigniani, B. Furht, M. Anisetti, et al., “Augmented reality technologies, systems and applications,” Multimedia Tools Appl. 51(1), 341–377 (2011). [CrossRef]
3. M. I. Olsson, M. J. Heinrich, D. Kelly, et al., “Wearable device with input and output structures,” U.S. Patent No. 9(285), 592 (2016).
4. L. Wei, Y. Li, J. Jing, et al., “Design and fabrication of a compact off-axis see-through head-mounted display using a freeform surface,” Opt. Express 26(7), 8550–8565 (2018). [CrossRef]
5. Z. Wang, X. Zhang, G. Lv, et al., “Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display,” Opt. Express 29(4), 4927–4935 (2021). [CrossRef]
6. Q. Jiang, G. Jin, and L. Cao, “When metasurface meets hologram: principle and advances,” Adv. Opt. Photon. 11(3), 518–576 (2019). [CrossRef]
7. Q. Wang, D. Cheng, Q. Hou, et al., “Design of an ultra-thin, wide-angle, stray-light-free near-eye display with a dual-layer geometrical waveguide,” Opt. Express 28(23), 35376–35394 (2020). [CrossRef]
8. A. Liu, Y. N. Zhang, Y. S. Weng, et al., “Diffraction efficiency distribution of output grating in holographic waveguide display system,” IEEE Photonics J. 10(4), 1–10 (2018). [CrossRef]
9. D. Ni, D. Cheng, Y. Wang, et al., “Design and fabrication method of holographic waveguide near-eye display with 2D eye box expansion,” Opt. Express 31(7), 11019–11040 (2023). [CrossRef]
10. Q. Wang, D. Cheng, Q. Hou, et al., “Stray light and tolerance analysis of an ultrathin waveguide display,” Appl. Opt. 54(28), 8354–8362 (2015). [CrossRef]
11. A. Frommer, “11-3: Invited paper: lumus optical technology for AR,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 48(1), 134–135 (2017). [CrossRef]
12. T. Levola, “7.1: Invited paper: novel diffractive optical components for near to eye display,” Dig. Tech. Pap. - Soc. Inf. Disp. Int. Symp. 37(1), 64–67 (2006). [CrossRef]
13. Z. Shen, Y. Zhang, A. Liu, et al., “Volume holographic waveguide display with large field of view using a Au-NPs dispersed acrylate-based photopolymer,” Opt. Mater. Express 10(2), 312–322 (2020). [CrossRef]
14. T. Levola and P. Laakkonen, “Replicated slanted gratings with a high refractive index material for in and outcoupling of light,” Opt. Express 15(5), 2067–2074 (2007). [CrossRef]
15. J. M. Miller, N. D. Beaucoudrey, P. Chavel, et al., “Design and fabrication of binary slanted surface-relief gratings for a planar optical interconnection,” Appl. Opt. 36(23), 5717–5727 (1997). [CrossRef]
16. R. Fernández, S. Bleda, S. Gallego, et al., “Holographic waveguides in photopolymers,” Opt. Express 27(2), 827–840 (2019). [CrossRef]
17. C. Pan, Z. Liu, Y. Pang, et al., “Design of a high-performance in-coupling grating using differential evolution algorithm for waveguide display,” Opt. Express 26(20), 26646–26662 (2018). [CrossRef]
18. D. Ni, D. Cheng, Y. Liu, et al., “Uniformity improvement of two-dimensional surface relief grating waveguide display using particle swarm optimization,” Opt. Express 30(14), 24523–24543 (2022). [CrossRef]
19. Y. Lin, H. Xu, R. Shi, et al., “Enhanced diffraction efficiency with angular selectivity by inserting an optical interlayer into a diffractive waveguide for augmented reality displays,” Opt. Express 30(17), 31244–31255 (2022). [CrossRef]
20. M. Moharam and T. K. Gaylord, “Rigorous coupled-wave analysis of planar-grating diffraction,” J. Opt. Soc. Am. 71(7), 811–818 (1981). [CrossRef]
21. M. Piao and N. Kim, “Achieving high levels of color uniformity and optical efficiency for a wedge-shaped waveguide head-mounted display using a photopolymer,” Appl. Opt. 53(10), 2180–2186 (2014). [CrossRef]
22. J. Yeom, Y. Son, and K. Choi, “Pre-compensation method for optimizing recording process of holographic optical element lenses with spherical wave reconstruction,” Opt. Express 28(22), 33318–33333 (2020). [CrossRef]
23. R. Gerchberg and W. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972).
24. M. Guo, G. Lv, J. Cai, et al., “Speckle-reduced diffractive optical elements beam shaping with regional padding algorithm,” Opt. Eng. 61(12), 125103 (2022). [CrossRef]
25. Y. Peng, S. Choi, N. Padmanaban, et al., “Neural holography with camera-in-the-loop training,” ACM Trans. Graph. 39(6), 1–14 (2020). [CrossRef]
26. J. Zhang, N. Pégard, J. Zhong, et al., “3D computer-generated holography by non-convex optimization,” Optica 4(10), 1306–1313 (2017). [CrossRef]
27. O. Russakovsky, J. Deng, H. Su, et al., “ImageNet large scale visual recognition challenge,” IJCV 115(3), 211–252 (2015). [CrossRef]
28. L. Shi, B. Li, C. Kim, et al., “Towards real-time photorealistic 3D holography with deep neural networks,” Nature 591(7849), 234–239 (2021). [CrossRef]
29. X. Shui, H. Zheng, X. Xia, et al., “Diffraction model-informed neural network for unsupervised layer-based computer-generated holography,” Opt. Express 30(25), 44814–44826 (2022). [CrossRef]
30. Y. Ting, S. Zhang, W. Chen, et al., “Phase dual-resolution networks for a computer-generated hologram,” Opt. Express 30(2), 2378–2389 (2022). [CrossRef]
31. J. Wu, K. Liu, X. Sui, et al., “High-speed computer-generated holography using an autoencoder-based deep neural network,” Opt. Lett. 46(12), 2908–2911 (2021). [CrossRef]
32. R. Zhang, P. Isola, A. Efros, et al., “The unreasonable effectiveness of deep features as a perceptual metric,” CVPR 586–595 (2018).
33. C. Chen, B. Lee, N. N. Li, et al., “Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function,” Opt. Express 29(10), 15089–15103 (2021). [CrossRef]
