Open Access
Add to BibSonomyAbstract
Head-worn displays have begun to infiltrate the commercial electronics scene as mobile computing power has decreased in price and increased in availability. A prototypical head-worn display is both lightweight and compact, while achieving high quality optical performance. In off-axis geometries, freeform optical surfaces allow an optical designer additional degrees of freedom necessary to create a device that meets these conditions. In this paper, we show two optical see-through head-worn display designs, both comprising two freeform elements with an emphasis on visual space assessment and parameters.
© 2014 Optical Society of America
1. Introduction
Wearable displays, such as head-worn displays (HWDs), represent the newest entry into the ever-evolving augmented reality market [1, 2]. Packaging constraints for HWDs align with the constraints of other devices that are worn on one’s head. This necessitates such devices to be lightweight and compact, meaning using a minimum number of optical elements and employing unique packaging geometries [3, 4]. The problem remains that when the number of optical elements is reduced and non-rotationally symmetric geometries are used, the resulting optical aberrations are too great to correct with rotationally symmetric optical surfaces. This limitation can be overcome by implementing freeform surfaces, or surfaces without rotational symmetry [5]. Because the design problem of HWDs is so highly constrained (element count, geometry), the surface shapes play an extremely important role. Some surface descriptions that have been applied to HWDs include: XY polynomials [6–8], φ-polynomials [9], and radial basis functions [10]. For this paper, we work with a set of φ-polynomials called Zernike polynomials because they have a close relationship with the wavefront expansion terms, allowing us to directly leverage the optical design insight provided by nodal aberration theory [11, 12].
When designing an optical system using computer aided optimization software, it is customary to design from the long conjugate to the short conjugate, and for a visual optical system, light directed towards the eye is either collimated or near collimated, meaning its conjugate is at or near optical infinity. As a result, for visual systems, the design starts at the entrance pupil to the eye, and the rays travel from the eye rather than to the eye. Coincident with the eye’s entrance pupil, the aperture stop of the design is placed in object space to define a perfect pupil to match with the eye.
To understand how a visual optical system will function when used as intended, a series of analyses in visual space were conducted [13]. Assessing the optical system in visual space requires the visual system as designed to be flipped so that the rays trace from the source to the eye, as it would when in use. Extra care must be taken when inverting a system with freeform surfaces because the ray tracing software may not properly handle these surface types with built-in functions. After inverting the design, the image space ray bundles are collimated or are nearly so. This puts the image plane an optically-infinite distance away, making any analysis in dimension (position) space not meaningful, so we must analyze the inverted optical system in angle space. In this paper, we present two optical see-through head-worn displays designs, including their sensitivity to potential manufacturing errors and misalignment at assembly, both comprising two freeform elements with an emphasis on visual space assessment .
2. Optical design
When starting an optical design, there are three critical parameters that need to be well-defined: the operational waveband, the aperture stop size, and the field of view (FOV). The waveband is important because it dictates the materials one can use, and, for a visual system that requires an external aperture stop, lateral chromatic aberrations are often the limiting aberrations of the system [14]. We avoid both of these potential problems in our design by using all reflective surfaces. As an added benefit, reflective optics in an off-axis geometry allow us to package the optics in a tight three dimensional geometry and, as such, bend the light into a small package size. The remaining two parameters often fight against one another according to the Lagrange invariant, where a given value for the Lagrange invariant reflects the overall complexity of the design [15]. A large aperture stop requires a smaller FOV, and vice-versa. Our design strategy was to set the eyebox size to a diameter that was slightly larger than the eye’s pupil in order to accommodate a sunglasses form factor as a first priority, while still allowing the simple placement of the human eye within the eyebox. A relaxed eye in photopic conditions has roughly a 2-3 mm entrance pupil diameter, so designing with a ~5 mm eyebox allows for an eyeball scanning tolerance of about ± 11 degrees [16]. We then aimed for a FOV in the 20-25 degree range and monitored both the optical aberrations and the package size.
A disadvantage of rotationally symmetric reflective systems is that they are obscured, which decreases the total intensity of light and introduces diffraction effects at the image plane. Designers can use a variety of strategies to make reflective systems unobscured: 1) the aperture stop can be offset from the mechanical axis, 2) the FOV can be biased, meaning a portion of the off-axis fields are considered the active fields for the system, 3) a combination of 1) and 2), or 4) the surfaces themselves can be tilted [9]. For systems with rotationally symmetric surfaces, the lattermost method is typically avoided because tilting powered surfaces results in optical aberrations such as field-constant astigmatism and coma that may not be balanced using rotationally symmetric surfaces. However, if the surfaces are allowed to be freeform, method 4) becomes feasible. For this reason, our design employed tilted reflective freeform surfaces.
Another packaging constraint for HWDs is the requirement that they fit around the facial structures near the eye. The distance from the entrance pupil of the optical design, which will be coincident with the entrance pupil of the eye, to the closest design feature (i.e. optical element, detector, microdisplay, etc.) must be great enough to clear the brows, nose, and/or cheek bones.
The starting designs were all-spherical designs folded into each final, unobscured geometry. To illustrate the deleterious effect that tilting the mirrors has on the astigmatism and coma, we show the astigmatism and coma contributions across the full FOV for an obscured rotationally symmetric system and an unobscured, tilted system in Fig. 1 and Fig. 2, respectively. This behavior can be thought of, in a nodal aberration theory sense, as the node(s) moving far outside the field of view as the system is made non-symmetric. The relatively small coma and astigmatism contributions are increased by an order of magnitude as a result of tilting the mirrors. The mirror tilts for each system were optimized to minimize aberrations while still maintaining an unobscured geometry using ray clearance constraints.
Fig. 1 (a) Astigmatism and (b) coma contributions across the full FOV for a representative obscured rotationally symmetric system. Coma is the dominant aberration with only about λ/5 P-V.
Fig. 2 (a) Astigmatism and (b) coma contributions across the full FOV after tilting the surfaces to form a representative unobscured, non-rotationally symmetric system. Tilting the surfaces has resulted in significant amounts of nearly field-constant coma and, the now dominant, astigmatism with about 4 waves.
The sag, z(x,y), of the freeform surfaces in each design is defined mathematically by,
where c is the curvature of the base sphere, k is the conic constant, ρ is the radial coordinate of the surface, Cj is weight factor on the jth Zernike term, Zj. The first term is a base conic (a sphere being a special case), upon which the second term, a weighted sum of Zernike polynomials, is overlaid. A consequence of having an external stop in an optical system that is significantly displaced from the first surface is that the object fields may be thought to “walk” about the surfaces; this means that each field point experiences a different portion of the surface. As explained in [9], each field receives a different contribution to its net aberration field from a surface depending on its location with respect to the vertex of the surface and the surface shape. The type of aberration correction is proportional to the pupil derivatives of the surface description. For example, if the surface is purely comatic, the resulting aberration correction would be astigmatic as well as other aberrations with lower-order pupil dependence. As a result of the external stop, the system cannot easily correct for field-constant aberrations. However, we can still correct the residual field-dependent aberrations and quantify that a high performance system can still be designed. The final specifications for each design are reported in Table 1.
Table 1. Optical Design Specification Table
The final design forms for the two HWDs can be seen in Fig. 3. Sags maps with respect to the base sphere are shown in Fig. 4. The main contribution of all four freeform surfaces is astigmatism. The mirrors have been constrained to be concave to minimize the package size. Convex mirrors diverge light, requiring larger subsequent mirror apertures and, thus, a larger package size as a whole. It is important to minimize the tilts of all of the mirrors to reduce the tilt induced astigmatism and coma. This will reduce the overall amount of correction the mirrors need to provide, decreasing the amount of freeform departure for the surfaces, which facilitates the fabrication of the surfaces. Each HWD fits inside a spherical volume with a radius slightly less than 14 mm. Figure 5 shows Design 2 mounted on a model of a head to depict the relative scale of the system.
Fig. 3 2-dimensional optical layout of (a) Design 1 and (b) Design 2. They both consist of two powered mirrors. Design 1 has an additional fold flat that serves as the combiner, whereas a powered mirror serves that purpose in Design 2 [17].
Fig. 4 Surface figure maps with the base sphere removed. Clockwise from top left, the secondary mirror of Design 1, the tertiary mirror of Design 1, the secondary mirror of Design 2 and the primary mirror of Design 2.
3. Display space analysis
We evaluated the performance of the systems by calculating the modulation transfer function (MTF) over the full FOV at 100% and 75% of the Nyquist frequency of the OLED microdisplay (Model: MICROOLED Maryland). Figure 6 and Fig. 7 show the MTF results for Design 1 and Design 2, respectively. We are using the CODE V Full-Field Display option because freeform surface optimization techniques often lead to excellent performance at the field points for which the system was optimized, but unsatisfactory performance for those field points in-between. We also calculated the distortion of each HWD using the CODE V distortion grid macro, also shown in Fig. 6 and Fig. 7 for Design 1 and 2, respectively. While the distortion is small (< 6.2%) real-time correction of off-axis distortion can be implemented to completely correct the residual distortion [18].
Fig. 6 Design 1 performance analysis. (Left) MTF FFDs shown for two object orientations (0° and 90°) and two frequencies (50 lp/mm and 35 lp/mm). (Right) Distortion grid showing < 1.5% distortion.
Fig. 7 Design 2 performance analysis. (Left) MTF FFDs shown for two object orientations (0° and 90°) and two frequencies (50 lp/mm and 35 lp/mm). (Right) Distortion grid showing < 6.2% distortion.
4. Visual space analysis
While the display space analysis can give the designer a quick glimpse into the overall performance of the optical system, it cannot accurately predict how the system will perform when it is used as intended. There are, however, methods that can. In ray trace software, the analyses are typically done in the same configuration as the system is designed, but visual systems, including the HWDs described in this paper, are designed such that the OLED microdisplay is in image space, so we cannot immediately analyze what the eye will “see”. The solution is to flip the system. After flipping the optical system, image space is now the same space in which the eye is located; therefore any analysis completed will be in visual space.
In visual space, we choose to work in units of angle because the image plane is infinitely far away. The human eye can resolve roughly 1 cycle/arcminute on-axis where the photoreceptors are most densely packed (rods only) and degrades significantly off-axis due to a lower density of photoreceptors (rods and cones) [19]. This represents a maximum useful resolution for any design and analysis done in visual space. Systems with resolution greater than 1 arcmin may be considered as overdesigned for the visual system. Our system, however, is limited by the OLED microdisplay and its pixel pitch of 10 microns. Dividing the full FOV by the pixel array size in the horizontal direction gives us an idea of how close our system comes to being eye-limited. Design 1 and Design 2 have a maximum visual resolution of 1.5 arcmin/pixel. We evaluated the visual space MTF of both systems out to a frequency of 0.65 cycles/arcmin, which is an equivalent measurement of the maximum visual resolution of the system. Figure 8 shows the visual MTF plots for Design 1 and 2.
Fig. 8 FFD MTF plots in visual space for (a) Design 1 and (b) Design 2. The top and bottom rows represent object orientations of 0° and 90°, respectively. Plots are shown for 0.65 cylces/arcmin (the maximum resolution based on the OLED) and 0.45 cycles/arcmin (70% of the maximum resolution).
5. Sensitivity analysis
In an ideal world, as-built optical systems would have the same performance as optical design simulations. However, a harsh reality is that systems can only be built to within a certain range of accuracy and need to be designed with these uncertainties in mind, or, at least, tested to make certain the system maintains satisfactory performance when built. To test the assembly and alignment sensitivity of the HWD systems, each parameter is perturbed individually and the resulting drop in MTF is recorded. The Root-Sum-Square (RSS) of the MTF drops is then taken. This treatment assumes that for each tolerance type, the maximum tolerance values rarely occur. The RSS value is then subtracted from the nominal MTF to yield the predicted as-built performance. The assembly variables include the three independent tilts and two orthogonal decenters of each mirrors, the air spaces between the mirrors, and two tilts and decenters of the image plane. We use a set of loose tolerances: 5 arcminute tilts and 50 µm for airspaces and decenters. Focus compensators are assumed and are labeled as such in the table. Surface figure errors equal to 0.25 waves of spherical aberration, coma, and astigmatism are added to each surface to model the surface fabrication errors. Table 2 shows the percent MTF drop for each tolerance at both the on-axis field and the most sensitive off-axis field. After applying the tolerances, we see that the drop in MTF is quite manageable, even with the loose tolerances defined for this study. The most sensitive tolerances (shown in boldface) for Design 1 are the Y-decenter of the tertiary mirror and the airspace between the secondary and tertiary mirrors. The most sensitive tolerances (shown in boldface) for Design 2 are the Y-Decenter and Tip of the secondary mirror. Only negative effects were taken into consideration in this tolerancing study. Positive changes in MTF were treated as no change (indicated by 0).
Table 2. Percent MTF drop for each tolerance at 35 lp/mm.
6. See-through
A key component of these systems is that they are optical see-through HWDs, meaning they offer an unobstructed view of the surrounding environment, while overlaying a magnified image of the microdisplay. Optical see-through augmented reality is the preferred method of merging the real and the virtual worlds because it provides a high-resolution and large FOV view of the real world [20]. As shown in Fig. 3(a), the three mirror HWD is comprised of two powered mirrors and a single flat, fold mirror. The fold mirror will be coated to be partially reflective and partially transmissive to provide both images to the user. Additionally, the use of photochromic or electrochromic coatings will allow the user to darken the real scene in full sunlight to maintain sufficient contrast of the superimposed virtual image. The thin fold mirror of Design 1 offers a simple realization of optical see-through; it introduces no image degrading aberrations.
Compared to the three-mirror version, the two-mirror HWD requires a bit more finesse when devising a solution to the see-through problem. Light from the environment passes through the primary mirror, which, in this case, is a freeform element. Without additional aberration correction, the user would see a blurred image of the environment. This can be fixed by making the rear surface of the primary mirror freeform as well. By optimizing the rear surface of the primary mirror with the same set of Zernike polynomials as the front surface, we are able to attain diffraction limited performance for the see-through imaging. Due to the large dynamic range necessary to fabricate aggressive freeform elements, the current techniques used to generate these surfaces are limited to diamond turning, non-diamond milling or machining, and diamond micro-milling, as in [21].
7. Conclusion
See-through HWDs are some of the most challenging HWDs to design and fabricate because of the difficulty associated with creating an aesthetically pleasing and unobtrusive system while maintaining high optical performance. We presented two all-reflective, see-through HWD designs with freeform elements. Freeform elements are critical in these designs because they allow unique folded geometries that would otherwise be optically unacceptable. Specifically, these unique folded geometries were successfully demonstrated with an external pupil that imposes stringent constraints on these compact designs. Finally, the designs we presented image a small 8.5 mm x 5.0 mm active area microdisplay into up to a 25 degree full FOV. In this design paper, we emphasized a comprehensive analysis in visual space, which provides information to the optical designer that would otherwise be unknown – how the system performs in visual space – which may also support vision and perception studies with the manufactured system.
Acknowledgments
This work was supported by the National Science Foundation (EECS-1002179) and Revision Military, in the form of a student fellowship in freeform optical design. We also thank Carl Zeiss Corporation for a student fellowship in support of the development of general MATLAB tools for freeform optical design. We thank Synopsys, Inc. for the student license of CODE V, which was used for this research and Microoled Corporation for the donation of the microdisplays that will be used in the full implementation of a system. We also thank Kyle Fuerschbach for his meaningful conversations and help with the 3D models.
References and links
1. R. T. Azuma, “Augmented reality: Approaches and Technical Challenges,” in Fundamentals of Wearable Computers and Augumented Reality, W. Barfield and T. Caudell, eds. (L. Erlbaum Associates Inc., 2000), pp. 27–63.
2. O. Cakmakci and J. Rolland, “Head-Worn Displays: A Review,” J. Display Technol. 2(3), 199–216 (2006). [CrossRef]
3. J. P. Rolland, K. P. Thompson, H. Urey, and M. Thomas, “See-Through Head Worn Display (HWD) Architectures,” in Handbook of Visual Display Technology, J. Chen, W. Cranton, and M. Fihn, eds. (Springer, 2012), pp. 2145–2170.
4. O. Cakmakci and J. P. Rolland, “Examples of HWD Architectures: Low-, Mid- and Wide-Field of View Designs,” in Handbook of Visual Display Technology, J. Chen, W. Cranton, and M. Fihn, eds. (Springer, 2012), pp. 2195–2211.
5. W. T. Plummer, “Unusual optics of the Polaroid SX-70 Land camera,” Appl. Opt. 21(2), 196–208 (1982). [CrossRef] [PubMed]
6. A. Takagi, S. Yamazaki, Y. Saito, and N. Taniguchi, “Development of a stereo video see-through HMD for AR systems,” in Proceedings of IEEE and ACM International Symposium on Augmented Reality, 2000), 68–77. [CrossRef]
7. O. Cakmakci and J. Rolland, “Design and fabrication of a dual-element off-axis near-eye optical magnifier,” Opt. Lett. 32(11), 1363–1365 (2007). [CrossRef] [PubMed]
8. D. Cheng, Y. Wang, H. Hua, and M. M. Talha, “Design of an optical see-through head-mounted display with a low f-number and large field of view using a freeform prism,” Appl. Opt. 48(14), 2655–2668 (2009). [CrossRef] [PubMed]
9. K. Fuerschbach, J. P. Rolland, and K. P. Thompson, “A new family of optical systems employing φ-polynomial surfaces,” Opt. Express 19(22), 21919–21928 (2011). [CrossRef] [PubMed]
10. O. Cakmakci, B. Moore, H. Foroosh, and J. P. Rolland, “Optimal local shape description for rotationally non-symmetric optical surface design and analysis,” Opt. Express 16(3), 1583–1589 (2008). [CrossRef] [PubMed]
11. R. V. Shack and K. Thompson, “Influence of alignment errors of a telescope system on its aberration field,” Proc. SPIE 0251, 146–153 (1980). [CrossRef]
12. K. Thompson, “Description of the third-order optical aberrations of near-circular pupil optical systems without symmetry,” J. Opt. Soc. Am. A 22(7), 1389–1401 (2005). [CrossRef] [PubMed]
13. Y. Ha and J. Rolland, “Optical assessment of head-mounted displays in visual space,” Appl. Opt. 41(25), 5282–5289 (2002). [CrossRef] [PubMed]
14. W. J. Smith, Modern Optical Engineering, 3rd ed. (McGraw-Hill, 2000).
15. M. Born and E. Wolf, Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light (CUP Archive, 1999).
16. C. E. Rash, Helmet Mounted Displays: Design Issues for Rotary-Wing Aircraft (SPIE Press, 1999).
17. A. Bauer, US Provisional Patent Application (61/827,033) filed 5/23/13.
18. A. Bauer, S. Vo, K. Parkins, F. Rodriguez, O. Cakmakci, and J. P. Rolland, “Computational optical distortion correction using a radial basis function-based mapping method,” Opt. Express 20(14), 14906–14920 (2012). [CrossRef] [PubMed]
19. B. H. Walker, Optical Design for Visual Systems (SPIE Press, 2000).
20. J. P. Rolland and H. Fuchs, “Optical versus video see-through head-mounted displays in medical visualization,” Presence (Camb. Mass.) 9(3), 287–309 (2000). [CrossRef]
21. P. J. Smilie, B. S. Dutterer, J. L. Lineberger, M. A. Davies, and T. J. Suleski, “Design and characterization of an infrared Alvarez lens,” Opt. Eng. 51, 013006 (2012).
