Summary

In this project, we explore how digital holography can be used to build novel near-eye displays for virtual and mixed (or augmented) reality.  We experiment with true, phase-only holograms in which the image is formed by the interference of laser light. We address some of the known limitations of digital holograms and demonstrate how holography can add powerful, new features to near-eye displays: per-pixel focus control, vision correction, and unpresented combinations of form factor and field of view.  See the technical paper and video for limitations, additional details, and a description of future work. Also see our blog post on the project.

Note that this Microsoft Research project is not necessarily indicative of any Microsoft product roadmap, but relates to basic research around holographic displays.

Addressing the Limitations of Holograms

Image quality

Digital holograms are often associated with noisy, low contrast, low resolution, and mono color imagery. We demonstrate how high contrast, high resolution, and full color digital holograms can be formed using existing hardware devices, yielding images that more closely match the quality of conventional displays. The following images are photographs taken of a prototype holographic near-eye display. The field of view is 70 degrees horizontally.

Computation Speed

Due to the complex required calculations, digital holograms are also often associated with slow, off-line calculations. We propose the use of eye tracked approximate holograms that have correct image focus and best image quality where the user is looking. Combined with GPU-accelerated algorithms, we demonstrate real-time hologram generation at rates of 90-260 Hz on a desktop GPU (NVIDIA GeForce GTX 980 TI).

Adding a Powerful Feature Set to Near-Eye Display

Focus Control

The ability to change the image focus is desirable in a near-eye display: it addresses the accommodation-convergence conflict, allows image focus to match eye focus in a see-through display, and adds realism. Unlike vari-focal, multi-focal, and light field displays, a holographic display is able to provide per-pixel focus control with virtually no discretization, enabling smooth and natural focal cues to imagery. Below, we demonstrate the ability to display holograms with per-pixel focus control with high resolution and image quality on a prototype display. Note that the region of camera focus (the dragon’s chest) is in focus, while closer regions (e.g. the head)  farther regions (e.g. the main body and tail) are out of focus. The inset images show various regions of the dragon’s body when they are brought in and out of focus; see the video for an animated change of focus. (This result was calculated offline.)

Vision Correction

Another desirable quality of a display is vision correction, the ability to fix defects in the user’s vision. Such a display allows a user to view the display without their glasses. We demonstrate that holographic displays are capable of a powerful vision correction capability: they can correct simple vision problems such as near- and far- sightedness as well as higher order vision problems like astigmatism. In the image below we demonstrate the ability of a prototype holographic display to correct astigmatism. In the left image, for reference, we show a image on a prototype display for a user with normal vision. In the center image, the display is viewed with astigmatic vision by the addition of a cylindrical lens in front of the camera; note that lines that are primarily horizontal are quite blurred in the vertical direction. Finally, in the right image, the display is viewed with astigmatic vision, but with vision correction applied in the hologram. The image looks virtually the same as the display viewed with normal vision.

Aberration correction

Holographic displays are also able to provide aberration correction, the ability to encode optical corrections to the display optics in software. Such a capability allows the use of simpler optics and enables new optical architectures, as we describe in the next section. The optical corrective capability of holograms are extremely powerful and allow arbitrary optical corrections on a per-pixel basis. This is akin to using an independently customized, complex lens to form each point in the image, and where each of these lenses acts independently yet can overlap with its neighbors. In the image below, we show images from an optical see-through holographic display in a challenging off-axis optical configuration. In the top left image, we show a hologram with no aberration correction applied; the off-axis optics result in severe astigmatism so that no part of the image is both in proper horizontal and vertical focus. In the top right image, we show a hologram with aberration correction applied — note that the lines are sharp in both directions over the whole field of view. Finally, in the bottom image, we demonstrate the ability to display complex, full color, optical see-through holograms with good image quality after the application of aberration correction.

Form Factor

One of the most important considerations for a near-eye display is form factor, especially for see-through, mixed reality devices. Lightweight, eyeglasses-like displays are needed to facilitate viewer comfort and all-day use. Using a powerful wavefront correction ability, we demonstrate that holographic near-eye displays enable a combination of form factor and field of view that have been inaccessible through conventional means. As a first step, we demonstrate a prototype near-eye holographic display in a sunglasses-like form factor with a wide 80 degree horizontal field of view (left image). The display uses a thin and highly transparent holographic optical element as a combiner, cut in the shape of an eyeglass lens, allowing an optical see-through capability (center image). Even with these miniaturized optics, the display can resolve pixel-scale details over the whole field of view (right image) — note in the image that the line width of the text and the spacing between the dot of the letter i are only one pixel. We find promise in this early prototype but note that the display is monoscopic and the driving electronics are external; significant challenges remain to build a practical stereo display.

Video