Microsoft HoloLens 2 and Azure Kinect DK as tools for computer vision research – Tutorial @ ECCV 2020

Conference URL: ECCV 2020 (opens in new tab)

Microsoft HoloLens 2 is a holographic computer, but it’s also a potent computer vision research device. Applications can access audio and video streams and surface meshes, all in a world coordinate space maintained by HoloLens’ highly accurate head-tracking. This tutorial will dive into the new “Research Mode” capability of HoloLens 2 and the open source SDK of Azure Kinect DK. For HoloLens 2, we will show you how to access all the raw streams on device, including depth and IMU, and combine them with the hand tracking and eye gaze tracking capabilities offered by the device.

We will also show how to use the open source SDK of Azure Kinect DK to synchronize and calibrate multiple cameras, extract extrinsic and intrinsic camera information and extract sound source information from the 7-microphone array. The tutorial will be illustrated by multiple example applications.

Program Committee Members

Marc Pollefeys
Jamie Shotton
Silvano Galliani
Federica Bogo
Johannes Schönberger
Dorin Ungureanu
Pooja Sama

Program

1. Microsoft HoloLens 2 and Azure Kinect DK as tools for computer vision research. M. Pollefeys
2. HoloLens 2 Research Mode: API overview. D. Ungureanu, P. Sama
3. The HoloLens2ForCV Repository. F. Bogo
4. Azure Kinect DK SDK for computer vision research. S. Galliani
5. Azure Kinect DK demo. S. Galliani