Parametric Directional Coding for Precomputed Sound Propagation

ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2018 | , Vol 37(4)

DOI

Convincing audio for games and virtual reality requires modeling directional propagation effects. The initial sound’s arrival direction is particularly salient and derives from multiply-diffracted paths in complex scenes. When source and listener straddle occluders, the initial sound and multiply-scattered reverberation stream through gaps and portals, helping the listener navigate. Geometry near the source and/or listener reveals its presence through anisotropic reflections. We propose the first precomputed wave technique to capture such directional effects in general scenes comprising millions of polygons. These effects are formally represented with the 9D directional response function of 3D source and listener location, time, and direction at the listener, making memory use the major concern. We propose a novel parametric encoder that compresses this function within a budget of ~100MB for large scenes, while capturing many salient acoustic effects indoors and outdoors. The encoder is complemented with a lightweight signal processing algorithm whose filtering cost is largely insensitive to the number of sound sources, resulting in an immediately practical system.

Interactive sound simulation: Rendering immersive soundscapes in games and virtual reality

The audio-visual immersion of game engines and virtual reality/mixed reality has a vast range of applications, from entertainment to productivity. Physical simulation is required in these applications to produce nuanced, believable renderings that respond fluidly to unpredictable user interaction. Simulating sound phenomena synchronized with visuals must be done within tight real-time constraints. The wave behavior of audible sound is quite different from visible light, requiring fundamentally distinct techniques. The resulting challenges have impeded practical adoption in the past, but these barriers are finally being overcome, with accelerating usage of advanced sound technologies in interactive applications today. In this webinar led by Microsoft Principal Researcher Dr. Nikunj Raghuvanshi, learn the ins and outs of creating practical, high-quality sound simulations. You will get an overview of the…

Parametric Directional Coding for Precomputed Sound Propagation

Convincing audio for games and virtual reality requires modeling directional propagation effects. The initial sound's arrival direction is particularly salient and derives from multiply-diffracted paths in complex scenes. When source and listener straddle occluders, the initial sound and multiply-scattered reverberation stream through gaps and portals, helping the listener navigate. Geometry near the source and/or listener reveals its presence through anisotropic reflections. We propose the first precomputed wave technique to capture such directional effects in general scenes comprising millions of polygons. These effects are formally represented with the 9D directional response function of 3D source and listener location, time, and direction at the listener, making memory use the major concern. We propose a novel parametric encoder that compresses this function within a budget of ~100MB for…