Recording, Producing, and Rendering Binaural 3D Audio for Virtual Reality
IEEE Rochester Section Technology and Enginneering Management Society Joint Meeting with Life Member Affinity Group
Spatial audio creates an immersive audio experience for listeners. Today, we can hear spatial audio in movies, streaming music, 360/3D videos, VR games, and VR apps. The most direct way to enjoy spatial audio is not through a Dolby Atmos or DTS:X multi-channel sound system, such as a soundbar, but through a pair of headphones. Binaural audio, which delivers the two-channel sound directly to the listener’s two ears with headphones, can recreate the acoustic signals a listener would hear under natural conditions. It can also preserve the interaural time difference (ITD) and interaural level difference (ILD) cues, which are crucial for sound localization.
Since Fall 2017, our 3D audio research team at the Department of Electrical and Computer Engineering, University of Rochester, has recorded over 70 live concerts at the Eastman School of Music and several other concert venues with various binaural and Ambisonic microphones, as well as 360/3D VR cameras. We have built a spatial audio recording database for research and published many concert videos on YouTube. This online lecture will introduce different methods of recording, producing, and rendering head-locked or head-tracking binaural audio for VR videos, games, and apps. It will also demonstrate how to measure binaural impulse responses in a hall and use them to generate binaural sounds for audiovisual VR testing. Please wear headphones to hear binaural 3D audio demos.
Date and Time
Location
Hosts
Registration
- Date: 27 Dec 2022
- Time: 07:00 PM to 08:00 PM
- All times are (UTC-05:00) Eastern Time (US & Canada)
-
Add Event to Calendar
- Contact Event Hosts
- Co-sponsored by Rochester Section Life Member Affinity Group
- Starts 14 December 2022 12:00 AM
- Ends 27 December 2022 09:00 PM
- All times are (UTC-05:00) Eastern Time (US & Canada)
- No Admission Charge
Speakers
Ming-Lun Lee of University of Rochester
Biography:
Ming-Lun Lee received his Ph.D. in historical musicology and music theory from the University at Buffalo, as well as master’s degrees in electrical engineering and musicology from National Taiwan University. Lee had done research on intersymbol interference simulation for indoor wireless communication and published a paper in IEEE Transactions on Vehicular Technology in 2000. He had also completed several empirical studies on classical recordings, including his Ph.D. dissertation titled Britten Conducting Britten: A Study of the Recordings Produced with John Culshaw.
Lee’s current research focuses on spatial audio for AR/VR. After receiving the AR/VR Pilot Funding Award in 2017, he has recorded over 70 concerts with binaural/Ambisonic microphones and 360/3D VR cameras at the Eastman School of Music and many concert venues in the Greater Rochester Area. He also collaborates in research projects on developing frameworks for evaluating spatial audio system performance and audiovisual integration in VR renderings of real physical spaces. Lee has presented papers or offered workshops at the Audio Engineering Society Convention, American Musicology Society Annual Meeting, RIT Frameless XR Symposium, and Buffalo Graduate Symposium on Music.
Lee started to teach at the University of Rochester in 2013 and is currently an Associate Professor of Electrical and Computer Engineering (ECE). With teaching and research experience in both electrical engineering and musicology, Lee is well fitted to the interdisciplinary nature of the Audio and Music Engineering (AME) program. He has taught audio programming using various programming languages, including C/C++, Objective-C, ChucK, Csound, Faust, Max, Pure Data, SuperCollider, and Swift. He is particularly interested in programming for interactive multimedia performance. His AME/ECE courses include Audio Software Design I & II, Interactive Computer Music, Intro to C/C++ Programming, and Revolutions in Sound: Artistic and Technical Evolution of Sound Recording. Lee has received the 2015 Wadsworth C. Sykes Engineering Faculty Award for developing the Interactive Computer Music course. He is the current faculty counselor of the IEEE University of Rochester Student Chapter.
Email:
Address:620 Computer Studies Building, , Rochester, New York, United States, 14627
Agenda
7:00 PM - 8:00 PM (Eastern Time): Lecture and Q&A
This is a virtual/online meeting. Instructions for joining the meeting will be sent to registrants before the meeting.