ECE Distinguished Lecture – Jan 26 (Fri) @ 2:00pm: "Augmenting Human Auditory Perception with AI," Shyam Gollakota, Washington Research Foundation Endowed Professor, UW

Come at 1:00p for Cookies, Coffee and Conversation!
DISTINGUISHED LECTURE at the ECE SEMINAR SERIES

Abstract

The human auditory perception can be limited in noisy environments. Imagine being in a crowded room with a cacophony of sounds and having the ability to focus on or remove sounds from a specific 2D region or based on their semantic descriptions. This requires understanding and manipulating an acoustic scene, isolating each sound, and associating a 2D spatial context or semantic meaning with each constituent sound — a challenging set of tasks even for the human brain.

In this talk, I will present a series of projects with my students and collaborators that  augment human auditory perception with AI. I will introduce a shape-changing smart speaker that allows us to separate and localize concurrent human speakers in previously unseen real-world 2D space. I will show that this can enable novel applications like mute zones (parts of the room where sounds are muted), active zones (regions where sounds are captured), multi-conversation separation, and location-aware interaction. I will then introduce semantic hearing, a novel capability for headsets and earbuds devices that enables them to focus on or ignore specific sounds from real-world environments in real-time, while also preserving spatial cues. Additionally, I will present ongoing work on hearables that can, in real-time, program and manipulate acoustic scenes to selectively pick speakers to hear based on their distance — a task that can be challenging for the human auditory system to perform.  Finally, I will hypothesize about what the future of augmented intelligence may look like and how we are entering a new era for intelligent mobile systems that can make science fiction a reality.

Bio

Shyam Gollakota is a Washington Research Foundation endowed Professor at the Paul G. Allen School of Computer Science & Engineering in the University of Washington. His work has been licensed by ResMed Inc, our startup Sound Life Sciences acquired by Google, and is in use by millions of users. He was also CEO of a startup where we obtained FDA 510(k) clearance for the technology developed in my lab. His lab also worked closely with the Washington Department of Agriculture to wirelessly track invasive "murder" hornets, which resulted in the destruction of the first nest in the United States. He is the recipient of the ACM Grace Murray Hopper Award in 2020 and recently named as a Moore Inventor Fellow in 2021. He was also named in MIT Technology Review’s 35 Innovators Under 35, Popular Science ‘brilliant 10’ and twice to the Forbes’ 30 Under 30 list. His group’s research has earned Best Paper awards at MOBICOM, SIGCOMM, UbiComp, SenSys, NSDI and CHI, appeared in interdisciplinary journals like Nature, Nature Communications, Nature Biomedical Engineering, Science Translational Medicine and Science Robotics as well as named as a MIT Technology Review Breakthrough technology of 2016 as well as Popular Science top innovations in 2015. He is an alumni of MIT (Ph.D., 2013, winner of ACM doctoral dissertation award) and IIT Madras.

Hosted by: The ECE Seminar Series

Submitted by: Yasamin Mostofi <ymostofi@ece.ucsb.edu>