Mar 13 (Wed) @ 9:00am: “Safe Robot Autonomy in Interactive Open-World Environments, Accelerated by Generative AI,” Roya Firoozi, Postdoc Researcher, Stanford

Abstract

Generative AI offers the potential to empower autonomous robots with the ability (i) to connect abstract concepts to 3D scene geometry and (ii) to enable complex intention prediction and collaboration capabilities for robots to interact intelligently with humans and other autonomous agents. However, there remain significant research challenges in how to integrate these generative AI technologies into the robot autonomy stack without compromising safety, reliability, and explainability. In this talk, I will present my recent research toward addressing these issues. In the first part, I will discuss how large pre-trained Vision-Language Models allow robots to connect abstract concepts to 3D scene geometry, enabling autonomous interaction with objects and 3D scenes.I'll demonstrate how language and feature distillation into 3D world representations, such as Neural Radiance Fields, facilitate robots’ open-vocabulary navigation and manipulation.  

Beyond interaction with objects and 3D scenes, robots must safely interact with other coexisting dynamic agents in the scene including other robots or humans. As they interact, they need to reason about and predict other agents' objectives and reactions. In the second part of the talk, I’ll delve into safe and intelligent multi-agent interactions modeled through distributed real-time optimization techniques such as model predictive control (MPC) and game-theoretic planning. I will detail how game-theoretic planning, closely related to distributed MPC, introduces an additional layer of abstraction to interactive autonomy by modeling agents' preferences and mutual influences. I will provide an overview of how to enhance these interaction modeling tools' capabilities with large generative AI models. I’ll briefly touch on paths forward, including employing generative AI models like diffusion in distributed multi-robot learning to foster the emergence of cooperation; using generative AI as adversarial agents to induce robustness in autonomous systems, building new foundation models for interaction modeling, and evaluating safety as we integrate these foundation models into physical robotic systems.

Bio

Roya Firoozi is currently a postdoctoral researcher at Stanford University, Multi-Robot Systems Laboratory. She received her bachelor's degree in mechanical engineering and her Ph.D. in control theory, with minors in optimization and machine learning from the University of California, Berkeley. Her research focuses on enabling safe robot autonomy powered by generative AI in interactive, open-world environments. Roya has received the NSF Postdoctoral Fellowship Award, the UC Berkeley Outstanding Graduate Student Instructor Award, and the UC Berkeley Chancellor’s Award. She was recognized as a rising star in EECS and Aerospace Engineering. Outside of her research, she has served as the robotics mentor at Stanford AI4ALL outreach program that aims to increase diversity in artificial intelligence. 

Hosted by: ECE Department

Submitted by: Megan Ashley <mmashley@ucsb.edu>