"Learning Vulnerabilities using Deception, Targeted Search, and Innovative Experimentation"

Project PI: João Hespana, Distinguished Prof. ECE, Control Systems

The research project is a collaborative effort between the University of Southern California (USC), the University of California, Santa Barbara (UCSB), the University of California, Berkeley, and the Naval Postgraduate School. This project addresses the complex challenge of managing and countering heterogeneous Autonomous Agents. These agents reason autonomously and make complex decisions using embedded artificial intelligence components and organic sensors. The core objective of the project is to develop strong theoretical foundations and novel algorithms to predict, shape, manipulate, and disrupt the actions of adversarial AI. By constructing a principled approach to adversarial experimentation, the research deliberately exposes flawed assumptions and critical vulnerabilities within intelligent systems, ensuring that future autonomous technologies are better understood and protected in complex, ambiguous environments

The team’s technical approach is organized into three main thrusts. First, they are pioneering "Targeted Learning," a new branch of machine learning designed to identify AI vulnerabilities and predict opponent actions even when relying on limited or incomplete data. Second, the project develops game theoretical models for deception, recognizing and exploiting vulnerabilities caused by limited information, imperfect machine learning models, or computing asymmetries between opponents. Finally, the researchers employ Atlatl, a simulation testbed for strategic decision making in environments that can be populated by AI agents, to continuously evaluate these algorithms. Using Atlatl, the academic team can rapidly test, simulate, and exposeweaknesses in AI decision-making within realistic, adversarial scenarios through a continuous feedback loop of gameplay and theoretical development.

Recently, the collaboration has achieved several major computational and theoretical milestones. The researchers advanced reinforcement learning techniques by proposing "opponent-informed" and "saddle-point" exploration methods. These advances allow AI systems to find secure, optimal policies under strict computing budgets without needing to exhaustively explore every possible game state. Furthermore, the team made critical breakthroughs in robust, data-driven decision-making by designing defense policies capable of thwarting attackers attempting to secretly deceive sensors or perturb system control channels. Additional work has focused on defining tools for the adversarial testing of AIs, using sensitivity analysis and weight-based diagnostics to evaluate system models and measure the impact of inputs on a system's state.

Project Lab: Networked Control Laboratory

Project Contact:  João Hespana