Jul 25th (Fri) @ 2:00pm: "Towards Efficient Machine Learning: Optimization of Computation, Architecture, and Data," Yu Wang, ECE PhD Defense
Location: https://ucsb.zoom.us/j/81591204159?pwd=XoDeFw7DeAMqv7vwJXMn83oxbqA0iU.1
Research Area: Computer Engineering
Research Keywords: Hardware-algorithm Co-design, Semi-Supervised Learning, Bayesian Optimization
Abstract
The rapid advancement of machine learning has enabled broad applications across diverse domains, yet its practical deployment remains constrained by computation, memory, and the available high-quality data. This thesis systematically addresses these challenges through innovations in algorithm design, hardware acceleration, and data-efficient learning.
First, to mitigate computational bottlenecks in probabilistic inference, the thesis introduces a hardware-algorithm co-design for accelerating Hamiltonian Monte Carlo (HMC) on FPGAs. By exploiting three levels of algorithmic parallelism, the proposed hardware architecture maximizes throughput by fully utilizing the computational capacity of FPGAs. Additionally, the integration of reservoir sampling further improves memory efficiency, which collectively yield up to a 50x speedup and nearly 200x improvement in energy efficiency over conventional software implementations.
Second, the thesis develops a neural architecture search (NAS) methodology for normalizing flows (NF), where manual design of optimal architecture is both computationally intensive and analytically challenging. The proposed framework, AutoNF, introduces a continuous relaxation of the discrete search space, converting the combinatorial optimization into a differentiable process that still admits the original discrete optimum. This enables efficient discovery of high-performing, resource-efficient NF architectures.
Third, to address the scarcity of labeled data in learning dynamical systems, the thesis proposes TS-NODE, a semi-supervised framework based on Neural Ordinary Differential Equations (Neural ODEs). Through a novel teacher-student paradigm with feedback, TS-NODE generates high-quality pseudo-rollouts that expand the state-space coverage, significantly improving model accuracy and generalization under limited supervision.
Finally, the thesis presents ADO-LLM, a hybrid framework that integrates Bayesian Optimization (BO) with Large Language Models (LLMs) for analog circuit design. This complementary dual-agent system combines the domain knowledge in LLMs with the exploratory power of BO, enabling guided and sample-efficient search in complex design spaces. The approach achieves substantial gains in identifying circuit designs with performance specifications.
Collectively, this thesis demonstrates how targeted optimization across computational, architectural, and data dimensions can significantly enhance the efficiency, scalability, and applicability of machine learning systems, paving the way for their broader adoption in real-world technological applications.
Bio
Yu Wang is a Ph.D. candidate in Computer Engineering at the University of California, Santa Barbara, advised by Professor Peng Li. He received his M.S. from Texas A&M University in 2019 and his B.S. from Fudan University.
Yu’s research focuses on the intersection of advanced machine learning and hardware system design, with an emphasis on Bayesian Optimization, Semi-Supervised Learning, and advanced ML methods for electronic design automation (EDA). He has done two internships at Qualcomm and Intuit, where he focused on implementing novel numerical system and hallucination mitigation of LLMs, respectively.
His works have been recognized at leading machine learning and circuit design venues, including AAAI, ICML, TMLR, NAACL and ICCAD. He also received the Best Paper Award at ASAP 2020.
Hosted By: ECE Professor Peng Li
Submitted By: Yu Wang <yu95@ucsb.edu>
