Feb 19 (Thu) @ 1:00pm: "Designing and Enabling Temporal Architectures for Neural Networks," Rhys Gretsch, ECE PhD Defense

Physical Location: Henley Hall (HH), Room 1010 (Auditorium)
Zoom Link: https://www.google.com/url?q=https://ucsb.zoom.us/j/6730048923
Research Area: Computer Engineering
Research Keywords: Computer Architecture, Bio inspired computing

Abstract

The time in which an event occurs can carry meaningful information when considering its relation to other events. Leveraging this information for direct computation provides an alternative to the current digital paradigm, allowing for a simpler interaction with the physical world. Relationships between the temporal response created by any physical process can be directly computed upon using the same hardware substrate that supports digital logic without expensive conversions to binary representations. This offers the potential for energy efficient computation, but requires appropriate applications and careful hardware organization.

This dissertation presents techniques that allow data to remain in the time domain while performing neural network operations. A general framework of temporal arithmetic is enabled through a negative log transformation with delay-based approximations. Fully temporal large scale, programmable architectures can leverage this framework through the use of hardware recurrence and memory devices that capture the temporal relationship between two signals. Neural network inference can be fully supported by these architectures, and a detailed analysis of the energy-accuracy tradeoff introduced by the architectural decisions is presented. Finally this dissertation explores Zeroth-Order optimization, a technique that can be used to improve the temporal neural networks, and presents a fully digital architecture for energy efficient transformer finetuning.

Bio 

Rhys is a 5th year PhD candidate in the ECE department advised by Timothy Sherwood. He received his BS in computer engineering from North Carolina State University in 2021. His research aims to leverage biologically inspired techniques to enable energy efficient edge intelligence. 

Hosted By: CS professor Timothy Sherwood (committee chair)

Submitted By: Rhys Gretsch <rhys@ucsb.edu>