Jul 19 (Wed) @ 2:00pm: "Bayesian In-Memory Computing," Dr. Damien Querlioz, CNRS – U. Paris-Saclay

Abstract

Emerging memory devices provide appealing features and outstanding energy efficiency, but they suffer from high variability and unpredictability, making them functionally analogous to random variables. In machine learning, Bayesian approaches are designed to operate with random variables. In this talk, we show that they can be an excellent way to exploit resistive memory devices without suffering from their drawbacks.

We first introduce a Bayesian machine that uses near-memory and stochastic computing to perform Bayesian inference at a very low energy cost [1]. It permits fully-explainable decision-making in situations with incomplete information, maximally incorporating all available evidence, assumptions, and prior knowledge. The machine, fabricated in a hybrid CMOS-memristor process, associates 2,048 hafnium-oxide memristors and 30,080 MOSFETs and can recognize gestures using thousands of times less energy than a microcontroller unit.

Next, we show how memristors can be used to implement Bayesian neural networks. This unique class of neural networks regards synapses and neurons as random variables and is capable of quantifying uncertainty in predictions. The random nature of Bayesian synapses can be matched to the intrinsic random nature of memristors. We demonstrate experimentally a memristor-based Bayesian neural network capable of detecting arrhythmia with uncertainty quantification [2].

We finally introduce a Bayesian technique specifically designed to leverage the random nature of memristors for learning. This technique enables the implementation of a Markov Chain Monte Carlo process. This approach yielded excellent results experimentally: an array consisting of 16,384 hafnium-oxide memristors learned to identify cancerous tissue images, delivering accuracy on par with conventional software approaches [3].

References

[1] K.-E. Harabi, T. Hirtzlin, C. Turck, E. Vianello, R. Laurent, J. Droulez, P. Bessière, J.-M. Portal, M. Bocquet, D. Querlioz, "A memristor-based Bayesian machine", Nature Electronics 6, 52, 2023

[2] D. Bonnet, T. Hirtzlin, A. Majumdar, T. Dalgaty, E. Esmanhotto, V. Meli, N. Castellani, S. Martin, J.-F. Nodin, G. Bourgeois, J.-M. Portal, D. Querlioz, E. Vianello, (2023). “Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks”, preprint available on DOI: 10.21203/rs.3.rs-2458251/v1.

[3] T. Dalgaty, N. Castellani, C. Turck, K.-E. Harabi, D. Querlioz, E. Vianello, "In situ learning using intrinsic memristor variability via Markov chain Monte Carlo sampling", Nature Electronics, Vol. 4, p. 151, 2021

Bio

Dr. Damien Querlioz is a CNRS Researcher at the Centre de Nanosciences et de Nanotechnologies of Université Paris-Saclay and CNRS. His research focuses on novel usages of emerging non-volatile memory and other nanodevices, in particular relying on inspirations from biology and machine learning. He received his predoctoral education at Ecole Normale Supérieure, Paris and his PhD from Université Paris-Sud in 2009. Before his appointment at CNRS, he was a Postdoctoral Scholar at Stanford University and at the Commissariat à l’Energie Atomique. Damien Querlioz is the coordinator of the interdisciplinary INTEGNANO research group, with colleagues working on all aspects of nanodevice physics and technology, from materials to systems. He is a member of the bureau of the French Biocomp research network. He has co-authored one book, nine book chapters, more than 150 journal articles, and conference proceedings, and given more than 80 invited talks at national and international workshops and conferences. In 2016, he was the recipient of an ERC Starting Grant to develop the concept of natively intelligent memory. In 2017, he received the CNRS Bronze medal. He has also been a co-recipient of the 2017 IEEE Guillemin-Cauer Best Paper Award and of the 2018 IEEE Biomedical Circuits and Systems Best Paper Award.

Hosted by: ECE Prof. Kerem Camsari

Submitted by: Kerem Camsari <camsari@ucsb.edu>