"Machine Learning meets Information Theory: from Analyzing Data to Discovering Communication Algorithms"

Hyeji Kim, Postdoctoral Research Associate, University of Illinois at Urbana-Champaign

March 13th (Tuesday), 9:00am
Harold Frank Hall (HFH), Rm 4164

My research is at the intersection of information theory and machine learning, where exciting recent advances are happening. I use tools from one discipline to solve central problems in the other. In bringing information theory to machine learning, we apply information theoretic tools to discover associations from large data sets. Discovering associations between two variables is of fundamental and scientific interest. However, the challenge is that in many scenarios, the associations are hidden or potential, and existing correlation measures such as Pearson correlation and maximal correlation fail to discover such potential correlations. We introduce hypercontractivity coefficient, i.e., the rate of information bottleneck, as a measure of potential correlations. We prove hypercontractivity superiority in capturing potential correlations, compared to existing correlation measures. We demonstrate its practical usefulness via numerical experiments on social and biological data.

In bringing machine learning to information theory, we apply deep learning to automate the discovery of communication algorithms. A critical aspect of reliable communication involves the design of robust communication schemes. However, the progress has been sporadic, as it is driven by individual human ingenuity. Deep learning is fast emerging as capable of learning sophisticated algorithms from observed data alone and has been remarkably successful in a variety of human endeavors (e.g. chess/go). We use neural networks to design new nonlinear codes that represent a major progress in the long standing open problem of communicating reliably over various standard channels.

About Hyeji Kim:

Photo of Hyeji Kim Hyeji Kim is a postdoctoral research associate with the Coordinated Science Laboratory at University of Illinois at Urbana-Champaign. She received her Ph.D. and M.S. degrees in Electrical Engineering from Stanford University in 2016 and 2013, respectively, and her B.S. degree with honors in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST) in 2011. Her research interests include information theory, machine learning, and wireless communications. She is a recipient of Stanford Graduate Fellowship and participated in the Rising Stars in EECS Workshop in 2015.

Hosted by: BS Manjunath