"Applications of signal processing to image synthesis"

Dr. Pradeep Sen, University of New Mexico, Faculty Candidate

April 11th (Wednesday), 11:00am
Harold Frank Hall, Room 1132 (ECE Conference Room)

Image synthesis is the process of generating an image from a scene description that includes geometry, material properties, and camera/light positions. This is a central problem in many applications, ranging from rendering images for movies/videogames to generating realistic environments for training and tele-presence applications. The most powerful methods for photorealistic image synthesis are based on Monte Carlo (MC) algorithms, which simulate the full physics of light transport in a scene by estimating a series of multi-dimensional integrals using a set of random point samples. Although these algorithms can produce spectacular images, they are plagued by noise at low sampling rates and therefore require long computation times (as long as a day per image) to produce acceptable results. This has made them impractical for many applications and limited their use in real production environments. Thus, solving this issue has become one of the most important open problems in image synthesis and has been the subject of extensive research for almost 30 years.

In this talk, I present a new way to think about the source of Monte Carlo noise, and propose how to identify it in an image using a small number of computed samples. To do this, we treat the rendering system as a black box and calculate the statistical dependency between the outputs and the random parameter inputs using mutual information. I then show how we can use this information with an image-space, cross-bilateral filter to remove the MC noise but preserve important scene details. This process allows us to generate images in a few minutes that are comparable to those that took hundreds of times longer to render. Furthermore, our algorithm is fully general and works for a wide range of Monte Carlo effects, including depth of field, area light sources, motion blur, and path tracing. This work opens the door to a new set of algorithms that make Monte Carlo rendering feasible for more applications.

I will also discuss some of my lab’s other work that applies ideas from signal processing and combines them with computation and optical system design to tackle problems in computational photography, image processing, and human-computer interaction. These include a new design for a high-dynamic range video camera that can capture considerably more realistic images, and a novel patch-based optimization framework that unifies previously-unrelated image processing tasks such as image stitching, image hole filling, and image morphing.

About Dr. Pradeep Sen:

Dr. Pradeep Sen is an Assistant Professor in the Department of Electrical and Computer Engineering at the University of New Mexico. He received his B.S. in Computer and Electrical Engineering from Purdue University in 1996 and his M.S. in Electrical Engineering from Stanford University in 1998 in the area of electron-beam lithography. After two years at a profitable startup company which he co-founded, he joined the Stanford Graphics Lab where he received his Ph.D. in Electrical Engineering in June 2006, advised by Dr. Pat Hanrahan.

He joined the faculty at UNM in the Fall of 2006, where he founded the UNM Advanced Graphics Lab. His core research combines signal processing theory with computation and optics/light-transport analysis to address problems in computer graphics, photography, and computational image processing. He is the co-author of five ACM SIGGRAPH papers and has been awarded more than $1.7 million in research funding, including an NSF CAREER award to study the application of sparse reconstruction algorithms to computer graphics and imaging. He received two best-paper awards at the Graphics Hardware conference in 2002 and 2004, and the Lawton-Ellis Award from the ECE department at UNM for outstanding research contributions in 2009. Dr. Sen has also started a successful educational program at UNM, where his videogame development program is now ranked by the Princeton Review as one of the top 10 undergraduate programs in North America.

Hosted by: Professor U. Madhow