PhD Defense: "Modeling Eye Tracking Data with Application to Object Detection"

Karthikeyan Shanmuga Vadivel

August 15th (Friday), 3:00pm
Harold Frank Hall, Room 4164

This research focuses on enhancing computer vision algorithms using eye tracking and visual saliency. Recent advances in eye tracking device technology have enabled large scale collection of eye tracking data, without affecting viewer experience. As eye tracking data is biased towards high level image and video semantics, it provides a valuable prior for object detection in images and object extraction in videos. We specifically explore the following problems in the thesis: 1) eye tracking and saliency enhanced object detection, 2) eye tracking assisted object extraction in videos, and 3) role of object co-occurrence and camera focus in visual attention modeling.

Since human attention is biased towards faces and text, in the first work we propose an approach to isolate face and text regions in images by analyzing eye tracking data from multiple subjects. Eye tracking data is clustered and region labels are predicted using a Markov random field model. In the second work, we study object extraction in videos using eye tracking prior. We propose an algorithm to extract dominant visual tracks in eye tracking data from multiple subjects by solving a linear assignment problem. Visual tracks localize object search and we propose a novel mixed graph association framework, inferred by binary integer linear programming. In the final work, we address the problem of predicting where people look in images. We specifically explore the importance of scene context in the form of object co-occurrence and camera focus. The proposed model extracts low-, mid- and high-level and scene context features and uses a regression framework to predict visual attention map. In all the above cases, extensive experimental results show that the proposed methods outperform current state-of-the-art.

About Karthikeyan Shanmuga Vadivel:

Karthikeyan was born in Chennai, India. He received his Bachelors in Technology in Electrical Engineering from Indian Institute of Technology Madras in 2008 and Masters in Electrical and Computer Engineering from UCSB in 2009. Subsequently, he joined Prof. Manjunath’s research group and worked on a variety of inter-disciplinary problems, focusing on the application of computer vision to psychology and biology. His research has been published in top-tier conferences such as ICCV, CVPR and ICIP. He is a recipient ECE Dissertation Fellowship in Spring 2013. During his time at UCSB he has interned in Samsung Research Lab, Dallas and Mayachitra Inc., Santa Barbara. His research interests include eye tracking and saliency based image/video analysis, multiple object tracking, activity recognition and subspace analysis.

Hosted by: Professor B.S. Manjunath