PhD Defense: "Distinctive and Efficient Local Features for Real-Time Mobile Applications"

Xin Yang

February 21st (Thursday), 11:30am
Elings Hall, Rm 1601

Local features are widely used in many computer vision tasks such as marker-less augmented reality (AR), object recognition and tracking. The distinctiveness and high efficiency of a local feature are critical to the user experience of real-world applications. However, existing local feature algorithms are either too compute-insensitive to achieve real-time performance, especially when running on low power mobile devices, or not sufficiently distinctive to identify correct matches from a large database.

This dissertation focuses on designing distinctive and efficient local features that can run in real-time on mobile devices such as smartphones and tablets. Typically, a local feature extraction pipeline includes two components: 1) interest point detector which identifies a set of salient points in an image, and 2) point descriptor which represents each detected point using a feature vector. For the interest point detection, this dissertation focuses on accelerating SURF point detector by solving two mismatches between the SURF algorithm and the mobile hardware that cause substantial slow-down of the detection process. For the point description, this dissertation presents a highly efficient and distinctive binary descriptor, called Local Difference Binary (LDB). LDB directly computes a binary string for an image patch using simple intensity and gradient difference tests on pair-wise grid cells within the patch. A multiple gridding strategy is applied to capture the distinct patterns of the patch at different spatial granularities. To further enhance the distinctiveness of LDB, a learning-based framework is proposed. Extensive experiments on real data demonstrate that the proposed point detector and descriptors achieve high distinctiveness and efficiency on mobile devices.

About Xin Yang:

Xin Yang is a graduate student working with Prof. Tim Cheng at the Learning-Based Multimedia Lab since September 2008. She received her MS in 2008 from University of Science and Technology of China and BS from Huazhong University of Science and Technology in 2005. Her research interests include: 1) mobile computer vision and its application to mobile augmented reality, and 2) large-scale image retrieval. She has spent time as a research intern during Summers of 2008 and 2009 at VIMA Technologies, Summer of 2010 and 2011 at FXPAL.