Carlo Tomasi (hosted by CIS) – “Motion Boundary Detection and Neural Architectures for Image-to-Image Estimation Problems”
“Motion Boundary Detection and Neural Architectures for Image-to-Image Estimation Problems”
Abstract: In an idealized imaging model, the optical flow is a piecewise continuous vector field that describes the motion of every point between video frames. The curves of spatial discontinuity of the optical flow field are called motion boundaries. Detecting these curves accurately helps in a variety of applications including video segmentation or editing and action recognition. While methods for computing optical flow from two consecutive video frames have rapidly improved over the last few years, their performance is still lagging along the motion boundaries themselves, making these difficult to detect. The reasons for this difficulty stem mainly from the fact that flow estimators tend to produce smooth outputs, and therefore blur boundaries. In addition, motion boundaries have measure zero in the image plane and are defined differentially, while imagery is discrete in both space and time.
I will present a method for motion boundary detection based on neural networks that combines several known techniques for pre-and post-processing the input frame pair and pushes the state of the art on the problem. Our method also includes a novel, simple modification of the standard encoder-decoder architecture that is used for this type of problems. This modification incurs no additional cost, as it merely inverts the flow of information in the decoder, and yet yields consistent improvements on a variety of image-toimage estimation problems. This research is joint work with Hannah Kim.
Bio: Carlo Tomasi received his PhD in Computer Science from Carnegie Mellon University in 1991. He was assistant professor at Cornell and Stanford, and is currently the Iris EinheuserProfessor of Computer Science at Duke University. His research spans computer vision from visual motion estimation, image retrieval, and activity recognition to shape reconstruction, stereo vision, texture analysis, and medical imaging. His papers have been cited more than 44,000 times according to Google Scholar, with more than half of those citations for his top three publications. He won two Helmholtz prizes awarded by the International Conference on Computer Vision for papers that have had significant long-term impact on computer vision. He is an ACM Fellow, holds eleven patents, and has been principal investigator or co-investigator on more than 40 research grants.