中央研究院 資訊科學研究所

活動訊息

友善列印

列印可使用瀏覽器提供的(Ctrl+P)功能

Deep Learning for Video Frame Interpolation and Object Co-segmentation

:::

Deep Learning for Video Frame Interpolation and Object Co-segmentation

  • 講者林彥宇 博士 (中央研究院資訊科技研究創新中心)
    邀請人:陳祝嵩
  • 時間2019-05-02 (Thu.) 10:30 ~ 12:00
  • 地點資訊所新館106演講廳
摘要

In this talk, I will present our recent research results on video frame interpolation and object co-segmentation.

For video frame interpolation, it predicts intermediateframes to produce videos with higher frame rates and smoothview transitions given two consecutive frames as inputs. Weobserve that synthesized frames are more reliable if theycan be used to reconstruct the input frames with high quality.Based on this observation, we introduce a new loss term, thecycle consistency loss. The cycle consistency loss can betterutilize the training data to not only enhance the interpolationresults, but also maintain the performance better withless training data. It can be integrated into any frame interpolationnetwork and trained in an end-to-end manner. Both qualitative andquantitative experiments demonstrate that our model outperformsthe state-of-the-art methods.

For object co-segmentation, it aims to segment the commonobjects in images. We present a CNN-basedmethod that is unsupervised and end-to-endtrainable to better solve this task. Our methodis unsupervised in the sense that it does not requireany training data in the form of object masksbut merely a set of images jointly covering objectsof a specific class. Our method comprisestwo collaborative CNN modules, a feature extractorand a co-attention map generator. The formermodule extracts the features of the estimated objectsand backgrounds. The latter moduleis learned to generate co-attention maps by whichthe estimated figure-ground segmentation can betterfit the former module. Experiments show thatour method achieves superior results, even outperformingthe state-of-the-art, supervised methods.

BIO

Yen-Yu Lin received the B.B.A. degree in Information Management, and the M.S. and Ph.D. degrees in Computer Science and Information Engineering from National Taiwan University, Taipei, Taiwan, in 2001, 2003, and 2010, respectively. He is currently an Associate Research Fellow with the Research Center for Information Technology Innovation, Academia Sinica, Taipei, Taiwan. His research interests include computer vision, machine learning, and artificial intelligence.

URL:  https://www.citi.sinica.edu.tw/pages/yylin/

Lab URL: http://cvlab.citi.sinica.edu.tw/