中央研究院 資訊科學研究所

活動訊息

友善列印

學術演講

Today!

利用反事實推理改善視覺-語言資料不足的問題

  • 傅子睿 先生 (加州大學聖塔芭芭拉分校)
    邀請人:馬偉雲
  • 2020-10-20 (Tue.) 10:30 – 12:30
  • 資訊所新館106演講廳
摘要

Tsu-Jui is currently a Ph.D. student at UCSB CS, advised by William Wang. His research interest lies in natural

language processing (NLP) and computer vision (CV). Primarily, he works on language grounding and interacting

with environments using language. Besides, he is also interested in information extraction and video analysis.

His research goal is to bridge the gap between vision and language via the AI system.

BIO

 

Vision-and-Language tasks, which require the agent to interact between the multi-modal source (ex: vision and language), are more practical and closed to people's daily scenarios.

Data scarcity is a significant issue for vision-and-language as it is challenging to collect large-scale multi-modal examples.  However, people still accomplish those tasks

even when presented with an unfamiliar situation.  Such ability results from counterfactual thinking and the ability to think about alternatives to events that have happened already.

We incorporate counterfactual thinking into the language-based image editing (LBIE) task and the vision-and-language navigation (VLN) task.

We show how to apply the counterfactual reasoning under data scarcity and make it more effective for vision-and-language tasks.