[Most-ai-contest] New dispatcher module on latest training data

[張光瑜]

In the first case, the set of predicted top-2 is overlapped with the gold answer modes. 
In the second case, the set of predicted top-2 is a superset of the gold answer modes. 

張光瑜 



寄件者: "kysu" <kysu at iis.sinica.edu.tw> 
收件者: "蔡仕竑" <doublebite at iis.sinica.edu.tw>, "范正忠" <jjfan at iis.sinica.edu.tw> 
副本: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw> 
寄件備份: 2020 4 月 4 星期六 下午 9:54:42 
主旨: Re: [Most-ai-contest] New dispatcher module on latest training data 



Test set: 

Top1 acc: 0.9842 

Top2 acc: 1.0 

Top2 coverage: 0.9421 

Type acc: 0.9368 

>Top2 acc: one of the top2 predicted modes in golden answer modes 

>Top2 coverage: the ratio that the top2 predicted modes cover the golden modes 



What is the difference between “Top2 acc” and “Top2 coverage”? They should be the same in usual setting. 



KY 




From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 蔡仕竑 
Sent: Saturday, April 4, 2020 9:04 PM 
To: 范正忠 <jjfan at iis.sinica.edu.tw> 
Cc: Most-ai Contest <Most-ai-contest at iis.sinica.edu.tw> 
Subject: [Most-ai-contest] New dispatcher module on latest training data 





Dear all, 





The following link directs to the latest dispatcher module with its performance in the attachment. 


[ https://drive.google.com/file/d/1FsH_1gYMCH6KKkAK3152esL1qax8GIYm/view?usp=sharing | https://drive.google.com/file/d/1FsH_1gYMCH6KKkAK3152esL1qax8GIYm/view?usp=sharing ] 





We made some adjustment to its answer mode prediction 


so that it won't output the unused module name "commonsense" as answer mode. 


Please find the attached ppt to modify the dispatcher code accordingly. 





Sincerely, 


Shih-hong 










_______________________________________________ 
Most-ai-contest mailing list 
Most-ai-contest at iis.sinica.edu.tw 
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200404/ff3c647b/attachment-0001.html>