<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000"><div><style></style></div><div><div>In the first case, the set of predicted top-2 is overlapped with the gold answer modes.</div><div>In the second case, the set of predicted top-2 is a superset of the gold answer modes.</div><div><br></div><div>張光瑜</div><br></div><div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><b>寄件者: </b>"kysu" <kysu@iis.sinica.edu.tw><br><b>收件者: </b>"蔡仕竑" <doublebite@iis.sinica.edu.tw>, "范正忠" <jjfan@iis.sinica.edu.tw><br><b>副本: </b>"Most-ai Contest" <Most-ai-contest@iis.sinica.edu.tw><br><b>寄件備份: </b>2020 4 月 4 星期六 下午 9:54:42<br><b>主旨: </b>Re: [Most-ai-contest] New dispatcher module on latest training data<br></div><div><br></div><div data-marker="__QUOTED_TEXT__"><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d">Test set:</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> Top1 acc: 0.9842</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> Top2 acc: 1.0 </span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> Top2 coverage: 0.9421</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> Type acc: 0.9368</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d">>Top2 acc: one of the top2 predicted modes in golden answer modes</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d">>Top2 coverage: the ratio that the top2 predicted modes cover the golden modes</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d">What is the difference between “Top2 acc” and “Top2 coverage”? They should be the same in usual setting.</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> </span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d">KY</span></p><p class="MsoNormal"><span lang="EN-US" style="font-family:'calibri' , sans-serif;color:#1f497d"> </span></p><div><div style="border:none;border-top:solid #e1e1e1 1pt;padding:3pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:11pt;font-family:'calibri' , sans-serif">From:</span></b><span lang="EN-US" style="font-size:11pt;font-family:'calibri' , sans-serif"> most-ai-contest-bounces@iis.sinica.edu.tw [mailto:most-ai-contest-bounces@iis.sinica.edu.tw] <b>On Behalf Of </b></span><span style="font-size:11pt">蔡仕竑</span><span lang="EN-US" style="font-size:11pt;font-family:'calibri' , sans-serif"><br><b>Sent:</b> Saturday, April 4, 2020 9:04 PM<br><b>To:</b> </span><span style="font-size:11pt">范正忠</span><span lang="EN-US" style="font-size:11pt;font-family:'calibri' , sans-serif"> <jjfan@iis.sinica.edu.tw><br><b>Cc:</b> Most-ai Contest <Most-ai-contest@iis.sinica.edu.tw><br><b>Subject:</b> [Most-ai-contest] New dispatcher module on latest training data</span></p></div></div><p class="MsoNormal"><span lang="EN-US"> </span></p><div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">Dear all,</span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">The following link directs to the latest dispatcher module with its performance in the attachment.</span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"><a href="https://drive.google.com/file/d/1FsH_1gYMCH6KKkAK3152esL1qax8GIYm/view?usp=sharing" target="_blank" rel="nofollow noopener noreferrer">https://drive.google.com/file/d/1FsH_1gYMCH6KKkAK3152esL1qax8GIYm/view?usp=sharing</a></span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">We made some adjustment to its answer mode prediction </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">so that it won't output the unused module name "commonsense" as answer mode.</span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">Please find the attached ppt to modify the dispatcher code accordingly.</span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">Sincerely,</span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black">Shih-hong </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div><div><p class="MsoNormal"><span lang="EN-US" style="font-family:'arial' , sans-serif;color:black"> </span></p></div></div></div><br>_______________________________________________<br>Most-ai-contest mailing list<br>Most-ai-contest@iis.sinica.edu.tw<br>https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest<br></div></div></body></html>