<html><body><div id="zimbraEditorContainer" style="font-family: arial, helvetica, sans-serif; font-size: 12pt; color: #000000" class="3"><div><style type="text/css" scoped="">
<!--table
{mso-displayed-decimal-separator:"\.";
mso-displayed-thousand-separator:"\,";}
@page
{margin:.75in .7in .75in .7in;
mso-header-margin:.3in;
mso-footer-margin:.3in;}
.zimbra1
{color:black;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:新細明體, serif;
mso-font-charset:136;}
tr
{mso-height-source:auto;
mso-ruby-visibility:none;}
col
{mso-width-source:auto;
mso-ruby-visibility:none;}
br
{mso-data-placement:same-cell;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:11.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:新細明體, serif;
mso-font-charset:136;
mso-number-format:General;
text-align:general;
vertical-align:bottom;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
.zimbra2
{vertical-align:top;
white-space:normal;}
.zimbra3
{text-align:left;
vertical-align:top;
white-space:normal;}
ruby
{ruby-align:left;}
rt
{color:black;
font-size:9.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:新細明體, serif;
mso-font-charset:136;
mso-char-type:none;
display:none;}
-->
</style></div><div><style></style></div><div></div><div>Dear all,</div><div><br data-mce-bogus="1"></div><div>A quick look at Date-Duration on 'test' & 'dev' data set and the results as follows.</div><div><br data-mce-bogus="1"></div><div>You can see </div><div>1. some answers are ranked Top-2 or Top-3 </div><div>2. Some need math. operations</div><div>3. Some need to format transform (西元 -> 民國) </div><div>4. <span style="font-size: 12pt;" data-mce-style="font-size: 12pt;"><span style="font-size: 12pt;" data-mce-style="font-size: 12pt;"> <!--StartFragment--><span style="color: rgb(0, 0, 0); font-family: 新細明體, serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; float: none; display: inline !important;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; float: none; display: inline !important;">D267Q07 -> I think it should be correct since only <span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">"戰國末年" occurs in the passage. (golden answer : 戰國時期)</span><!--EndFragment--></span></span></span><!--EndFragment--><div style="clear: both;" data-mce-style="clear: both;">5. <!--StartFragment--><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">D303Q05 -> <!--StartFragment--><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">"初冬" should be OK since only <!--StartFragment-->"初冬" occurs in the passage (golden answer: 冬)<!--EndFragment--></span></span></span></span><div style="clear: both;" data-mce-style="clear: both;"><br></div><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: nowrap; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;" data-mce-style="color: #000000; font-family: 新細明體, serif; font-size: 14.6667px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: -webkit-left; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"></span><!--EndFragment--></span></span><!--EndFragment--><div style="clear: both;" data-mce-style="clear: both;">jjfan</div></div></div><div><br data-mce-bogus="1"></div><div> <table border="1" cellpadding="0" cellspacing="0" width="1117" style="border-collapse: collapse; width: 838pt; height: 234px;" data-mce-style="border-collapse: collapse; width: 838pt;"><!--StartFragment--> <colgroup><col width="180" style="mso-width-source: userset; mso-width-alt: 6582; width: 135pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 6582; width: 135pt;"> <col width="82" style="mso-width-source: userset; mso-width-alt: 2998; width: 62pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 2998; width: 62pt;"> <col width="77" style="mso-width-source: userset; mso-width-alt: 2816; width: 58pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 2816; width: 58pt;"> <col width="79" style="mso-width-source: userset; mso-width-alt: 2889; width: 59pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 2889; width: 59pt;"> <col width="91" style="mso-width-source: userset; mso-width-alt: 3328; width: 68pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 3328; width: 68pt;"> <col width="96" style="mso-width-source: userset; mso-width-alt: 3510; width: 72pt;" data-mce-style="mso-width-source: userset; mso-width-alt: 3510; width: 72pt;"> <col width="64" span="8" style="width: 48pt;" data-mce-style="width: 48pt;"> </colgroup><tbody><tr height="21" style="height: 15px;" data-mce-style="height: 15.75pt;"><td height="21" align="left" width="180" style="height: 15px; width: 10px;">Dev</td><td align="left" width="82" style="width: 203px; height: 15px;">D061Q04</td><td align="left" width="77" style="width: 113px; height: 15px;">D073Q05</td><td align="left" width="79" style="width: 78px; height: 15px;">D097Q06</td><td align="left" width="91" style="width: 89px; height: 15px;">D103Q04</td><td align="left" width="96" style="width: 95px; height: 15px;">D241Q04</td><td align="left" width="64" style="width: 66px; height: 15px;">D247Q08</td><td align="left" width="64" style="width: 66px; height: 15px;">D307Q03</td><td width="64" style="width: 66px; height: 15px;"></td><td width="64" style="width: 66px; height: 15px;"></td><td width="64" style="width: 66px; height: 15px;"></td><td width="64" style="width: 66px; height: 15px;"></td><td width="64" style="width: 66px; height: 15px;"></td><td width="64" style="width: 66px; height: 15px;"></td></tr><tr height="126" style="height: 94px;" data-mce-style="height: 94.5pt;"><td height="126" class="zimbra3" width="180" style="height: 94px; width: 10px;"></td><td class="zimbra3" width="82" style="width: 203px; height: 94px;">Top-2</td><td class="zimbra3" width="77" style="width: 113px; height: 94px;">Top-2</td><td class="zimbra3" width="79" style="width: 78px; height: 94px;"></td><td class="zimbra3" width="91" style="width: 89px; height: 94px;">Top-2</td><td class="zimbra3" width="96" style="width: 95px; height: 94px;">Top1 相減 "1506年至1626年"</td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;">Dispatcher Error : Single-Span -> Date-Duration</td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td><td class="zimbra3" width="64" style="width: 66px; height: 94px;"></td></tr><tr height="21" style="height: 15px;" data-mce-style="height: 15.75pt;"><td height="21" align="left" style="height: 15px; width: 10px;">Test</td><td align="left" style="height: 15px; width: 203px;">D033Q11</td><td align="left" style="height: 15px; width: 113px;">D069Q04</td><td align="left" style="height: 15px; width: 78px;">D069Q05</td><td align="left" style="height: 15px; width: 89px;">D069Q06</td><td align="left" style="height: 15px; width: 95px;">D069Q07</td><td align="left" style="height: 15px; width: 66px;">D087Q02</td><td align="left" style="height: 15px; width: 66px;">D105Q02</td><td align="left" style="height: 15px; width: 66px;">D105Q04</td><td align="left" style="height: 15px; width: 66px;">D117Q01</td><td align="left" style="height: 15px; width: 66px;">D117Q02</td><td align="left" style="height: 15px; width: 66px;">D243Q07</td><td align="left" style="height: 15px; width: 66px;">D267Q07</td><td align="left" style="height: 15px; width: 66px;">D303Q05</td></tr><tr height="147" style="height: 110px;" data-mce-style="height: 110.25pt;"><td height="147" class="zimbra2" width="180" style="height: 110px; width: 10px;"></td><td class="zimbra2" align="left" width="82" style="width: 203px; height: 110px;">Top-2</td><td class="zimbra2" width="77" style="width: 113px; height: 110px;"></td><td class="zimbra2" width="79" style="width: 78px; height: 110px;"></td><td class="zimbra2" align="left" width="91" style="width: 89px; height: 110px;">Top1 相減<span style="mso-spacerun: yes;" data-mce-style="mso-spacerun: yes;"> </span>"1981年至纽约大学就读电影制作研究所，于1984年"</td><td class="zimbra2" align="left" width="96" style="width: 95px; height: 110px;">Top1 - Top2</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top1 前頭加上 2019 年</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top1 1995年, output formater 沒有轉成民國</td><td class="zimbra2" width="64" style="width: 66px; height: 110px;"></td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top-2</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Single-Span Top1</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top2 相減 "1506年4月18日，完成于1626年11月18日"</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top1 "戰國末年"</td><td class="zimbra2" align="left" width="64" style="width: 66px; height: 110px;">Top3 "初冬"</td></tr><!--EndFragment--></tbody></table><div style="clear: both;" data-mce-style="clear: both;"><br></div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><b>From: </b>"范正忠" <jjfan@iis.sinica.edu.tw><br><b>To: </b>"Simonc" <simonc@iis.sinica.edu.tw><br><b>Cc: </b>"Most-ai Contest" <Most-ai-contest@iis.sinica.edu.tw><br><b>Sent: </b>Friday, April 3, 2020 4:45:39 PM<br><b>Subject: </b>[Most-ai-contest] The performance of the last version FGC QA system<br></div><div><br></div><div data-marker="__QUOTED_TEXT__"><div style="font-family:'arial' , 'helvetica' , sans-serif;font-size:12pt;color:#000000"><div></div><div>Dear all,</div><br><div>Enclosed please find the current performance of the FGC QA system.</div><br><div> <table border="1" cellpadding="0" cellspacing="0" width="925" style="border-collapse:collapse;width:924px"><colgroup><col width="64" style="width:48pt"><col width="180" style="width:135pt"><col width="82" style="width:62pt"><col width="77" style="width:58pt"><col width="79" style="width:59pt"><col width="91" style="width:68pt"><col width="96" style="width:72pt"><col width="64" span="4" style="width:48pt"></colgroup><tbody><tr style="height:15.75pt"><td height="21" class="zimbra4" width="64" style="height:15.75pt;width:69px"></td><td class="zimbra3" width="180" style="width:187px"></td><td align="left" width="82" style="width:86px">Single-Span</td><td align="left" width="77" style="width:81px">Kuo</td><td align="left" width="91" style="width:95px">Multi-Span</td><td align="left" width="96" style="width:101px">Date-Duration</td><td class="zimbra2" width="64" style="width:99px"></td><td align="left" width="64" style="width:68px">Train</td><td align="left" width="64" style="width:69px">Dev</td><td align="left" width="64" style="width:69px">Test</td></tr><tr style="height:47.25pt"><td height="63" class="zimbra5" style="height:47.25pt;width:69px">4月2日</td><td class="zimbra6" width="180" style="width:187px">ENSEMBLEModule: 12 models (new)</td><td class="zimbra6" width="82" style="width:86px">train: 0.890<br>dev: 0.801<br>test: 0.750</td><td class="zimbra6" width="77" style="width:81px">train: 0.899<br>dev: 0.699<br>test: 0.741</td><td class="zimbra3" width="91" style="width:95px">train: 0.493<br>dev: 0.476<br>test: 0.350</td><td class="zimbra3" width="96" style="width:101px">train: 0.589<br>dev: 0.720<br>test: 0.581</td><td class="zimbra2" style="width:99px"></td><td class="zimbra7" style="width:68px">0.786</td><td class="zimbra7" style="width:69px">0.709</td><td class="zimbra7" style="width:69px">0.658</td></tr><tr style="height:63pt"><td height="84" class="zimbra4" style="height:63pt;width:69px">4月3日</td><td class="zimbra3" width="180" style="width:187px">ENSEMBLEModule: 12 models (new)<br>MSPE_v18_branchy27<br>date_duration_module_4</td><td class="zimbra3" width="82" style="width:86px">train: 0.890<br>dev: 0.801<br>test: 0.750</td><td class="zimbra3" width="77" style="width:81px">train: 0.899<br>dev: 0.699<br>test: 0.741</td><td class="zimbra3" width="91" style="width:95px">train: 0.548<br>dev: 0.667<br>test: 0.450</td><td class="zimbra3" width="96" style="width:101px">train: 0.589<br>dev: 0.720<br>test: 0.581</td><td class="zimbra2" style="width:99px"></td><td class="zimbra2" style="width:68px">0.791</td><td class="zimbra2" style="width:69px">0.725</td><td class="zimbra2" style="width:69px">0.668</td></tr></tbody></table><br><div>Please <span style="color:#000000;font-family:'helvetica neue' , 'helvetica' , 'arial' , 'liberation sans' , sans-serif;font-size:14.16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none">郭家銍 help to check Date-Duration ver.4. </span></div><div><span style="color:#000000;font-family:'helvetica neue' , 'helvetica' , 'arial' , 'liberation sans' , sans-serif;font-size:14.16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none"><br></span></div><div><span style="color:#000000;font-family:'helvetica neue' , 'helvetica' , 'arial' , 'liberation sans' , sans-serif;font-size:14.16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none">Best,</span></div><div><span style="color:#000000;font-family:'helvetica neue' , 'helvetica' , 'arial' , 'liberation sans' , sans-serif;font-size:14.16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none">jjfan</span></div><br></div><br><hr id="zwchr"><div><b>From: </b>"范正忠" <jjfan@iis.sinica.edu.tw><br><b>To: </b>"Simonc" <simonc@iis.sinica.edu.tw><br><b>Cc: </b>"Most-ai Contest" <Most-ai-contest@iis.sinica.edu.tw><br><b>Sent: </b>Thursday, April 2, 2020 3:56:36 PM<br><b>Subject: </b>Re: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate<br></div><br><div><div style="font-family:'arial' , 'helvetica' , sans-serif;font-size:12pt;color:#000000"><div>Dear all,</div><br><div>Enclosed please find the performance of the last code:</div><br><div><div>ENSEMBLEModule: 12 models (new), single_span_multi_hop_v2_1</div><div>Multi-Spans: MSPE_v18_branchy20</div><div>date_duration_module_3</div><div>arithmetic_module_3</div><div>kinship_module6</div><br><div>Overall:</div><div>train -> total: 882, correct: 668, accuracy: 0.757<br></div><div>dev -> total: 247, correct: 161, accuracy: 0.652</div><div>test -> total: 193, correct: 126, accuracy: 0.653</div><br><div>Single-Span:</div><div>train -> portion:0.619, count:546, errors: 86, accuracy: 0.842</div><div> ensemble only -> portion:0.619, count:546, errors: 37, accuracy: <span style="color:rgb( 255 , 0 , 0 )"><strong>0.932</strong></span></div><div> multi-hop only -> portion:0.619, count:546, errors: 76, accuracy: 0.861</div><br><div>dev -> portion:0.632, count:156, errors: 45, accuracy: 0.712</div><div> <span style="color:#000000;font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none">ensemble only -> portion:0.632, count:156, errors: 37, accuracy: <span style="color:rgb( 255 , 0 , 0 )"><strong>0.763</strong></span></span><div> <span style="color:#000000;font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff;display:!important;float:none"> multi-hop only -> portion:0.632, count:156, errors: 43, accuracy: 0.724</span><br><div>test -> portion:0.580, count:112, errors: 29, accuracy: 0.741</div></div></div><div> <span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important">ensemble only -> portion:0.580, count:112, errors: 23, accuracy: </span><span style="font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important;color:rgb( 255 , 0 , 0 )"><strong>0.795</strong></span><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important"> multi-hop only -> portion:0.580, count:112, errors: 26, accuracy: 0.768</span></div><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important"><br></span></div><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important">Multi-Spans:</span></div><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important">train -> portion:0.083, count:73, errors: 37, accuracy: 0.493</span></div><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important">dev -> portion:0.085, count:21, errors: 11, accuracy: 0.476</span></div><div style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 )"><span style="color:rgb( 0 , 0 , 0 );font-family:'arial' , 'helvetica' , sans-serif;font-size:16px;font-style:normal;font-weight:400;letter-spacing:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb( 255 , 255 , 255 );float:none;display:!important">test -> portion:0.104, count:20, errors: 13, accuracy: 0.350</span></div><br></div></div><div>jjfan</div><hr id="zwchr"><div><b>From: </b>"Simonc" <simonc@iis.sinica.edu.tw><br><b>To: </b>"Most-ai Contest" <Most-ai-contest@iis.sinica.edu.tw><br><b>Sent: </b>Tuesday, March 31, 2020 6:00:52 PM<br><b>Subject: </b>[Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate<br></div><br><div><div style="font-family:'arial' , 'helvetica' , sans-serif;font-size:12pt;color:#000000"><div>Dear all,</div><br><div>The attached file contains the performance data from our results in 3/27.</div><div>This time, the gold answer inclusion rate is also included. (That is, counting the cases where the correct answer is included in the answer candidates.)</div><br><div>Regards,</div><div>張光瑜</div></div><br>_______________________________________________<br>Most-ai-contest mailing list<br>Most-ai-contest@iis.sinica.edu.tw<br>https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest<br></div></div><br>_______________________________________________<br>Most-ai-contest mailing list<br>Most-ai-contest@iis.sinica.edu.tw<br>https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest<br></div></div><br>_______________________________________________<br>Most-ai-contest mailing list<br>Most-ai-contest@iis.sinica.edu.tw<br>https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest<br></div></div></body></html>