[Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate

[范正忠]

Dear all, 

Enclosed please find the performance of the last code: 

ENSEMBLEModule: 12 models (new), single_span_multi_hop_v2_1 
Multi-Spans: MSPE_v18_branchy20 
date_duration_module_3 
arithmetic_module_3 
kinship_module6 

Overall: 
train -> total: 882, correct: 668, accuracy: 0.757 
dev -> total: 247, correct: 161, accuracy: 0.652 
test -> total: 193, correct: 126, accuracy: 0.653 

Single-Span: 
train -> portion:0.619, count:546, errors: 86, accuracy: 0.842 
ensemble only -> portion:0.619, count:546, errors: 37, accuracy: 0.932 
multi-hop only -> portion:0.619, count:546, errors: 76, accuracy: 0.861 

dev -> portion:0.632, count:156, errors: 45, accuracy: 0.712 
ensemble only -> portion:0.632, count:156, errors: 37, accuracy: 0.763 
multi-hop only -> portion:0.632, count:156, errors: 43, accuracy: 0.724 

test -> portion:0.580, count:112, errors: 29, accuracy: 0.741 
ensemble only -> portion:0.580, count:112, errors: 23, accuracy: 0.795 
multi-hop only -> portion:0.580, count:112, errors: 26, accuracy: 0.768 

Multi-Spans: 
train -> portion:0.083, count:73, errors: 37, accuracy: 0.493 
dev -> portion:0.085, count:21, errors: 11, accuracy: 0.476 
test -> portion:0.104, count:20, errors: 13, accuracy: 0.350 

jjfan 

From: "Simonc" <simonc at iis.sinica.edu.tw> 
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw> 
Sent: Tuesday, March 31, 2020 6:00:52 PM 
Subject: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate 

Dear all, 

The attached file contains the performance data from our results in 3/27. 
This time, the gold answer inclusion rate is also included. (That is, counting the cases where the correct answer is included in the answer candidates.) 

Regards, 
張光瑜 

_______________________________________________ 
Most-ai-contest mailing list 
Most-ai-contest at iis.sinica.edu.tw 
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200402/3787f277/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 4.2-1.7z
Type: application/x-7z-compressed
Size: 1766336 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200402/3787f277/attachment-0001.bin>