[Most-ai-contest] The performance of the last version FGC QA system

[范正忠]

Dear all, 

Enclosed please find the current performance of the FGC QA system. 

		Single-Span 	Kuo 	Multi-Span 	Date-Duration 		Train 	Dev 	Test 
4月2日 	ENSEMBLEModule: 12 models (new) 	train: 0.890 
dev: 0.801 
test: 0.750 	train: 0.899 
dev: 0.699 
test: 0.741 	train: 0.493 
dev: 0.476 
test: 0.350 	train: 0.589 
dev: 0.720 
test: 0.581 		0.786 	0.709 	0.658 
4月3日 	ENSEMBLEModule: 12 models (new) 
MSPE_v18_branchy27 
date_duration_module_4 	train: 0.890 
dev: 0.801 
test: 0.750 	train: 0.899 
dev: 0.699 
test: 0.741 	train: 0.548 
dev: 0.667 
test: 0.450 	train: 0.589 
dev: 0.720 
test: 0.581 		0.791 	0.725 	0.668 

Please 郭家銍 help to check Date-Duration ver.4. 

Best, 
jjfan 



From: "范正忠" <jjfan at iis.sinica.edu.tw> 
To: "Simonc" <simonc at iis.sinica.edu.tw> 
Cc: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw> 
Sent: Thursday, April 2, 2020 3:56:36 PM 
Subject: Re: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate 

Dear all, 

Enclosed please find the performance of the last code: 

ENSEMBLEModule: 12 models (new), single_span_multi_hop_v2_1 
Multi-Spans: MSPE_v18_branchy20 
date_duration_module_3 
arithmetic_module_3 
kinship_module6 

Overall: 
train -> total: 882, correct: 668, accuracy: 0.757 
dev -> total: 247, correct: 161, accuracy: 0.652 
test -> total: 193, correct: 126, accuracy: 0.653 

Single-Span: 
train -> portion:0.619, count:546, errors: 86, accuracy: 0.842 
ensemble only -> portion:0.619, count:546, errors: 37, accuracy: 0.932 
multi-hop only -> portion:0.619, count:546, errors: 76, accuracy: 0.861 

dev -> portion:0.632, count:156, errors: 45, accuracy: 0.712 
ensemble only -> portion:0.632, count:156, errors: 37, accuracy: 0.763 
multi-hop only -> portion:0.632, count:156, errors: 43, accuracy: 0.724 
test -> portion:0.580, count:112, errors: 29, accuracy: 0.741 
ensemble only -> portion:0.580, count:112, errors: 23, accuracy: 0.795 
multi-hop only -> portion:0.580, count:112, errors: 26, accuracy: 0.768 

Multi-Spans: 
train -> portion:0.083, count:73, errors: 37, accuracy: 0.493 
dev -> portion:0.085, count:21, errors: 11, accuracy: 0.476 
test -> portion:0.104, count:20, errors: 13, accuracy: 0.350 

jjfan 

From: "Simonc" <simonc at iis.sinica.edu.tw> 
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw> 
Sent: Tuesday, March 31, 2020 6:00:52 PM 
Subject: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate 

Dear all, 

The attached file contains the performance data from our results in 3/27. 
This time, the gold answer inclusion rate is also included. (That is, counting the cases where the correct answer is included in the answer candidates.) 

Regards, 
張光瑜 

_______________________________________________ 
Most-ai-contest mailing list 
Most-ai-contest at iis.sinica.edu.tw 
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest 

_______________________________________________ 
Most-ai-contest mailing list 
Most-ai-contest at iis.sinica.edu.tw 
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200403/77b615db/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 4.3-2.7z
Type: application/x-7z-compressed
Size: 1424095 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200403/77b615db/attachment-0001.bin>