[Most-ai-contest] The performance of the last version FGC QA system (4/4)
范正忠
jjfan於iis.sinica.edu.tw
日 4月 5 16:39:31 CST 2020
Dear all,
申論題測試結果查詢 (第三週)
jjfan
From: "范正忠" <jjfan at iis.sinica.edu.tw>
To: "kysu" <kysu at iis.sinica.edu.tw>
Cc: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
Sent: Saturday, April 4, 2020 3:19:02 PM
Subject: Re: [Most-ai-contest] The performance of the last version FGC QA system (4/4)
Resend the output json file.
From: "范正忠" <jjfan at iis.sinica.edu.tw>
To: "kysu" <kysu at iis.sinica.edu.tw>
Cc: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
Sent: Saturday, April 4, 2020 10:57:15 AM
Subject: Re: [Most-ai-contest] The performance of the last version FGC QA system (4/4)
Sorry! Correction of the performance of single-span-ensemble module
Single-Span Kuo Simonc Multi-Span Date-Duration Train Dev Test
4月2日 ENSEMBLEModule: 12 models (new) + single_span_multi_hop_v2_1 train: 0.842
dev: 0.712
test: 0.741 train: 0.899
dev: 0.699
test: 0.741 train: 0.771
dev: 0.635
test: 0.705 0.757 0.652 0.653
4月2日 ENSEMBLEModule: 12 models (new) train: 0.890
dev: 0.801
test: 0.750 train: 0.899
dev: 0.699
test: 0.741 train: 0.493
dev: 0.476
test: 0.350 train: 0.589
dev: 0.720
test: 0.581 0.786 0.709 0.658
4月2日 single_span_multi_hop_v2_1 train: 0.775
dev: 0.692
test: 0.714 train: 0.771
dev: 0.635
test: 0.705 0.714 0.64 0.637
4月3日 ENSEMBLEModule: 12 models (new)
MSPE_v18_branchy27
date_duration_module_4 train: 0.890
dev: 0.801
test: 0.750 train: 0.899
dev: 0.699
test: 0.741 train: 0.548
dev: 0.667
test: 0.450 train: 0.589
dev: 0.720
test: 0.581 0.791 0.725 0.668
4月4日 ENSEMBLEModule: 12 models (new)
MSPE_v18_branchy27
date_duration_module_4 train: 0.888
dev: 0.801
test: 0.759
train: 0.897
dev: 0.699
test: 0.750
train: 0.949
dev: 0.837
test: 0.876
train: 0.548
dev: 0.667
test: 0.450 train: 0.753
dev: 0.720
test: 0.774 0.817 0.737 0.705
From: "kysu" <kysu at iis.sinica.edu.tw>
To: "范正忠" <jjfan at iis.sinica.edu.tw>, "Simonc" <simonc at iis.sinica.edu.tw>
Cc: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
Sent: Saturday, April 4, 2020 10:48:57 AM
Subject: RE: [Most-ai-contest] The performance of the last version FGC QA system (4/4)
Thanks. Glad to know that we have got a new record high performance! Let’s keep pushing the performance.
KY
From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 范正忠
Sent: Saturday, April 4, 2020 10:24 AM
To: Simonc <simonc at iis.sinica.edu.tw>
Cc: Most-ai Contest <Most-ai-contest at iis.sinica.edu.tw>
Subject: Re: [Most-ai-contest] The performance of the last version FGC QA system (4/4)
Dear all,
Enclosed please find the current performance of the FGC QA system.
Single-Span
Kuo
Simonc
Multi-Span
Date-Duration
Train
Dev
Test
4 月 2 日
ENSEMBLEModule: 12 models (new)
train: 0.890
dev: 0.801
test: 0.750
train: 0.899
dev: 0.699
test: 0.741
train: 0.493
dev: 0.476
test: 0.350
train: 0.589
dev: 0.720
test: 0.581
0.786
0.709
0.658
4 月 3 日
ENSEMBLEModule: 12 models (new)
MSPE_v18_branchy27
date_duration_module_4
train: 0.890
dev: 0.801
test: 0.750
train: 0.899
dev: 0.699
test: 0.741
train: 0.548
dev: 0.667
test: 0.450
train: 0.589
dev: 0.720
test: 0.581
0.791
0.725
0.668
4 月 4 日
ENSEMBLEModule: 12 models (new)
MSPE_v18_branchy27
date_duration_module_4
train: 0.888
dev: 0.801
test: 0.759
train: 0.949
dev: 0.837
test: 0.876
train: 0.548
dev: 0.667
test: 0.450
train: 0.753
dev: 0.720
test: 0.774
0.817
0.737
0.705
Please check the detail in Performance.txt file and Date-Duration performance is similar to 郭家銍 report.
Best,
jjfan
From: " 范正忠 " < [ mailto:jjfan at iis.sinica.edu.tw | jjfan at iis.sinica.edu.tw ] >
To: "Simonc" < [ mailto:simonc at iis.sinica.edu.tw | simonc at iis.sinica.edu.tw ] >
Cc: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Friday, April 3, 2020 4:45:39 PM
Subject: [Most-ai-contest] The performance of the last version FGC QA system
Dear all,
Enclosed please find the current performance of the FGC QA system.
Single-Span
Kuo
Multi-Span
Date-Duration
Train
Dev
Test
4 月 2 日
ENSEMBLEModule: 12 models (new)
train: 0.890
dev: 0.801
test: 0.750
train: 0.899
dev: 0.699
test: 0.741
train: 0.493
dev: 0.476
test: 0.350
train: 0.589
dev: 0.720
test: 0.581
0.786
0.709
0.658
4 月 3 日
ENSEMBLEModule: 12 models (new)
MSPE_v18_branchy27
date_duration_module_4
train: 0.890
dev: 0.801
test: 0.750
train: 0.899
dev: 0.699
test: 0.741
train: 0.548
dev: 0.667
test: 0.450
train: 0.589
dev: 0.720
test: 0.581
0.791
0.725
0.668
Please 郭家銍 help to check Date-Duration ver.4.
Best,
jjfan
From: " 范正忠 " < [ mailto:jjfan at iis.sinica.edu.tw | jjfan at iis.sinica.edu.tw ] >
To: "Simonc" < [ mailto:simonc at iis.sinica.edu.tw | simonc at iis.sinica.edu.tw ] >
Cc: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Thursday, April 2, 2020 3:56:36 PM
Subject: Re: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate
Dear all,
Enclosed please find the performance of the last code:
ENSEMBLEModule: 12 models (new), single_span_multi_hop_v2_1
Multi-Spans: MSPE_v18_branchy20
date_duration_module_3
arithmetic_module_3
kinship_module6
Overall:
train -> total: 882, correct: 668, accuracy: 0.757
dev -> total: 247, correct: 161, accuracy: 0.652
test -> total: 193, correct: 126, accuracy: 0.653
Single-Span:
train -> portion:0.619, count:546, errors: 86, accuracy: 0.842
ensemble only -> portion:0.619, count:546, errors: 37, accuracy: 0.932
multi-hop only -> portion:0.619, count:546, errors: 76, accuracy: 0.861
dev -> portion:0.632, count:156, errors: 45, accuracy: 0.712
ensemble only -> portion:0.632, count:156, errors: 37, accuracy: 0.763
multi-hop only -> portion:0.632, count:156, errors: 43, accuracy: 0.724
test -> portion:0.580, count:112, errors: 29, accuracy: 0.741
ensemble only -> portion:0.580, count:112, errors: 23, accuracy: 0.795
multi-hop only -> portion:0.580, count:112, errors: 26, accuracy: 0.768
Multi-Spans:
train -> portion:0.083, count:73, errors: 37, accuracy: 0.493
dev -> portion:0.085, count:21, errors: 11, accuracy: 0.476
test -> portion:0.104, count:20, errors: 13, accuracy: 0.350
jjfan
From: "Simonc" < [ mailto:simonc at iis.sinica.edu.tw | simonc at iis.sinica.edu.tw ] >
To: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Tuesday, March 31, 2020 6:00:52 PM
Subject: [Most-ai-contest] Updated Performance Data with Gold Answer Inclusion Rate
Dear all,
The attached file contains the performance data from our results in 3/27.
This time, the gold answer inclusion rate is also included. (That is, counting the cases where the correct answer is included in the answer candidates.)
Regards,
張光瑜
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200405/1f642afc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 27517953.160000034
Type: image/png
Size: 51605 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200405/1f642afc/attachment-0001.png>
More information about the Most-ai-contest
mailing list