[Most-ai-contest] Release dataset 1.2
范正忠
jjfan於iis.sinica.edu.tw
Fri 11月 22 07:23:05 CST 2019
Here are some examples (basically the question will be ended with ... 有哪些？) , please see if we need to adjust
jjfan
From: "kysu" <kysu at iis.sinica.edu.tw>
To: "范正忠" <jjfan at iis.sinica.edu.tw>, "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw>
Cc: "kysu" <kysu at iis.sinica.edu.tw>
Sent: Thursday, November 21, 2019 9:56:17 PM
Subject: RE: [Most-ai-contest] Release dataset 1.2
Dev:
Answer Type= Kinship: # 15
Answer Mode= Kinship: # 12
It is a bit strange that these two numbers are not the same (similarly in test-set). Could you please give a few examples that Answer Type= Kinship but Answer Mode \= Kinship? Thanks
KY
From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 范正忠
Sent: Thursday, November 21, 2019 4:45 PM
To: Most-ai Contest <Most-ai-contest at iis.sinica.edu.tw>
Subject: Re: [Most-ai-contest] Release dataset 1.2
Dear all,
Enclosed please find the refined statistic table (remove some ambiguous items)
All
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
53
87
83
77
137
99
86
79
19
26
746
7.10%
11.66%
11.13%
10.32%
18.36%
13.27%
11.53%
10.59%
2.55%
3.49%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
53
101
75
426
61
6
23
1
0
746
7.10%
13.54%
10.05%
57.10%
8.18%
0.80%
3.08%
0.13%
0.00%
100.00%
Train
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
21
43
59
39
50
44
56
24
11
16
363
5.79%
11.85%
16.25%
10.74%
13.77%
12.12%
15.43%
6.61%
3.03%
4.41%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
21
40
59
208
18
3
14
0
0
363
5.79%
11.02%
16.25%
57.30%
4.96%
0.83%
3.86%
0.00%
0.00%
100.00%
Dev
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
17
23
15
21
56
34
14
18
2
9
209
8.13%
11.00%
7.18%
10.05%
26.79%
16.27%
6.70%
8.61%
0.96%
4.31%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
17
26
12
117
29
2
6
0
0
209
8.13%
12.44%
5.74%
55.98%
13.88%
0.96%
2.87%
0.00%
0.00%
100.00%
Test
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
15
21
9
17
31
21
16
37
6
1
174
8.62%
12.07%
5.17%
9.77%
17.82%
12.07%
9.20%
21.26%
3.45%
0.57%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
15
35
4
101
14
1
3
0
0
173
8.67%
20.23%
2.31%
58.38%
8.09%
0.58%
1.73%
0.00%
0.00%
100.00%
From: " 范正忠 " < [ mailto:jjfan at iis.sinica.edu.tw | jjfan at iis.sinica.edu.tw ] >
To: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Thursday, November 21, 2019 4:06:45 PM
Subject: [Most-ai-contest] Release dataset 1.2
Dear all,
Enclosed please find FGC_Release_1.1 data-set , which
1. DRCD, ASR, Kaggle, Lee
2. FGC_release_A_train, FGC_release_A_dev, FGC_release_A_test
Please note that all data are in cn language and FGC format
The following is the answer-type & answer-mode distributions for each dataset ( less "Misc" answer-type )
All
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
53
59
83
73
125
92
83
71
19
88
746
53
87
83
77
137
99
86
79
19
26
746
7.10%
11.66%
11.13%
10.32%
18.36%
13.27%
11.53%
10.59%
2.55%
3.49%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
53
101
75
442
57
3
15
0
0
746
53
101
75
426
61
6
23
1
0
746
7.10%
13.54%
10.05%
57.10%
8.18%
0.80%
3.08%
0.13%
0.00%
100.00%
Train
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
21
43
59
39
50
44
56
24
11
16
363
5.79%
11.85%
16.25%
10.74%
13.77%
12.12%
15.43%
6.61%
3.03%
4.41%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
21
40
59
208
18
3
14
0
0
363
5.79%
11.02%
16.25%
57.30%
4.96%
0.83%
3.86%
0.00%
0.00%
100.00%
Dev
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
17
23
15
21
56
34
14
18
2
9
209
8.13%
11.00%
7.18%
10.05%
26.79%
16.27%
6.70%
8.61%
0.96%
4.31%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
17
26
12
117
29
2
6
0
0
209
8.13%
12.44%
5.74%
55.98%
13.88%
0.96%
2.87%
0.00%
0.00%
100.00%
Test
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
15
21
9
17
31
21
16
37
6
1
174
8.62%
12.07%
5.17%
9.77%
17.82%
12.07%
9.20%
21.26%
3.45%
0.57%
100.00%
Answer Mode
YesNo ( 是否題 )
Multi-Spans-Extraction ( 列舉題型 )
Kinship
Single-Span-Extraction ( 單一答案 )
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
15
35
4
101
14
1
3
0
0
173
8.67%
20.23%
2.31%
58.38%
8.09%
0.58%
1.73%
0.00%
0.00%
100.00%
Best,
jjfan
From: " 范正忠 " < [ mailto:jjfan at iis.sinica.edu.tw | jjfan at iis.sinica.edu.tw ] >
To: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Tuesday, November 19, 2019 5:04:12 PM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
Dear all,
Enclosed please find FGC_Release_1.1 data-set, which include
1. DRCD, ASR, Kaggle, Lee
2. FGC_release_A_train_1.1, FGC_release_A_dev_1.1, FGC_release_A_test_1.1
Please use this data-set as the standard benchmark.
Also note that you can use item 1 + FGC_release_A_train_1.1 as your training set, FGC_release_A_dev_1.1 as development set, and FGC_release_A_test_1.1 as testing set.
Please feel free to let me know any questions.
Best,
jjfan
From: " 范正忠 " < [ mailto:jjfan at iis.sinica.edu.tw | jjfan at iis.sinica.edu.tw ] >
To: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Monday, November 18, 2019 8:56:28 AM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
Dear all,
Please send me error list of Answer-Type and Answer-Mode annotations end of today.
Then I will divide FGC release data-set into training, development, and test, and release them tomorrow for your benchmark.
Thanks.
Best,
jjfan
From: "Chiangyulun0914" < [ mailto:chiangyulun0914 at iis.sinica.edu.tw | chiangyulun0914 at iis.sinica.edu.tw ] >
To: "Most-ai Contest" < [ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ] >
Sent: Wednesday, November 13, 2019 5:19:49 PM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
大家好，
檔案以 .xlsx 或 .csv 檔為主。附檔為範例。感謝 !
江侑倫
自然語言理解實驗室
中央研究院資訊科學研究所
Yu-Lun Chiang
Natural Language Understanding Lab
Institute of Information Science, Academia Sinica
Mobile: +886-975279013 (Taiwan)
江侑倫 < [ mailto:chiangyulun0914 at iis.sinica.edu.tw | chiangyulun0914 at iis.sinica.edu.tw ] > 於 2019 年 11 月 13 日 週三 下午 4:57 寫道：
大家好，
有鑑於范博士最新釋出的 fgc-2019 dataset 中，可能因使用 rule-based 標記 anstype 和 ansmode 而造成一些錯誤，因此若團隊成員在使用數據集時發現致命錯誤， 請隨手紀錄，並依照附檔的格式與檔名 ，將修正前和修正後的 anstype 與 ansmode 回傳給范博士，以利范博士更新數據集。在此亦附上 20191112 當天范博士釋出最新版的 anstype 與 ansmode 種類。
若 anstype 與 ansmode 中 僅出現一個 需要被修正，仍請將 不需修正的另一個 也填進附檔的 refined 那行中，以利范博士直接依照 refined 行中的資訊進行數據集更新。
感謝 !
江侑倫
自然語言理解實驗室
中央研究院資訊科學研究所
Yu-Lun Chiang
Natural Language Understanding Lab
Institute of Information Science, Academia Sinica
Mobile: +886-975279013 (Taiwan)
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
_______________________________________________
Most-ai-contest mailing list
[ mailto:Most-ai-contest at iis.sinica.edu.tw | Most-ai-contest at iis.sinica.edu.tw ]
[ https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest | https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest ]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 110656.42999997362
Type: image/png
Size: 16239 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0017.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 152352.13500005193
Type: image/png
Size: 15473 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0018.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0019.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0020.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0021.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0022.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0023.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0024.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0025.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0026.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0027.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0028.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0029.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0030.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0031.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0032.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191122/12a5c993/attachment-0033.png>
More information about the Most-ai-contest
mailing list