[Most-ai-contest] Release dataset 1.2
kysu
kysu於iis.sinica.edu.tw
Thu 11月 21 21:56:17 CST 2019
Dev:
Answer Type= Kinship: #15
Answer Mode= Kinship: #12
It is a bit strange that these two numbers are not the same (similarly in test-set). Could you please give a few examples that Answer Type= Kinship but Answer Mode \= Kinship? Thanks
KY
From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 范正忠
Sent: Thursday, November 21, 2019 4:45 PM
To: Most-ai Contest <Most-ai-contest at iis.sinica.edu.tw>
Subject: Re: [Most-ai-contest] Release dataset 1.2
Dear all,
Enclosed please find the refined statistic table (remove some ambiguous items)
All
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
53
87
83
77
137
99
86
79
19
26
746
7.10%
11.66%
11.13%
10.32%
18.36%
13.27%
11.53%
10.59%
2.55%
3.49%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
53
101
75
426
61
6
23
1
0
746
7.10%
13.54%
10.05%
57.10%
8.18%
0.80%
3.08%
0.13%
0.00%
100.00%
Train
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
21
43
59
39
50
44
56
24
11
16
363
5.79%
11.85%
16.25%
10.74%
13.77%
12.12%
15.43%
6.61%
3.03%
4.41%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
21
40
59
208
18
3
14
0
0
363
5.79%
11.02%
16.25%
57.30%
4.96%
0.83%
3.86%
0.00%
0.00%
100.00%
Dev
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
17
23
15
21
56
34
14
18
2
9
209
8.13%
11.00%
7.18%
10.05%
26.79%
16.27%
6.70%
8.61%
0.96%
4.31%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
17
26
12
117
29
2
6
0
0
209
8.13%
12.44%
5.74%
55.98%
13.88%
0.96%
2.87%
0.00%
0.00%
100.00%
Test
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
15
21
9
17
31
21
16
37
6
1
174
8.62%
12.07%
5.17%
9.77%
17.82%
12.07%
9.20%
21.26%
3.45%
0.57%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
15
35
4
101
14
1
3
0
0
173
8.67%
20.23%
2.31%
58.38%
8.09%
0.58%
1.73%
0.00%
0.00%
100.00%
_____
From: "范正忠" <jjfan at iis.sinica.edu.tw <mailto:jjfan at iis.sinica.edu.tw> >
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw> >
Sent: Thursday, November 21, 2019 4:06:45 PM
Subject: [Most-ai-contest] Release dataset 1.2
Dear all,
Enclosed please find FGC_Release_1.1 data-set , which
1. DRCD, ASR, Kaggle, Lee
2. FGC_release_A_train, FGC_release_A_dev, FGC_release_A_test
Please note that all data are in cn language and FGC format
The following is the answer-type & answer-mode distributions for each dataset ( less "Misc" answer-type )
All
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
53
59
83
73
125
92
83
71
19
88
746
53
87
83
77
137
99
86
79
19
26
746
7.10%
11.66%
11.13%
10.32%
18.36%
13.27%
11.53%
10.59%
2.55%
3.49%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
53
101
75
442
57
3
15
0
0
746
53
101
75
426
61
6
23
1
0
746
7.10%
13.54%
10.05%
57.10%
8.18%
0.80%
3.08%
0.13%
0.00%
100.00%
Train
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
21
43
59
39
50
44
56
24
11
16
363
5.79%
11.85%
16.25%
10.74%
13.77%
12.12%
15.43%
6.61%
3.03%
4.41%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
21
40
59
208
18
3
14
0
0
363
5.79%
11.02%
16.25%
57.30%
4.96%
0.83%
3.86%
0.00%
0.00%
100.00%
Dev
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
17
23
15
21
56
34
14
18
2
9
209
8.13%
11.00%
7.18%
10.05%
26.79%
16.27%
6.70%
8.61%
0.96%
4.31%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
17
26
12
117
29
2
6
0
0
209
8.13%
12.44%
5.74%
55.98%
13.88%
0.96%
2.87%
0.00%
0.00%
100.00%
Test
Answer Type
YesNo
Num-Measure
Kinship
Person
Date-Duration
Location
Organization
Object
Event
Misc
Total
15
21
9
17
31
21
16
37
6
1
174
8.62%
12.07%
5.17%
9.77%
17.82%
12.07%
9.20%
21.26%
3.45%
0.57%
100.00%
Answer Mode
YesNo (是否題)
Multi-Spans-Extraction (列舉題型)
Kinship
Single-Span-Extraction (單一答案)
Date-Duration
Arithmetic-Operations
Counting
Comparing-Members
Common-Sense
15
35
4
101
14
1
3
0
0
173
8.67%
20.23%
2.31%
58.38%
8.09%
0.58%
1.73%
0.00%
0.00%
100.00%
Best,
jjfan
_____
From: "范正忠" <jjfan at iis.sinica.edu.tw <mailto:jjfan at iis.sinica.edu.tw> >
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw> >
Sent: Tuesday, November 19, 2019 5:04:12 PM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
Dear all,
Enclosed please find FGC_Release_1.1 data-set, which include
1. DRCD, ASR, Kaggle, Lee
2. FGC_release_A_train_1.1, FGC_release_A_dev_1.1, FGC_release_A_test_1.1
Please use this data-set as the standard benchmark.
Also note that you can use item 1 + FGC_release_A_train_1.1 as your training set, FGC_release_A_dev_1.1 as development set, and FGC_release_A_test_1.1 as testing set.
Please feel free to let me know any questions.
Best,
jjfan
_____
From: "范正忠" <jjfan at iis.sinica.edu.tw <mailto:jjfan at iis.sinica.edu.tw> >
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw> >
Sent: Monday, November 18, 2019 8:56:28 AM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
Dear all,
Please send me error list of Answer-Type and Answer-Mode annotations end of today.
Then I will divide FGC release data-set into training, development, and test, and release them tomorrow for your benchmark.
Thanks.
Best,
jjfan
_____
From: "Chiangyulun0914" <chiangyulun0914 at iis.sinica.edu.tw <mailto:chiangyulun0914 at iis.sinica.edu.tw> >
To: "Most-ai Contest" <Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw> >
Sent: Wednesday, November 13, 2019 5:19:49 PM
Subject: Re: [Most-ai-contest] refinement of anstype and ansmode for fgc-2019 dataset
大家好，
檔案以 .xlsx 或 .csv 檔為主。附檔為範例。感謝!
江侑倫
自然語言理解實驗室
中央研究院資訊科學研究所
Yu-Lun Chiang
Natural Language Understanding Lab
Institute of Information Science, Academia Sinica
Mobile: +886-975279013 (Taiwan)
江侑倫 <chiangyulun0914 at iis.sinica.edu.tw <mailto:chiangyulun0914 at iis.sinica.edu.tw> > 於 2019年11月13日 週三 下午4:57寫道：
大家好，
有鑑於范博士最新釋出的 fgc-2019 dataset 中，可能因使用 rule-based 標記 anstype 和 ansmode 而造成一些錯誤，因此若團隊成員在使用數據集時發現致命錯誤，請隨手紀錄，並依照附檔的格式與檔名，將修正前和修正後的 anstype 與 ansmode 回傳給范博士，以利范博士更新數據集。在此亦附上 20191112 當天范博士釋出最新版的 anstype 與 ansmode 種類。
若 anstype 與 ansmode 中僅出現一個需要被修正，仍請將不需修正的另一個也填進附檔的 refined 那行中，以利范博士直接依照 refined 行中的資訊進行數據集更新。
感謝 !
江侑倫
自然語言理解實驗室
中央研究院資訊科學研究所
Yu-Lun Chiang
Natural Language Understanding Lab
Institute of Information Science, Academia Sinica
Mobile: +886-975279013 (Taiwan)
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw>
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw>
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw>
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
_______________________________________________
Most-ai-contest mailing list
Most-ai-contest at iis.sinica.edu.tw <mailto:Most-ai-contest at iis.sinica.edu.tw>
https://www.iis.sinica.edu.tw/mailman/listinfo/most-ai-contest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191121/6770f5c1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 51309 bytes
Desc: not available
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20191121/6770f5c1/attachment-0001.png>
More information about the Most-ai-contest
mailing list