[Most-ai-contest] Updated WikiQA (V2.0)

Sat 4月 4 17:53:22 CST 2020

Glad to see the significant improvement from your WikiQA module!

KY

From: most-ai-contest-bounces at iis.sinica.edu.tw [mailto:most-ai-contest-bounces at iis.sinica.edu.tw] On Behalf Of 許喬為
Sent: Saturday, April 4, 2020 5:46 PM
To: jjfan <jjfan at iis.sinica.edu.tw>
Cc: Most-ai-contest <Most-ai-contest at iis.sinica.edu.tw>
Subject: [Most-ai-contest] Updated WikiQA (V2.0)

Dear Dr. Fan,

Please download the latest Wiki answer module V2.0 from the following link. In this release, I have modified these:

https://www.dropbox.com/s/65ts8paj93no08o/WIKIQA_V2.0.tar.gz?dl=0

- Add 高度, 墓地 and custom 寿命, 成立或建立年份 attributes in parsing questions

- Fix a bug of entity linking (regarding conversion between traditional chinese and simplified one)

This version answers 50% more questions than the previous version and at the same time increase the precision from 90.9 to 94.1%.

Previous (V1.0.3): 20/22 = 90.9%

Current (V2.0): 33/35 = 94.1%

Note that in this version, I have specifically looked at the errors of the overall system and focused on using information from Wikidata to solve them, i.e., 

(Q) 苏轼几岁时过世? (WikiQA) get_year(苏轼的死亡日期 -  苏轼的出生日期) = 62岁

== gained corrects (15) ==

D097Q05 诺贝尔于哪一年过世?

> D097Q06 诺贝尔享年几岁? ['63岁']

D077Q10 马可.波罗是哪一国人？(文本中) 威尼斯共和国 ['威尼斯共和国']

> D064Q15 郭雪湖享年几岁 103岁 ['103岁', '104岁']

D064Q07 郭雪湖的妻子是谁？林阿琴 ['林阿琴']

D254Q01 杨震出生于哪个朝代？东汉 ['东汉']

D069Q02 李安的职业是什么?导演 ['导演']

D038Q06 图灵是哪一国人?英国 ['英国']

D038Q07 图灵出生于哪一年?1912年 ['1912年']

D038Q08 图灵于哪一年过逝?1954年 ['1954年']

> D038Q09 图灵几岁时过逝?41岁 ['42岁', '41岁']

D246Q04 哈里发塔高度有多少公尺?828公尺 ['828公尺']

D245Q01 圣伯多禄大殿于哪一年开始重建?1506年 ['1506年']

> D071Q03 张大千几岁时过逝呢?83岁 ['83岁']

D283Q05 刘侠的职业是什么？作家 ['作家']

(>: the QID where the whole system answered it wrong)

== sacrificed corrects (2) ==

1.

D004Q05 Date-Duration Date-Duration: 苏东坡 死于 哪 一 年 ?(Gold) ['1101年']

(Gold SE) [(12, '\n建中靖国元年（1101年），'), (17, '七月二十八日于常州孙氏馆病卒，')]

(Pred SE) [(17, '七月二十八日于常州孙氏馆病卒，')]

---

(ParseQ) 苏东坡 | 死亡年份

(EL Queries) ['苏东坡', '苏东坡']

(EL) [('蘇軾', 'Q36020')]

(Traverse) ['1101-08-24T00:00:00.000Z'] from ['Q36020']

(Post-Proc) ['1101年', '-810年']

(Match) True False 

(Passage Match) ['1101年']

(Answers 1) (SE Match) [] <= error happened here

(WikiQA) None

[REASON] due to supporting evidence error (predicted answer not in extracted supporting evidences)

[Strategy] Tune the threshold of supporting evidence score

2.

D001Q01 Date-Duration Date-Duration: 苏东坡 在 中国 历史 上 ， 是 哪 一 个 朝代 的 人 ？(Gold) ['北宋']

(Gold SE) [(0, '苏轼（1037年1月8日－1101年8月24日），'), (2, '北宋时著名的文学家、政治家、艺术家、医学家。'), (4, '号东坡居士、铁冠道人。')]

(Pred SE) [(0, '苏轼（1037年1月8日－1101年8月24日），')]

---

(ParseQ) 苏东坡在中国历史上， | 朝代

(EL Queries) ['苏东坡在中国历史上，', '苏东坡']

(EL) [('蘇軾', 'Q36020')]

(Traverse) ['Q319460 北宋'] from ['Q36020']

(Post-Proc) ['北宋']

(Match) True 

(Passage Match) ['北宋', '北宋']

(Answers 1) (SE Match) []  <= error happened here

(WikiQA) None

[REASON] due to supporting evidence error (predicted answer not in extracted supporting evidences)

[Strategy] Tune the threshold of supporting evidence score

=== errors (2) ===

The only two errors are shown below:

D247Q09 哈里发塔总高度是几英尺？( WikiQA) 828公尺 (Gold) ['2,717英尺']

[Strategy] The author of output formatter said he will do unit conversion for me.

D001Q09 辛弃疾是哪一个朝代的人？(WikiQA) 宋  (Gold) ['南宋']

[Strategy] consider the length of the answer as feature

Best Regards,

Chiao-Wei

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iis.sinica.edu.tw/pipermail/most-ai-contest/attachments/20200404/4aa277c3/attachment-0001.html>