OVERALL ACC: (BEFORE OUTPUT FORMATTER) 114 / 208 = 0.5480769230769231 -------------------------------------------------------------------------------- OVERALL ACC: (AFTER OUTPUT FORMATTER) 112 / 208 = 0.5384615384615384 Formatter Errors: ['D260Q06', 'D283Q01', 'D296Q01'] -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER MODE TOP-1): GRND/PRED SS MS YN ARI CNT CMP CS DD KS Single-Span-Extraction 130 3 0 0 0 0 0 0 0 Multi-Spans-Extraction 0 25 0 0 0 0 0 0 0 YesNo 0 0 20 0 0 0 0 0 0 Arithmetic-Operations 0 0 0 1 0 0 0 0 0 Counting 0 0 0 0 3 0 0 0 0 Comparing-Members 0 0 0 0 0 0 0 0 0 CommonSense 0 0 0 0 0 0 0 0 0 Date-Duration 0 0 0 0 0 0 0 12 0 Kinship 0 1 0 0 0 0 0 0 13 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER TYPE): GRND/PRED PER D-D LOC ORG NUM YN KIN EVT OBJ MISC Person 25 0 0 0 0 0 0 0 0 0 Date-Duration 0 35 0 0 0 0 0 0 0 0 Location 0 0 24 0 0 0 0 0 0 0 Organization 0 0 0 15 0 0 0 0 0 0 Num-Measure 0 0 0 0 16 0 0 0 0 0 YesNo 0 0 0 0 0 20 0 0 0 0 Kinship 0 0 0 0 0 0 16 1 0 0 Event 0 0 0 0 0 0 0 1 0 0 Object 0 0 0 0 0 0 0 0 50 2 Misc 0 0 0 0 0 0 0 0 0 3 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.4662 0.9042 0.1618 Single-Span-Multi-Hop-Extraction 0.6241 0.9087 0.1663 Multi-Spans-Extraction 0.0000 1.0000 0.0000 YesNo 0.9000 0.9884 0.0275 Arithmetic-Operations 1.0000 1.0000 0.0000 Counting 0.0000 0.0000 0.0000 Date-Duration 0.2500 0.7655 0.3243 Kinship 0.9231 1.0000 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED): ACC SCORE MEAN SCORE STD-VAR Single-Span-Extraction 0.0000 0.8495 0.1923 Single-Span-Multi-Hop-Extraction 0.0000 0.7900 0.2285 Multi-Spans-Extraction 0.0000 1.0000 0.0000 YesNo 0.0000 0.9920 0.0074 Arithmetic-Operations 0.0556 0.5073 0.3127 Counting 0.0556 0.0000 0.0000 Date-Duration 0.5669 0.8241 0.2460 Kinship 0.0000 0.6154 0.4865 -------------------------------------------------------------------------------- ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED): Single-Span-Extraction: ['D002Q01', 'D002Q02', 'D002Q04', 'D002Q10', 'D009Q02', 'D009Q03', 'D046Q03', 'D046Q04', 'D046Q05', 'D046Q08', 'D064Q03', 'D064Q04', 'D064Q05', 'D064Q06', 'D064Q07', 'D064Q09', 'D064Q12', 'D064Q16', 'D073Q01', 'D073Q12', 'D083Q01', 'D083Q03', 'D083Q04', 'D083Q08', 'D083Q10', 'D090Q10', 'D090Q11', 'D096Q01', 'D106Q02', 'D106Q03'] Single-Span-Multi-Hop-Extraction: ['D002Q02', 'D002Q04', 'D002Q10', 'D009Q02', 'D009Q03', 'D046Q08', 'D064Q03', 'D064Q05', 'D064Q06', 'D064Q13', 'D064Q16', 'D073Q12', 'D083Q01', 'D083Q03', 'D083Q08', 'D083Q10', 'D090Q01', 'D106Q03', 'D106Q07', 'D117Q02', 'D181Q02', 'D181Q03', 'D181Q04', 'D215Q01', 'D215Q06', 'D215Q09', 'D215Q10', 'D245Q01', 'D245Q05', 'D310Q07'] Multi-Spans-Extraction: ['D009Q04', 'D016Q01', 'D016Q02', 'D016Q04', 'D037Q06', 'D037Q11', 'D037Q12', 'D046Q02', 'D046Q06', 'D064Q08', 'D073Q02', 'D073Q06', 'D073Q07', 'D073Q08', 'D073Q09', 'D073Q11', 'D090Q09', 'D096Q02', 'D117Q04', 'D310Q02', 'D318Q01', 'D268Q09', 'D268Q10', 'D296Q05', 'D303Q06'] YesNo: ['D037Q19', 'D037Q20'] Arithmetic-Operations: [] Counting: ['D117Q01', 'D245Q04', 'D283Q01'] Date-Duration: ['D083Q02', 'D215Q11', 'D245Q03', 'D310Q03', 'D310Q05', 'D310Q06', 'D318Q07', 'D318Q08', 'D318Q09'] Kinship: ['D037Q04'] --------------------------------------------------------------------------------