OVERALL ACC: (BEFORE OUTPUT FORMATTER) 13 / 50 = 0.26 -------------------------------------------------------------------------------- OVERALL ACC: (AFTER OUTPUT FORMATTER) 14 / 50 = 0.28 Formatter Errors: [] -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER MODE TOP-1): GRND/PRED SS MS YN ARI CNT CMP CS DD KS Single-Span-Extraction 20 0 0 0 0 0 0 0 0 Multi-Spans-Extraction 0 4 0 0 0 0 0 0 0 YesNo 0 0 4 0 0 0 0 0 0 Arithmetic-Operations 0 0 0 1 0 0 0 0 0 Counting 0 0 0 0 3 0 0 0 0 Comparing-Members 0 0 0 0 0 0 0 0 0 CommonSense 0 0 0 0 0 0 0 0 0 Date-Duration 0 0 0 0 0 0 0 4 0 Kinship 0 0 0 0 0 0 0 0 6 -------------------------------------------------------------------------------- DISPATCHER PERFORMANCE (ANSWER TYPE): GRND/PRED PER D-D LOC ORG NUM YN KIN EVT OBJ MISC Person 2 0 0 0 0 0 0 0 0 0 Date-Duration 0 8 0 0 0 0 0 0 0 0 Location 0 0 4 0 0 0 0 0 0 0 Organization 0 0 0 5 0 0 0 0 0 0 Num-Measure 0 0 0 0 7 0 0 0 0 0 YesNo 0 0 0 0 0 4 0 0 0 0 Kinship 0 0 0 0 0 0 6 0 0 0 Event 0 0 0 0 0 0 0 0 0 0 Object 0 0 0 0 0 0 0 0 5 0 Misc 0 0 0 0 0 0 0 0 0 1 -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED): ACC Single-Span-Extraction 11 / 20 = 0.5500 Single-Span-Multi-Hop-Extraction 6 / 20 = 0.3000 Multi-Spans-Extraction 0 / 4 = 0.0000 YesNo 1 / 2 = 0.5000 Arithmetic-Operations 0 / 0 = NaN Counting 0 / 0 = NaN Date-Duration 0 / 1 = 0.0000 Kinship 0 / 0 = NaN -------------------------------------------------------------------------------- ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED): ACC Single-Span-Extraction 0 / 22 = 0.0000 Single-Span-Multi-Hop-Extraction 0 / 7 = 0.0000 Multi-Spans-Extraction 0 / 23 = 0.0000 YesNo 0 / 25 = 0.0000 Arithmetic-Operations 12 / 27 = 0.4444 Counting 0 / 27 = 0.0000 Date-Duration 12 / 26 = 0.4615 Kinship 0 / 27 = 0.0000 -------------------------------------------------------------------------------- ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED): Single-Span-Extraction: ['D033Q05', 'D064Q04', 'D087Q07', 'D116Q03', 'D244Q04', 'D245Q01', 'D247Q02', 'D312Q05', 'D312Q07'] Single-Span-Multi-Hop-Extraction: ['D001Q09', 'D033Q05', 'D042Q05', 'D072Q06', 'D087Q07', 'D090Q01', 'D107Q08', 'D116Q03', 'D245Q01', 'D247Q01', 'D247Q02', 'D247Q05', 'D312Q05', 'D312Q07'] Multi-Spans-Extraction: ['D008Q06', 'D010Q06', 'D088Q06', 'D105Q05'] YesNo: ['D087Q05'] Arithmetic-Operations: [] Counting: [] Date-Duration: ['D105Q02'] Kinship: [] --------------------------------------------------------------------------------