OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	157 / 247 = 0.6356275303643725
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	159 / 247 = 0.6437246963562753
Formatter Errors:
	['D241Q07']
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 157 / 158 = 0.9936708860759493
	HARD: 0 / 34 = 0.0
--------------------------------------------------------------------------------
DISPATCHER AMODE TOP-3 PERFORMANCE: (COVERAGE)
243 / 247 = 0.9838056680161943
--------------------------------------------------------------------------------
DISPATCHER ATYPE-3 PERFORMANCE: (COVERAGE)
241 / 247 = 0.9757085020242915
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           78 / 124 = 0.6290
Single-Span-Multi-Hop-Extraction 88 / 124 = 0.7097
Multi-Spans-Extraction           0 / 21 = 0.0000
YesNo                            19 / 28 = 0.6786
Arithmetic-Operations            14 / 23 = 0.6087
Counting                         0 / 0 = NaN
Date-Duration                    16 / 23 = 0.6957
Kinship                          18 / 23 = 0.7826
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 5 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           31 / 123 = 0.2520
Single-Span-Multi-Hop-Extraction 33 / 123 = 0.2683
Multi-Spans-Extraction           12 / 226 = 0.0531
YesNo                            0 / 219 = 0.0000
Arithmetic-Operations            3 / 224 = 0.0134
Counting                         0 / 0 = NaN
Date-Duration                    4 / 224 = 0.0179
Kinship                          3 / 224 = 0.0134
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 242 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D001Q02', 'D007Q02', 'D007Q03', 'D013Q01', 'D013Q11', 'D031Q01', 'D031Q02', 'D037Q11', 'D037Q12', 'D049Q01', 'D049Q05', 'D049Q07', 'D073Q01', 'D073Q11', 'D079Q05', 'D079Q06', 'D079Q07', 'D085Q03', 'D085Q04', 'D085Q05', 'D085Q06', 'D091Q03', 'D115Q03', 'D115Q04', 'D115Q07', 'D127Q01', 'D181Q02', 'D181Q04', 'D181Q05', 'D181Q06', 'D247Q02', 'D247Q06', 'D253Q01', 'D253Q11', 'D253Q12', 'D271Q06', 'D283Q07', 'D289Q08', 'D289Q10', 'D289Q11', 'D295Q02', 'D295Q04', 'D295Q09', 'D301Q05', 'D307Q06', 'D325Q03']

Single-Span-Multi-Hop-Extraction:
['D007Q02', 'D007Q03', 'D013Q01', 'D013Q08', 'D013Q12', 'D031Q01', 'D031Q02', 'D037Q12', 'D049Q01', 'D049Q05', 'D061Q03', 'D073Q11', 'D079Q06', 'D079Q07', 'D085Q04', 'D091Q03', 'D097Q03', 'D115Q03', 'D115Q04', 'D115Q07', 'D127Q01', 'D181Q02', 'D211Q03', 'D211Q05', 'D247Q02', 'D247Q06', 'D253Q01', 'D253Q06', 'D289Q08', 'D295Q04', 'D295Q07', 'D301Q02', 'D307Q03', 'D307Q06', 'D325Q03', 'D325Q04']

Multi-Spans-Extraction:
['D001Q08', 'D007Q01', 'D007Q04', 'D007Q05', 'D013Q02', 'D031Q03', 'D031Q04', 'D049Q02', 'D073Q02', 'D073Q06', 'D073Q07', 'D073Q08', 'D073Q09', 'D085Q07', 'D115Q06', 'D115Q08', 'D127Q07', 'D241Q08', 'D241Q09', 'D313Q01', 'D313Q02']

YesNo:
['D037Q17', 'D037Q18', 'D115Q02', 'D127Q06', 'D253Q07', 'D301Q06', 'D313Q04', 'D313Q05', 'D325Q05']

Arithmetic-Operations:
['D049Q03', 'D073Q04', 'D085Q01', 'D085Q08', 'D247Q01', 'D247Q05', 'D247Q07', 'D247Q09', 'D283Q01']

Counting:
[]

Date-Duration:
['D061Q04', 'D073Q05', 'D097Q06', 'D103Q04', 'D241Q04', 'D247Q08', 'D259Q05']

Kinship:
['D037Q04', 'D037Q05', 'D037Q06', 'D091Q11', 'D091Q12']

Wiki-Json-Inference:
[]

Summarize:
['D001Q11', 'D049Q04', 'D073Q12', 'D103Q07', 'D289Q07']

--------------------------------------------------------------------------------