OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	22 / 50 = 0.44
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	21 / 50 = 0.42
Formatter Errors:
	['D324Q09']
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 22 / 22 = 1.0
	HARD: 0 / 18 = 0.0
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           9 / 30 = 0.3000
Single-Span-Multi-Hop-Extraction 16 / 30 = 0.5333
Multi-Spans-Extraction           0 / 2 = 0.0000
YesNo                            8 / 15 = 0.5333
Arithmetic-Operations            0 / 1 = 0.0000
Counting                         0 / 0 = NaN
Date-Duration                    0 / 1 = 0.0000
Kinship                          0 / 0 = NaN
Wiki-Json-Inference              1 / 1 = 1.0000
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           0 / 20 = 0.0000
Single-Span-Multi-Hop-Extraction 0 / 20 = 0.0000
Multi-Spans-Extraction           1 / 48 = 0.0208
YesNo                            0 / 35 = 0.0000
Arithmetic-Operations            10 / 49 = 0.2041
Counting                         0 / 50 = 0.0000
Date-Duration                    11 / 49 = 0.2245
Kinship                          0 / 50 = 0.0000
Wiki-Json-Inference              0 / 0 = NaN
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D058Q11', 'D058Q13', 'D058Q14', 'D058Q15', 'D058Q17', 'D124Q05', 'D124Q06', 'D124Q08', 'D126Q02', 'D126Q03', 'D126Q04', 'D126Q05', 'D126Q06', 'D126Q09', 'D204Q07', 'D320Q01', 'D324Q02', 'D324Q03', 'D324Q05', 'D324Q09', 'D325Q03']

Single-Span-Multi-Hop-Extraction:
['D058Q11', 'D058Q14', 'D124Q05', 'D124Q06', 'D124Q08', 'D126Q03', 'D126Q05', 'D126Q06', 'D126Q09', 'D204Q02', 'D324Q01', 'D324Q03', 'D325Q01', 'D325Q03']

Multi-Spans-Extraction:
['D058Q07', 'D058Q10']

YesNo:
['D201Q04', 'D201Q05', 'D320Q02', 'D320Q03', 'D320Q05', 'D320Q06', 'D320Q08']

Arithmetic-Operations:
['D204Q04']

Counting:
[]

Date-Duration:
['D124Q07']

Kinship:
[]

Wiki-Json-Inference:
[]

--------------------------------------------------------------------------------