OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	18 / 50 = 0.36
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	16 / 50 = 0.32
Formatter Errors:
	['D081Q03', 'D081Q04']
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 18 / 19 = 0.9473684210526315
	HARD: 0 / 14 = 0.0
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN ACTIVATED AS THE TOP-1 MODE):
                                 ACC
Single-Span-Extraction           11 / 16 = 0.6875
Single-Span-Multi-Hop-Extraction 0 / 0 = NaN
Multi-Spans-Extraction           0 / 13 = 0.0000
YesNo                            4 / 9 = 0.4444
Arithmetic-Operations            2 / 2 = 1.0000
Counting                         0 / 0 = NaN
Date-Duration                    2 / 6 = 0.3333
Kinship                          0 / 0 = NaN
Wiki-Kb-Inference                0 / 0 = NaN
Summarize                        0 / 0 = NaN
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN ACTIVATED NOT AS THE TOP-1 MODE):
                                ACC
Single-Span-Extraction           5 / 34 = 0.1471
Single-Span-Multi-Hop-Extraction 0 / 0 = NaN
Multi-Spans-Extraction           1 / 37 = 0.0270
YesNo                            0 / 41 = 0.0000
Arithmetic-Operations            2 / 48 = 0.0417
Counting                         0 / 38 = 0.0000
Date-Duration                    1 / 44 = 0.0227
Kinship                          0 / 50 = 0.0000
Wiki-Kb-Inference                0 / 0 = NaN
Summarize                        0 / 46 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN ACTIVATED AS THE TOP-1 MODE):
Single-Span-Extraction:
['D123Q01', 'D123Q09', 'D142Q04', 'D331Q06', 'D332Q02']

Single-Span-Multi-Hop-Extraction:
[]

Multi-Spans-Extraction:
['D074Q03', 'D075Q13', 'D081Q11', 'D114Q08', 'D118Q02', 'D118Q03', 'D119Q05', 'D123Q03', 'D162Q06', 'D331Q03', 'D331Q05', 'D331Q09', 'D332Q09']

YesNo:
['D114Q09', 'D118Q04', 'D118Q05', 'D162Q08', 'D331Q02']

Arithmetic-Operations:
[]

Counting:
[]

Date-Duration:
['D075Q11', 'D114Q07', 'D119Q06', 'D119Q08']

Kinship:
[]

Wiki-Kb-Inference:
[]

Summarize:
[]

--------------------------------------------------------------------------------