OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	17 / 46 = 0.3695652173913043
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	18 / 46 = 0.391304347826087
Formatter Errors:
	[]
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 17 / 18 = 0.9444444444444444
	HARD: 0 / 7 = 0.0
--------------------------------------------------------------------------------
DISPATCHER AMODE TOP-3 PERFORMANCE: (COVERAGE)
	45 / 46 = 0.9782608695652174
	ERRORS: ['D039Q03']
--------------------------------------------------------------------------------
DISPATCHER ATYPE-3 PERFORMANCE: (COVERAGE)
	46 / 46 = 1.0
	ERRORS: []
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           12 / 18 = 0.6667
Single-Span-Multi-Hop-Extraction 7 / 18 = 0.3889
Multi-Spans-Extraction           1 / 11 = 0.0909
YesNo                            3 / 4 = 0.7500
Arithmetic-Operations            2 / 2 = 1.0000
Counting                         0 / 0 = NaN
Date-Duration                    3 / 7 = 0.4286
Kinship                          0 / 4 = 0.0000
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 0 = NaN
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           3 / 28 = 0.1071
Single-Span-Multi-Hop-Extraction 4 / 28 = 0.1429
Multi-Spans-Extraction           1 / 35 = 0.0286
YesNo                            0 / 42 = 0.0000
Arithmetic-Operations            1 / 44 = 0.0227
Counting                         0 / 0 = NaN
Date-Duration                    1 / 39 = 0.0256
Kinship                          0 / 42 = 0.0000
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 46 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D031Q01', 'D033Q04', 'D039Q03', 'D042Q12', 'D049Q07', 'D314Q03']

Single-Span-Multi-Hop-Extraction:
['D031Q01', 'D033Q04', 'D035Q06', 'D039Q03', 'D042Q12', 'D042Q13', 'D046Q07', 'D097Q03', 'D120Q03', 'D248Q07', 'D314Q03']

Multi-Spans-Extraction:
['D008Q06', 'D013Q02', 'D015Q04', 'D046Q01', 'D072Q07', 'D092Q01', 'D094Q04', 'D107Q01', 'D120Q02', 'D318Q04']

YesNo:
['D117Q08']

Arithmetic-Operations:
[]

Counting:
[]

Date-Duration:
['D042Q11', 'D076Q01', 'D082Q07', 'D215Q05']

Kinship:
['D036Q15', 'D036Q18', 'D037Q05', 'D040Q14']

Wiki-Json-Inference:
[]

Summarize:
[]

--------------------------------------------------------------------------------