OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	16 / 46 = 0.34782608695652173
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	17 / 46 = 0.3695652173913043
Formatter Errors:
	[]
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 16 / 16 = 1.0
	HARD: 0 / 11 = 0.0
--------------------------------------------------------------------------------
DISPATCHER AMODE TOP-3 PERFORMANCE: (COVERAGE)
	45 / 46 = 0.9782608695652174
	ERRORS: ['D039Q03']
--------------------------------------------------------------------------------
DISPATCHER ATYPE-3 PERFORMANCE: (COVERAGE)
	46 / 46 = 1.0
	ERRORS: []
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           12 / 15 = 0.8000
Single-Span-Multi-Hop-Extraction 4 / 15 = 0.2667
Multi-Spans-Extraction           1 / 7 = 0.1429
YesNo                            5 / 7 = 0.7143
Arithmetic-Operations            1 / 2 = 0.5000
Counting                         0 / 0 = NaN
Date-Duration                    3 / 12 = 0.2500
Kinship                          0 / 3 = 0.0000
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 0 = NaN
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           3 / 31 = 0.0968
Single-Span-Multi-Hop-Extraction 5 / 31 = 0.1613
Multi-Spans-Extraction           2 / 39 = 0.0513
YesNo                            0 / 39 = 0.0000
Arithmetic-Operations            1 / 44 = 0.0227
Counting                         0 / 0 = NaN
Date-Duration                    0 / 34 = 0.0000
Kinship                          1 / 43 = 0.0233
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 46 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D039Q03', 'D042Q12', 'D104Q03']

Single-Span-Multi-Hop-Extraction:
['D035Q06', 'D039Q03', 'D039Q09', 'D042Q12', 'D042Q13', 'D046Q07', 'D097Q03', 'D104Q03', 'D120Q03', 'D160Q04', 'D311Q06']

Multi-Spans-Extraction:
['D007Q01', 'D008Q06', 'D045Q02', 'D072Q07', 'D094Q04', 'D107Q01']

YesNo:
['D115Q02', 'D117Q08']

Arithmetic-Operations:
['D096Q03']

Counting:
[]

Date-Duration:
['D035Q07', 'D042Q11', 'D082Q01', 'D100Q02', 'D104Q01', 'D215Q05', 'D241Q04', 'D243Q07', 'D311Q08']

Kinship:
['D040Q14', 'D040Q15', 'D091Q11']

Wiki-Json-Inference:
[]

Summarize:
[]

--------------------------------------------------------------------------------