OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	20 / 50 = 0.4
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	23 / 50 = 0.46
Formatter Errors:
	[]
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 20 / 20 = 1.0
	HARD: 0 / 9 = 0.0
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           9 / 22 = 0.4091
Single-Span-Multi-Hop-Extraction 10 / 22 = 0.4545
Multi-Spans-Extraction           0 / 9 = 0.0000
YesNo                            4 / 7 = 0.5714
Arithmetic-Operations            0 / 0 = NaN
Counting                         0 / 0 = NaN
Date-Duration                    3 / 8 = 0.3750
Kinship                          1 / 4 = 0.2500
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           0 / 28 = 0.0000
Single-Span-Multi-Hop-Extraction 0 / 28 = 0.0000
Multi-Spans-Extraction           0 / 41 = 0.0000
YesNo                            0 / 43 = 0.0000
Arithmetic-Operations            2 / 50 = 0.0400
Counting                         0 / 50 = 0.0000
Date-Duration                    11 / 42 = 0.2619
Kinship                          0 / 46 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERROR SAMPLES (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D004Q05', 'D035Q06', 'D042Q07', 'D042Q12', 'D046Q07', 'D071Q11', 'D085Q02', 'D096Q03', 'D104Q01', 'D120Q03', 'D138Q09', 'D241Q04', 'D314Q03']

Single-Span-Multi-Hop-Extraction:
['D035Q06', 'D042Q12', 'D046Q07', 'D071Q11', 'D085Q02', 'D090Q01', 'D096Q03', 'D097Q03', 'D104Q01', 'D160Q04', 'D241Q04', 'D314Q03']

Multi-Spans-Extraction:
['D007Q01', 'D008Q06', 'D014Q03', 'D039Q03', 'D045Q02', 'D072Q07', 'D094Q04', 'D107Q01', 'D318Q04']

YesNo:
['D113Q04', 'D117Q08', 'D127Q05']

Arithmetic-Operations:
[]

Counting:
[]

Date-Duration:
['D042Q11', 'D082Q01', 'D104Q03', 'D215Q05', 'D243Q07']

Kinship:
['D040Q14', 'D040Q15', 'D091Q11']

--------------------------------------------------------------------------------