OVERALL ACC: (BEFORE OUTPUT FORMATTER)
	119 / 193 = 0.616580310880829
--------------------------------------------------------------------------------
OVERALL ACC: (AFTER OUTPUT FORMATTER)
	121 / 193 = 0.6269430051813472
Formatter Errors:
	['D285Q04']
--------------------------------------------------------------------------------
AGGREGATOR PERFORMANCE:
	EASY: 119 / 121 = 0.9834710743801653
	HARD: 0 / 23 = 0.0
--------------------------------------------------------------------------------
DISPATCHER AMODE TOP-3 PERFORMANCE: (COVERAGE)
	183 / 190 = 0.9631578947368421
	ERRORS: ['D039Q03', 'D039Q04', 'D039Q05', 'D039Q06', 'D117Q01', 'D117Q03', 'D117Q06']
--------------------------------------------------------------------------------
DISPATCHER ATYPE-3 PERFORMANCE: (COVERAGE)
	182 / 190 = 0.9578947368421052
	ERRORS: ['D117Q03', 'D117Q04', 'D117Q06', 'D291Q09', 'D309Q04', 'D309Q05', 'D315Q04', 'D315Q09']
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN CORRECTLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                 ACC
Single-Span-Extraction           58 / 97 = 0.5979
Single-Span-Multi-Hop-Extraction 65 / 97 = 0.6701
Multi-Spans-Extraction           2 / 20 = 0.1000
YesNo                            18 / 25 = 0.7200
Arithmetic-Operations            9 / 14 = 0.6429
Counting                         0 / 0 = NaN
Date-Duration                    18 / 32 = 0.5625
Kinship                          0 / 2 = 0.0000
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 0 = NaN
--------------------------------------------------------------------------------
ANSWER MODULE STATS (WHEN WRONGLY ACTIVATED ACCORDING TO THE DISPATCHER):
                                ACC
Single-Span-Extraction           20 / 96 = 0.2083
Single-Span-Multi-Hop-Extraction 22 / 96 = 0.2292
Multi-Spans-Extraction           16 / 173 = 0.0925
YesNo                            0 / 168 = 0.0000
Arithmetic-Operations            2 / 179 = 0.0112
Counting                         0 / 0 = NaN
Date-Duration                    4 / 161 = 0.0248
Kinship                          5 / 191 = 0.0262
Wiki-Json-Inference              0 / 0 = NaN
Summarize                        0 / 190 = 0.0000
--------------------------------------------------------------------------------
ANSWER MODULE ERRORS (WHEN CORRECTLY ACTIVATED):
Single-Span-Extraction:
['D009Q02', 'D033Q01', 'D033Q02', 'D033Q03', 'D033Q04', 'D033Q05', 'D033Q06', 'D033Q07', 'D033Q08', 'D033Q12', 'D033Q14', 'D039Q03', 'D039Q04', 'D039Q05', 'D039Q06', 'D069Q12', 'D087Q06', 'D093Q07', 'D105Q06', 'D105Q07', 'D105Q08', 'D249Q03', 'D249Q04', 'D249Q06', 'D255Q06', 'D255Q08', 'D255Q09', 'D255Q12', 'D255Q13', 'D261Q07', 'D267Q02', 'D273Q01', 'D291Q08', 'D291Q09', 'D303Q03', 'D309Q04', 'D315Q03', 'D315Q07', 'D315Q09']

Single-Span-Multi-Hop-Extraction:
['D003Q04', 'D009Q02', 'D033Q01', 'D033Q03', 'D033Q04', 'D033Q05', 'D033Q06', 'D033Q07', 'D033Q08', 'D033Q14', 'D033Q15', 'D039Q03', 'D039Q04', 'D039Q05', 'D039Q06', 'D039Q08', 'D039Q09', 'D045Q04', 'D069Q03', 'D069Q09', 'D069Q12', 'D087Q01', 'D087Q06', 'D093Q07', 'D243Q04', 'D255Q04', 'D255Q10', 'D255Q12', 'D291Q09', 'D303Q03', 'D315Q01', 'D315Q09']

Multi-Spans-Extraction:
['D003Q03', 'D003Q06', 'D009Q04', 'D015Q02', 'D015Q04', 'D033Q09', 'D033Q10', 'D045Q02', 'D087Q03', 'D087Q04', 'D093Q04', 'D105Q05', 'D117Q04', 'D249Q02', 'D303Q06', 'D309Q05', 'D315Q02', 'D315Q04']

YesNo:
['D069Q11', 'D117Q08', 'D201Q05', 'D255Q07', 'D291Q03', 'D291Q05', 'D315Q06']

Arithmetic-Operations:
['D015Q05', 'D015Q06', 'D069Q13', 'D183Q02', 'D309Q02']

Counting:
[]

Date-Duration:
['D033Q11', 'D069Q04', 'D069Q05', 'D069Q06', 'D069Q07', 'D087Q02', 'D105Q02', 'D105Q04', 'D117Q01', 'D117Q02', 'D243Q07', 'D255Q01', 'D267Q07', 'D303Q05']

Kinship:
['D039Q01', 'D039Q02']

Wiki-Json-Inference:
[]

Summarize:
[]

--------------------------------------------------------------------------------