Summary
Objectives:
To introduce abductive network classifier committees as an ensemble method for improving
classification accuracy in medical diagnosis. While neural networks allow many ways
to introduce enough diversity among member models to improve performance when forming
a committee, the self-organizing, automatic-stopping nature, and learning approach
used by abductive networks are not very conducive for this purpose. We explore ways
of overcoming this limitation and demonstrate improved classification on three standard
medical datasets.
Methods:
Two standard 2-class medical datasets (Pima Indians Diabetes and Heart Disease) and
a 6-class dataset (Dermatology) were used to investigate ways of training abductive
networks with adequate independence, as well as methods of combining their outputs
to form a network that improves performance beyond that of single models.
Results:
Two- or three-member committees of models trained on completely or partially different
subsets of training data and using simple output combination methods achieve improvements
between 2 and 5 percentage points in the classification accuracy over the best single
model developed using the full training set.
Conclusions:
Varying model complexity alone gives abductive network models that are too correlated
to ensure enough diversity for forming a useful committee. Diversity achieved through
training member networks on independent subsets of the training data outweighs limitations
of the smaller training set for each, resulting in net gain in committee performance.
As such models train faster and can be trained in parallel, this can also speed up
classifier development.
Keywords
Abductive networks - neural networks - ensemble methods - network committee - committee
of experts - classification accuracy - medical diagnosis - diabetes - heart disease
- dermatology
