J Korean Soc Med Inform.  2005 Sep;11(3):247-253.

Applying a Mutual Information Theory Based Feature Selection Method to a Classifier

• Lee SM

Abstract

OBJECTIVE
The purpose of this study was to explore the usability of a feature selection method based on the mutual information theory to increase predictive performance of a classifier in data mining.
METHODS
The HIV Cost and Services Utilization Study(HCSUS) dataset was used to apply the feature selection method to a classifier. Its contribution to increasing the predictive performance of the classifier was evaluated by comparing the Naive Bayes(NB) and the Logistic Regression(LG) models using different variables. The infrequent office visit representing limited health service utilization was selected as an outcome variable. HUGIN Researcher(TM) 6.3 was used to train and test the NB models and SAS(R) 8.0 was used for the LG modeling.
RESULTS
Higher AUC in the NB model was obtained using the variables selected by the mutual information based feature selection method(AUC=.639, CI=.611, .660); lower AUC using the variables defined by a previous study(AUC=.599, CI=.570, .620). There was no difference between the LG models with different variables.
CONCLUSION
This study demonstrated the mutual information method may be useful in identifying relevant predictors as the feature selection method, which can contribute to an increase in the predictive performance of a classifier.

Keyword

Feature Selection; Classifier; Naive Bayes; Data Mining; Prediction

MeSH Terms

Area Under Curve
Data Mining
Dataset
Health Services
HIV
Information Theory*
Office Visits