Quantifying the Impact of Learning Algorithm Parameter Tuning

Niklas Lavesson, Paul Davidsson

The impact of learning algorithm optimization by means of parameter tuning is studied. To do this, two quality attributes, sensitivity and classification performance, are investigated, and two metrics for quantifying each of these attributes are suggested. Using these metrics, a systematic comparison has been performed between four induction algorithms on eight data sets. The results indicate that parameter tuning is often more important than the choice of algorithm and there does not seem to be a trade-off between the two quality attributes. Moreover, the study provides quantitative support to the assertion that some algorithms are more robust than others with respect to parameter configuration. Finally, it is briefly described how the quality attributes and their metrics could be used for algorithm selection in a systematic way.

Subjects: 12. Machine Learning and Discovery

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.