A Consistent Information Criterion for Support Vector Machines in Diverging Model Spaces.

TitleA Consistent Information Criterion for Support Vector Machines in Diverging Model Spaces.
Publication TypeJournal Article
Year of Publication2016
AuthorsZhang, Xiang, Yichao Wu, Lan Wang, and Runze Li
JournalJ Mach Learn Res
Volume17
Issue16
Pagination1-26
Date Published2016
ISSN1532-4435
Abstract

Information criteria have been popularly used in model selection and proved to possess nice theoretical properties. For classification, Claeskens et al. (2008) proposed support vector machine information criterion for feature selection and provided encouraging numerical evidence. Yet no theoretical justification was given there. This work aims to fill the gap and to provide some theoretical justifications for support vector machine information criterion in both fixed and diverging model spaces. We first derive a uniform convergence rate for the support vector machine solution and then show that a modification of the support vector machine information criterion achieves model selection consistency even when the number of features diverges at an exponential rate of the sample size. This consistency result can be further applied to selecting the optimal tuning parameter for various penalized support vector machine methods. Finite-sample performance of the proposed information criterion is investigated using Monte Carlo studies and one real-world gene selection problem.

Alternate JournalJ Mach Learn Res
Original PublicationA consistent information criterion for support vector machines in diverging model spaces.
PubMed ID27239164
PubMed Central IDPMC4883123
Grant ListP50 DA010075 / DA / NIDA NIH HHS / United States
P50 DA039838 / DA / NIDA NIH HHS / United States
R01 CA149569 / CA / NCI NIH HHS / United States
P01 CA142538 / CA / NCI NIH HHS / United States
P50 DA036107 / DA / NIDA NIH HHS / United States