Hard or Soft Classification? Large-margin Unified Machines.

TitleHard or Soft Classification? Large-margin Unified Machines.
Publication TypeJournal Article
Year of Publication2011
AuthorsLiu, Yufeng, Hao Helen Zhang, and Yichao Wu
JournalJ Am Stat Assoc
Volume106
Issue493
Pagination166-177
Date Published2011 Mar 01
ISSN0162-1459
Abstract

Margin-based classifiers have been popular in both machine learning and statistics for classification problems. Among numerous classifiers, some are hard classifiers while some are soft ones. Soft classifiers explicitly estimate the class conditional probabilities and then perform classification based on estimated probabilities. In contrast, hard classifiers directly target on the classification decision boundary without producing the probability estimation. These two types of classifiers are based on different philosophies and each has its own merits. In this paper, we propose a novel family of large-margin classifiers, namely large-margin unified machines (LUMs), which covers a broad range of margin-based classifiers including both hard and soft ones. By offering a natural bridge from soft to hard classification, the LUM provides a unified algorithm to fit various classifiers and hence a convenient platform to compare hard and soft classification. Both theoretical consistency and numerical performance of LUMs are explored. Our numerical study sheds some light on the choice between hard and soft classifiers in various classification problems.

DOI10.1198/jasa.2011.tm10319
Alternate JournalJ Am Stat Assoc
Original PublicationHard or soft classification? Large-margin unified machines.
PubMed ID22162896
PubMed Central IDPMC3233196
Grant ListP01 CA142538-01 / CA / NCI NIH HHS / United States
R01 CA085848 / CA / NCI NIH HHS / United States
R01 CA149569-02 / CA / NCI NIH HHS / United States
R01 CA149569 / CA / NCI NIH HHS / United States
P01 CA142538 / CA / NCI NIH HHS / United States
R01 CA085848-11 / CA / NCI NIH HHS / United States