Literary and Linguistic Computing Advance Access originally published online on September 1, 2008
Literary and Linguistic Computing 2008 23(3):327-343; doi:10.1093/llc/fqn015
| ||||||||||||||||||||||||||||||||||||||||||||||||||
An evaluation of text classification methods for literary study
Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign, USA
Correspondence: Bei Yu, Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA. E-mail: beiyu.work{at}gmail.com
 |
Abstract |
|---|
This article presents an empirical evaluation of text classification methods in literary domain. This study compared the performance of two popular algorithms, naïve Bayes and support vector machines (SVMs) in two literary text classification tasks: the eroticism classification of Dickinson's poems and the sentimentalism classification of chapters in early American novels. The algorithms were also combined with three text pre-processing tools, namely stemming, stopword removal, and statistical feature selection, to study the impact of these tools on the classifiers’ performance in the literary setting. Existing studies outside the literary domain indicated that SVMs are generally better than naïve Bayes classifiers. However, in this study SVMs were not all winners. Both algorithms achieved high accuracy in sentimental chapter classification, but the naïve Bayes classifier outperformed the SVM classifier in erotic poem classification. Self-feature selection helped both algorithms improve their performance in both tasks. However, the two algorithms selected relevant features in different frequency ranges, and therefore captured different characteristics of the target classes. The evaluation results in this study also suggest that arbitrary feature-reduction steps such as stemming and stopword removal should be taken very carefully. Some stopwords were highly discriminative features for Dickinson's erotic poem classification. In sentimental chapter classification, stemming undermined subsequent feature selection by aggressively conflating and neutralizing discriminative features.