Please use this identifier to cite or link to this item: https://doi.org/10.1109/TPAMI.2008.110
Title: Supervised and traditional term weighting methods for automatic text categorization
Authors: Lan, M. 
Tan, C.L. 
Su, J.
Lu, Y.
Keywords: kNN
SVM
Term weighting
Text categorization
Text representation
Issue Date: 2009
Source: Lan, M.,Tan, C.L.,Su, J.,Lu, Y. (2009). Supervised and traditional term weighting methods for automatic text categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (4) : 721-735. ScholarBank@NUS Repository. https://doi.org/10.1109/TPAMI.2008.110
Abstract: In vector space model (VSM), text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a computer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the performance of text categorization. In this study, we investigate several widely-used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with SVM and kNN algorithms. In consideration of the distribution of relevant documents in the collection, we propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms' discriminating power for text categorization task. From the controlled experimental results, these supervised term weighting methods have mixed performance. Specifically, our proposed supervised term weighting method, tf.rf, has a consistently better performance than other term weighting methods while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets. © 2009 IEEE.
Source Title: IEEE Transactions on Pattern Analysis and Machine Intelligence
URI: http://scholarbank.nus.edu.sg/handle/10635/39323
ISSN: 01628828
DOI: 10.1109/TPAMI.2008.110
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

233
checked on Dec 5, 2017

WEB OF SCIENCETM
Citations

147
checked on Nov 2, 2017

Page view(s)

54
checked on Dec 9, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.