Please use this identifier to cite or link to this item:
|Title:||Finding representative set from massive data||Authors:||Pan, F.
|Issue Date:||2005||Citation:||Pan, F.,Wang, W.,Tung, A.K.H.,Yang, J. (2005). Finding representative set from massive data. Proceedings - IEEE International Conference on Data Mining, ICDM : 338-345. ScholarBank@NUS Repository. https://doi.org/10.1109/ICDM.2005.69||Abstract:||In the information age, data is pervasive. In some applications, data explosion is a significant phenomenon. The massive data volume poses challenges to both human users and computers. In this project, we propose a new model for identifying representative set from a large database. A representative set is a special subset of the original dataset, which has three main characteristics: It is significantly smaller in size compared to the original dataset. It captures the most information from the original dataset compared to other subsets of the same size. It has low redundancy among the representatives it contains. We use informationtheoretic measures such as mutual information and relative entropy to measure the representativeness of the representative set. We first design a greedy algorithm and then present a heuristic algorithm that delivers much better performance. We run experiments on two real datasets and evaluate the effectiveness of our representative set in terms of coverage and accuracy. The experiments show that our representative set attains expected characteristics and captures information more efficiently. © 2005 IEEE.||Source Title:||Proceedings - IEEE International Conference on Data Mining, ICDM||URI:||http://scholarbank.nus.edu.sg/handle/10635/40731||ISBN:||0769522785||ISSN:||15504786||DOI:||10.1109/ICDM.2005.69|
|Appears in Collections:||Staff Publications|
Show full item record
Files in This Item:
There are no files associated with this item.
checked on Apr 13, 2019
checked on Apr 19, 2019
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.