Please use this identifier to cite or link to this item: https://doi.org/10.1109/ICDM.2005.69
Title: Finding representative set from massive data
Authors: Pan, F.
Wang, W.
Tung, A.K.H. 
Yang, J.
Issue Date: 2005
Source: Pan, F.,Wang, W.,Tung, A.K.H.,Yang, J. (2005). Finding representative set from massive data. Proceedings - IEEE International Conference on Data Mining, ICDM : 338-345. ScholarBank@NUS Repository. https://doi.org/10.1109/ICDM.2005.69
Abstract: In the information age, data is pervasive. In some applications, data explosion is a significant phenomenon. The massive data volume poses challenges to both human users and computers. In this project, we propose a new model for identifying representative set from a large database. A representative set is a special subset of the original dataset, which has three main characteristics: It is significantly smaller in size compared to the original dataset. It captures the most information from the original dataset compared to other subsets of the same size. It has low redundancy among the representatives it contains. We use informationtheoretic measures such as mutual information and relative entropy to measure the representativeness of the representative set. We first design a greedy algorithm and then present a heuristic algorithm that delivers much better performance. We run experiments on two real datasets and evaluate the effectiveness of our representative set in terms of coverage and accuracy. The experiments show that our representative set attains expected characteristics and captures information more efficiently. © 2005 IEEE.
Source Title: Proceedings - IEEE International Conference on Data Mining, ICDM
URI: http://scholarbank.nus.edu.sg/handle/10635/40731
ISBN: 0769522785
ISSN: 15504786
DOI: 10.1109/ICDM.2005.69
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

25
checked on Dec 11, 2017

Page view(s)

62
checked on Dec 16, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.