An efficient data preprocessing method for mining customer survey data

Abstract

It is well known that over 80% of the time required to carry out any real world data mining project is usually spent on data preprocessing. Data preprocessing lays the groundwork for data mining. Before the discovery of useful information/knowledge, the target data set must be properly prepared. But it is unfortunately ignored by most researchers on data mining due to its perceived difficulty. This paper describes an efficient approach for data preprocessing for mining Web based customer survey data in order to speed up the data preparation process. The proposed approach is based on a unified data model derived from analysis of the characteristics of the customer survey data. The unified data model is used as a standard representation for the incoming data so that it can be mined. It not only provides flexibility for data preprocessing but also reduce complexity and difficulty of preparation for mining customer survey data. © 2007 IEEE.