Mining of textual databases within the product development process

Abstract

New trends such as increased product complexity and shortening development time have given rise to an increase in the number of unexpected events in the produce development process. Traditional approaches have been found to be inadequate to address this. Therefore new tools are being sought. This thesis investigates the use of one such tool, data mining, within the product development process. The focus is on the application of data mining techniques to textual databases. Initially, various textual databases within the industry are identified and analyzed. As a particular application, the classification of the textual databases, Call Center database, is examined in detail. Extensive experiments investigating, the optimal design of a classification system, the effect of various term weighting schemes, the use latent semantic analysis and the effectiveness of filter based feature reduction schemes have been conducted. A novel feature selection method using the design of experiment methodology is also proposed.