ScholarBank@NUShttps://scholarbank.nus.edu.sgThe DSpace digital repository system captures, stores, indexes, preserves, and distributes digital research material.Fri, 28 Jan 2022 20:15:47 GMT2022-01-28T20:15:47Z50751- Some issues on scalable feature selectionhttps://scholarbank.nus.edu.sg/handle/10635/99415Title: Some issues on scalable feature selection
Authors: Liu, H.; Setiono, R.
Abstract: Feature selection determines relevant features in the data. It is often applied in pattern classification, data mining, as well as machine learning. A special concern for feature selection nowadays is that the size of a database is normally very large, both vertically and horizontally. In addition, feature sets may grow as the data collection process continues. Effective solutions are needed to accommodate the practical demands. This paper concentrates on three issues: large number of features, large data size, and expanding feature set. For the first issue, we suggest a probabilistic algorithm to select features. For the second issue, we present a scalable probabilistic algorithm that expedites feature selection further and can scale up without sacrificing the quality of selected features. For the third issue, we propose an incremental algorithm that adapts to the newly extended feature set and captures 'concept drifts' by removing features from previously selected and newly added ones. We expect that research on scalable feature selection will be extended to distributed and parallel computing and have impact on applications of data mining and machine learning. © 1998 Elsevier Science Ltd. All rights reserved.
Thu, 01 Oct 1998 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/994151998-10-01T00:00:00Z
- Symbolic rule extraction from neural networks An application to identifying organizations adopting IThttps://scholarbank.nus.edu.sg/handle/10635/99424Title: Symbolic rule extraction from neural networks An application to identifying organizations adopting IT
Authors: Setiono, R.; Thong, J.Y.L.; Yap, C.-S.
Abstract: Interest in the application of neural networks as tools for decision support has been growing in recent years. A major drawback often associated with neural networks is the difficulty in understanding the knowledge represented by a trained network. This paper describes an approach that can extract symbolic rules from neural networks. We illustrate how the approach successfully extracted rules from a data set collected from a survey of the service sectors in the United Kingdom. The extracted rules were then used to distinguish between organizations using computers from those that do not. The classification scheme based on these rules was used to identify specific segments of a market for promoting adoption of information technology. The extracted rules are not only concise but also outperform discriminant analysis in terms of predictive accuracy. © 1998 Elsevier Science B.V. All rights reserved.
Thu, 10 Sep 1998 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/994241998-09-10T00:00:00Z
- Rule Extraction from Neural Networks and Support Vector Machines for Credit Scoringhttps://scholarbank.nus.edu.sg/handle/10635/77912Title: Rule Extraction from Neural Networks and Support Vector Machines for Credit Scoring
Authors: Setiono, R.; Baesens, B.; Martens, D.
Abstract: In this chapter we describe how comprehensible rules can be extracted from artificial neural networks (ANN) and support vector machines (SVM). ANN and SVM are two very popular techniques for pattern classification. In the business intelligence application domain of credit scoring, they have been shown to be effective tools for distinguishing between good credit risks and bad credit risks. The accuracy obtained by these two techniques is often higher than that from decision tree methods. Unlike decision tree methods, however, the classifications made by ANN and SVM are difficult to understand by the end-users as outputs from ANN and SVM are computed as nonlinear mapping of the input data attributes. We describe two rule extraction methods that we have developed to overcome this difficulty. These rule extraction methods enable the users to obtain comprehensible propositional rules from ANN and SVM. Such rules can be easily verified by the domain experts and would lead to a better understanding about the data in hand. © Springer-Verlag Berlin Heidelberg 2011.
Sun, 01 Jan 2012 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/779122012-01-01T00:00:00Z
- Improved SOM labeling methodology for data mining applicationshttps://scholarbank.nus.edu.sg/handle/10635/78459Title: Improved SOM labeling methodology for data mining applications
Authors: Azcarraga, A.; Hsieh, M.-H.; Pan, S.-L.; Setiono, R.
Abstract: Self-Organizing Maps (SOMs) have been useful in gaining insights about the information content of large volumes of data in various data mining applications. As a special form of neural networks, they have been attractive as a data mining tool because they are able to extract information from data even with very little user-intervention. However, although learning in self-organizing maps is considered unsupervised because training patterns do not need desired output information to be supplied by the user, a trained SOM often has to be labeled prior to use in many real-world applications. Unfortunately, this labeling phase is usually supervised as patterns need accompanying output information that have to be supplied by the user. Because labeled patterns are not always available or may not even be possible to construct, the supervised nature of the labeling phase restricts the deployment of SOM to a wider range of potential data mining applications. This work proposes a methodical and semi-automatic SOM labeling procedure that does not require a set of labeled patterns. Instead, nodes in the trained map are clustered and subsets of training patterns associated to each of the clustered nodes are identified. Salient dimensions per node cluster, that constitute the basis for labeling each node in the map, are then identified. The effectiveness of the method is demonstrated on a data mining application involving customer-profiling based on an international market segmentation study. © 2008 Springer-Verlag US.
Tue, 01 Jan 2008 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/784592008-01-01T00:00:00Z
- A neural network construction algorithm with application to image compressionhttps://scholarbank.nus.edu.sg/handle/10635/99159Title: A neural network construction algorithm with application to image compression
Authors: Setiono, R.; Lu, G.
Abstract: We propose an algorithm for constructing a feedforward neural network with a single hidden layer. This algorithm is applied to image compression and it is shown to give satisfactory results. The neural network construction algorithm begins with a simple network topology containing a single unit in the hidden layer. An optimal set of weights for this network is obtained by applying a variant of the quasi-Newton method for unconstrained optimisation. If this set of weights does not give a network with the desired accuracy then one more unit is added to the hidden layer and the network is retrained. This process is repeated until the desired network is obtained. We show that each addition of the hidden unit to the network is guaranteed to increase the signal to noise ratio of the compressed image. © 1994 Springer-Verlag London Limited.
Wed, 01 Jun 1994 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/991591994-06-01T00:00:00Z
- A penalty-function approach for pruning feedforward neural networkshttps://scholarbank.nus.edu.sg/handle/10635/99162Title: A penalty-function approach for pruning feedforward neural networks
Authors: Setiono, R.
Abstract: This article proposes the use of a penalty function for pruning feedforward neural network by weight elimination. The penalty function proposed consists of two terms. The first term is to discourage the use of unnecessary connections, and the second term is to prevent the weights of the connections from taking excessively large values. Simple criteria for eliminating weights from the network are also given. The effectiveness of this penalty function is tested on three well-known problems: the contiguity problem, the parity problems, and the monks problems. The resulting pruned networks obtained for many of these problems have fewer connections than previously reported in the literature.
Wed, 01 Jan 1997 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/991621997-01-01T00:00:00Z
- Analysis of Hidden Representations by Greedy Clusteringhttps://scholarbank.nus.edu.sg/handle/10635/99189Title: Analysis of Hidden Representations by Greedy Clustering
Authors: Setiono, R.; Liu, H.
Abstract: The hidden layer of backpropagation neural networks (NNs) holds the key to the networks' success in solving pattern classification problems. The units in the hidden layer encapsulate the network's internal representations of the outside world described by the input data. In this paper, the hidden representations of trained networks are investigated by means of a simple greedy clustering algorithm. This clustering algorithm is applied to networks that have been trained to solve well-known problems: the monks problems, the 5-bit parity problem and the contiguity problem. The results from applying the algorithm to problems with known concepts provide us with a better understanding of NN learning. These results also explain why NNs achieve higher predictive accuracy than that of decision-tree methods. The results of this study can be readily applied to rule extraction from NNs. Production rules are extracted for the parity and the monks problems, as well as for a benchmark data set: Pima Indian diabetes diagnosis. The extracted rules from the Pima Indian diabetes data set compare favorably with rules extracted from ARTMAP NNs in terms of predictive accuracy and simplicity.
Sun, 01 Mar 1998 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/991891998-03-01T00:00:00Z
- Efficient neural network training algorithm for the Cray Y-MP supercomputerhttps://scholarbank.nus.edu.sg/handle/10635/99507Title: Efficient neural network training algorithm for the Cray Y-MP supercomputer
Authors: Leung, Chung Siu; Setiono, Rudy
Abstract: An efficient implementation of a quasi-newton algorithm for feedforward neural network training on a Cray Y-MP is presented. The most timeconsuming step of a neural network training using the quasi-Newton algorithm is the computation of the error function and its gradient. We describe in this paper how this step can be implemented so that the neural network training may take full advantage of the Cray vectorization capabilities.
Fri, 01 Jan 1993 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/995071993-01-01T00:00:00Z
- Fragmentation problem and automated feature constructionhttps://scholarbank.nus.edu.sg/handle/10635/99521Title: Fragmentation problem and automated feature construction
Authors: Setiono, Rudy; Liu, Huan
Abstract: Selective induction algorithms are efficient in learning target concepts but inherit a major limitation - each time only one feature is used to partition the data until the data is divided into uniform segments. This limitation results in problems like replication, repetition, and fragmentation. Constructive induction has been an effective means to overcome some of the problems. The underlying idea is to construct compound features that increase the representation power so as to enhance the learning algorithm 's capability in partitioning data. Unfortunately, many constructive operators are often manually designed and choosing which one to apply poses a serious problem itself. We propose an automatic way of constructing compound features. The method can be applied to both continuous and discrete data and thus all the three problems can be eliminated or alleviated. Our empirical results indicate the effectiveness of the proposed method.
Thu, 01 Jan 1998 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/995211998-01-01T00:00:00Z
- Chi2: feature selection and discretization of numeric attributeshttps://scholarbank.nus.edu.sg/handle/10635/99483Title: Chi2: feature selection and discretization of numeric attributes
Authors: Liu, Huan; Setiono, Rudy
Abstract: Discretization can turn numeric attributes into discrete ones. Feature selection can eliminate some irrelevant attributes. This paper describes Chi2, a simple and general algorithm that uses the X2 statistic to discretize numeric attributes repeatedly until some inconsistencies are found in the data, and achieves feature selection via discretization. The empirical results demonstrate that Chi2 is effective in feature selection and discretization of numeric and ordinal attributes.
Sun, 01 Jan 1995 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/994831995-01-01T00:00:00Z
- Image compression using a feedforward neural networkhttps://scholarbank.nus.edu.sg/handle/10635/99531Title: Image compression using a feedforward neural network
Authors: Setiono, Rudy; Lu, Guojun
Abstract: We present an image compression technique using a feedforward neural network. The neural network has three layers: one input layer, one hidden layer and one output layer. The inputs of the neural network are original image data, while the outputs are reconstructed image data which are close to the inputs. If the amount of data required to store the hidden unit values and the connection weights to the output layer of the neural network is less than the original data, compression is achieved. In our experiments, we achieved a compression ratio of about 10 with good reconstructed image quality. The neural network construction algorithm we used has an attractive feature that each addition of a hidden unit to the network will always improve the image quality. Thus our compression scheme is flexible in the sense that the user can trade between image quality and compression ratio depending on the application requirements.
Sat, 01 Jan 1994 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/995311994-01-01T00:00:00Z
- Interior dual proximal point algorithm for linear programshttps://scholarbank.nus.edu.sg/handle/10635/99318Title: Interior dual proximal point algorithm for linear programs
Authors: Setiono, R.
Abstract: A new algorithm for solving a linear program based on an interior point method applied to the dual of a proximal point formulation of the linear program is presented. This dual formulation contains only the nonnegativity constraint on some of the variables. This simple constraint allows us to start the algorithm without a Phase 1 method required by many other variants of the interior point method. Numerical results from a large set of test problems show that the proposed algorithm can be very competitive with other interior point methods and with MINOS 5.3, a state-of-the-art linear programming package based on the simplex method. Global convergence of the algorithm is also established. © 1994.
Thu, 25 Aug 1994 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/993181994-08-25T00:00:00Z
- Improving backpropagation learning with feature selectionhttps://scholarbank.nus.edu.sg/handle/10635/99308Title: Improving backpropagation learning with feature selection
Authors: Setiono, R.; Liu, H.
Abstract: There exist redundant, irrelevant and noisy data. Using proper data to train a network can speed up training, simplify the learned structure, and improve its performance. A two-phase training algorithm is proposed. In the first phase, the number of input units of the network is determined by using an information base method. Only those attributes that meet certain criteria for inclusion will be considered as the input to the network. In the second phase, the number of hidden units of the network is selected automatically based on the performance of the network on the training data. One hidden unit is added at a time only if it is necessary. The experimental results show that this new algorithm can achieve a faster learning time, a simpler network and an improved performance. © 1996 Kluwer Academic Publishers.
Mon, 01 Jan 1996 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/993081996-01-01T00:00:00Z
- Extracting rules from neural networks by pruning and hidden-unit splittinghttps://scholarbank.nus.edu.sg/handle/10635/99280Title: Extracting rules from neural networks by pruning and hidden-unit splitting
Authors: Setiono, R.
Abstract: An algorithm for extracting rules from a standard three-layer feedforward neural network is proposed. The trained network is first pruned not only to remove redundant connections in the network but, more important, to detect the relevant inputs. The algorithm generates rules from the pruned network by considering only a small number of activation values at the hidden units. If the number of inputs connected to a hidden unit is sufficiently small, then rules that describe how each of its activation values is obtained can be readily generated. Otherwise the hidden unit will be split and treated as output units, with each output unit corresponding to an activation value. A hidden layer is inserted and a new subnetwork is formed, trained, and pruned. This process is repeated until every hidden unit in the network has a relatively small number of input units connected to it. Examples on how the proposed algorithm works are shown using real-world data arising from molecular biology and signal processing. Our results show that for these complex problems, the algorithm can extract reasonably compact rule sets that have high predictive accuracy rates.
Wed, 01 Jan 1997 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/992801997-01-01T00:00:00Z
- Some n-bit parity problems are solvable by feed-forward networks with less than n hidden unitshttps://scholarbank.nus.edu.sg/handle/10635/99596Title: Some n-bit parity problems are solvable by feed-forward networks with less than n hidden units
Authors: Setiono, Rudy; Hui, Lucas Chi Kwong
Abstract: Starting with two hidden units, we train a simple single hidden layer feed-forward neural network to solve the n-bit parity problem. If the network fails to recognize correctly all the input patterns, an additional hidden unit is added to the hidden layer and the network is retrained. This process is repeated until a network that correctly classifies all the input patterns has been constructed. Using a variant of the quasi-Newton methods for training, we have been able to find networks with a single layer containing less than n hidden units that solve the n-bit parity problem for some value of n. This proves the power of combining quasi-Newton method and node incremental approach.
Fri, 01 Jan 1993 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/995961993-01-01T00:00:00Z
- Incremental Feature Selectionhttps://scholarbank.nus.edu.sg/handle/10635/52990Title: Incremental Feature Selection
Authors: Liu, H.; Setiono, R.
Abstract: Feature selection is a problem of finding relevant features. When the number of features of a dataset is large and its number of patterns is huge, an effective method of feature selection can help in dimensionality reduction. An incremental probabilistic algorithm is designed and implemented as an alternative to the exhaustive and heuristic approaches. Theoretical analysis is given to support the idea of the probabilistic algorithm in finding an optimal or near-optimal subset of features. Experimental results suggest that (1) the probabilistic algorithm is effective in obtaining optimal/suboptimal feature subsets; (2) its incremental version expedites feature selection further when the number of patterns is large and can scale up without sacrificing the quality of selected features.
Thu, 01 Jan 1998 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/529901998-01-01T00:00:00Z
- A comparative study of centroid-based, neighborhood-based and statistical approaches for effective document categorizationhttps://scholarbank.nus.edu.sg/handle/10635/42622Title: A comparative study of centroid-based, neighborhood-based and statistical approaches for effective document categorization
Authors: Tam, V.; Santoso, A.; Setiono, R.
Abstract: Associating documents to relevant categories is critical for effective document retrieval. Here, we compare the well-known k-Nearest Neighborhood (kNN) algorithm, the centroid-based classifier and the Highest Average Similarity over Retrieved Documents (HASRD) algorithm, for effective document categorization. We use various measures such as the micro and macro F1 values to evaluate their performance on the Reuters-21578 corpus. The empirical results show that kNN performs the best, followed by our adapted HASRD and the centroid-based classifier for common document categories, while the centroid-based classifier and kNN outperform our adapted HASRD for rare document categories. Additionally, our study clearly indicates that each classifier performs optimally only when a suitable term weighting scheme is used. All these significant results lead to many exciting directions for future exploration. © 2002 IEEE.
Tue, 01 Jan 2002 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/426222002-01-01T00:00:00Z
- Knowledge acquisition and revision via neural networkshttps://scholarbank.nus.edu.sg/handle/10635/42637Title: Knowledge acquisition and revision via neural networks
Authors: Azcarraga, A.; Hsieh, M.; Pan, S.-L.; Setiono, R.
Abstract: We investigate how knowledge acquired by a neural network from one input environment can be transferred and revised for similar application in a new environment. Knowledge revision is achieved by re-training the neural network. Knowledge common to both environments are retained, while localized knowledge components are introduced during network retraining. Various network performance measures are computed to measure how much knowledge is transferred and revised. Furthermore, because the knowledge acquired by a neural network can be expressed as an accurate set of simple rules, we are able to compare knowledge extracted from one network with that from another. In a cross-national study of car image perceptions, a comparison of the original and revised knowledge gives us insights into the commonalities and differences in brand perceptions across countries.
Thu, 01 Jan 2004 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/426372004-01-01T00:00:00Z
- Neural network pruning for function approximationhttps://scholarbank.nus.edu.sg/handle/10635/42620Title: Neural network pruning for function approximation
Authors: Setiono, Rudy; Gaweda, Adam
Abstract: A simple algorithm for pruning feedforward neural networks with a single hidden layer trained for function approximation is presented. The algorithm assumes that the networks have been trained with more then the necessary number of hidden units and it consists of two stages. In the first stage, redundant hidden units are removed and in the second stage, irrelevant input units are removed. Experimental results on seven publicly available data sets show that the proposed algorithm outperforms other methods such as nearest neighbor-, decision tree- and regression-based methods.
Sat, 01 Jan 2000 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/426202000-01-01T00:00:00Z
- Keyword extraction using backpropagation neural networks and rule extractionhttps://scholarbank.nus.edu.sg/handle/10635/42638Title: Keyword extraction using backpropagation neural networks and rule extraction
Authors: Azcarraga, A.; Liu, M.D.; Setiono, R.
Abstract: Keyword extraction is vital for Knowledge Management System, Information Retrieval System, and Digital Libraries as well as for general browsing of the web. Keywords are often the basis of document processing methods such as clustering and retrieval since processing all the words in the document can be slow. Common models for automating the process of keyword extraction are usually done by using several statistics-based methods such as Bayesian, K-Nearest Neighbor, and Expectation-Maximization. These models are limited by word-related features that can be used since adding more features will make the models more complex and difficult to comprehend. In this research, a Neural Network, specifically a backpropagation network, will be used in generalizing the relationship of the title and the content of articles in the archive by following word features other than TF-IDF, such as position of word in the sentence, paragraph, or in the entire document, and formats such as heading, and other attributes defined beforehand. In order to explain how the backpropagation network works, a rule extraction method will be used to extract symbolic data from the resulting backpropagation network. The rules extracted can then be transformed into decision trees performing almost as accurate as the network plus the benefit of being in an easily comprehensible format. © 2012 IEEE.
Sun, 01 Jan 2012 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/426382012-01-01T00:00:00Z
- Rule extraction from support vector machines: An overview of issues and application in credit scoringhttps://scholarbank.nus.edu.sg/handle/10635/42574Title: Rule extraction from support vector machines: An overview of issues and application in credit scoring
Authors: Martens, D.; Huysmans, J.; Setiono, R.; Vanthienen, J.; Baesens, B.
Abstract: Innovative storage technology and the rising popularity of the Internet have generated an ever-growing amount of data. In this vast amount of data much valuable knowledge is available, yet it is hidden. The Support Vector Machine (SVM) is a state-of-the-art classification technique that generally provides accurate models, as it is able to capture non-linearities in the data. However, this strength is also its main weakness, as the generated non-linear models are typically regarded as incomprehensible black-box models. By extracting rules that mimic the black box as closely as possible, we can provide some insight into the logics of the SVM model. This explanation capability is of crucial importance in any domain where the model needs to be validated before being implemented, such as in credit scoring (loan default prediction) and medical diagnosis. If the SVM is regarded as the current state-of-the-art, SVM rule extraction can be the state-of-the-art of the (near) future. This chapter provides an overview of recently proposed SVM rule extraction techniques, complemented with the pedagogical Artificial Neural Network (ANN) rule extraction techniques which are also suitable for SVMs. Issues related to this topic are the different rule outputs and corresponding rule expressiveness; the focus on high dimensional data as SVM models typically perform well on such data; and the requirement that the extracted rules are in line with existing domain knowledge. These issues are explained and further illustrated with a credit scoring case, where we extract a Trepan tree and a RIPPER rule set from the generated SVM model. The benefit of decision tables in a rule extraction context is also demonstrated. Finally, some interesting alternatives for SVM rule extraction are listed. © 2008 Springer-Verlag Berlin Heidelberg.
Tue, 01 Jan 2008 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/425742008-01-01T00:00:00Z
- Using neural network rule extraction and decision tables for credit-risk evaluationhttps://scholarbank.nus.edu.sg/handle/10635/42528Title: Using neural network rule extraction and decision tables for credit-risk evaluation
Authors: Baesens, B.; Setiono, R.; Mues, C.; Vanthienen, J.
Abstract: Credit-risk evaluation is a very challenging and important management science problem in the domain of financial analysis. Many classification methods have been suggested in the literature to tackle this problem. Neural networks, especially, have received a lot of attention because of their universal approximation property. However, a major drawback associated with the use of neural networks for decision making is their lack of explanation capability. While they can achieve a high predictive accuracy rate, the reasoning behind how they reach their decisions is not readily available. In this paper, we present the results from analysing three real-life credit-risk data sets using neural network rule extraction techniques. Clarifying the neural network decisions by explanatory rules that capture the learned knowledge embedded in the networks can help the credit-risk manager in explaining why a particular applicant is classified as either bad or good. Furthermore, we also discuss how these rules can be visualized as a decision table in a compact and intuitive graphical format that facilitates easy consultation. It is concluded that neural network rule extraction and decision tables are powerful management tools that allow us to build advanced and userfriendly decision-support systems for credit-risk evaluation.
Wed, 01 Jan 2003 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/425282003-01-01T00:00:00Z
- Greedy rule generation from discrete data and its use in neural network rule extractionhttps://scholarbank.nus.edu.sg/handle/10635/42798Title: Greedy rule generation from discrete data and its use in neural network rule extraction
Authors: Odajima, K.; Hayashi, Y.; Setiono, R.
Abstract: This paper proposes GRG (Greedy Rule Generation) algorithm for generating classification rules from a data set with discrete attributes. The algorithm is "greedy" in the sense that at every iteration, it searches for the best rule to generate. The criteria for the best rule include the number of samples that it covers, the number of attributes involved in the rule, and the size of the input subspace it covers. This method is applied for extracting rules from neural networks that have been trained and pruned for solving classification problems. Neural networks with one hidden layer are trained and the proposed GRG algorithm is applied to their discretized hidden unit activation values. Our results show that rule extraction with the GRG method produces rule sets that are more accurate and concise compared to those obtained by a decision tree method and an existing neural network rule extraction method. © 2006 IEEE.
Sun, 01 Jan 2006 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/427982006-01-01T00:00:00Z
- Special issue of the IEEE transactions on neural networks: White box nonlinear prediction modelshttps://scholarbank.nus.edu.sg/handle/10635/42873Title: Special issue of the IEEE transactions on neural networks: White box nonlinear prediction models
Authors: Baesens, B.; Martens, D.; Setiono, R.; Zurada, J.
Fri, 01 Jan 2010 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428732010-01-01T00:00:00Z
- Special issue of IEEE transactions on neural networks: White box non-linear prediction modelshttps://scholarbank.nus.edu.sg/handle/10635/42871Title: Special issue of IEEE transactions on neural networks: White box non-linear prediction models
Authors: Bart, B.; David, M.; Rudy, S.; Jacek, Z.
Fri, 01 Jan 2010 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428712010-01-01T00:00:00Z
- Special issue on white box nonlinear prediction modelshttps://scholarbank.nus.edu.sg/handle/10635/42867Title: Special issue on white box nonlinear prediction models
Authors: Baesens, B.; Martens, D.; Setiono, R.; Zurada, J.
Fri, 01 Jan 2010 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428672010-01-01T00:00:00Z
- A hybrid SOM-SVM method for analyzing zebra fish gene expressionhttps://scholarbank.nus.edu.sg/handle/10635/43003Title: A hybrid SOM-SVM method for analyzing zebra fish gene expression
Authors: Wu, W.; Xin, L.; Min, X.; Jinrong, P.; Setiono, R.
Abstract: Microarray technology can be employed to quantitatively measure the expression of thousands of genes in a single experiment. It has become one of the main tools for global gene expression analysis in molecular biology research in recent years. The large amount of expression data generated by this technology makes the study of certain complex biological problems possible, and machine learning methods are expected to play a crucial role in the analysis process. We present our results from integrating a Self-Organizing Maps (SOM) and a Support Vector Machine (SVM) for the analysis of the various functions of Zebra fish genes based on their expression. We discuss how SOM can be used as a data-filtering tool to improve the classification performance of the SVM on this data set.
Thu, 01 Jan 2004 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/430032004-01-01T00:00:00Z
- Effective neural network pruning using cross-validationhttps://scholarbank.nus.edu.sg/handle/10635/42824Title: Effective neural network pruning using cross-validation
Authors: Huynh, T.Q.; Setiono, R.
Abstract: This paper addresses the problem of finding neural networks with optimal topology such that their generalization capability is maximized. Our approach is to combine the use of a penalty function during network training and a subset of the training samples for cross-validation. The penalty is added to the error function so that the weights of network connections that are not useful have small magnitude. Such network connections can be pruned if the resulting accuracy of the network does not change beyond a preset level. Training samples in the cross-validation set are used to indicate when network pruning is terminated. Our results on 32 publicly available data sets show that the proposed method outperforms existing neural network and decision tree methods for classification. © 2005 IEEE.
Sat, 01 Jan 2005 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428242005-01-01T00:00:00Z
- Risk management and regulatory compliance: A data mining framework based on neural network rule extractionhttps://scholarbank.nus.edu.sg/handle/10635/42846Title: Risk management and regulatory compliance: A data mining framework based on neural network rule extraction
Authors: Setiono, R.; Mues, C.; Baesens, B.
Abstract: The recent introduction of various regulatory standards such as Basel II, Sarbanes-Oxley, and IFRS stimulates the need to develop new types of information systems based on data mining that will help improve the quality and automation of the decisions that need to be taken. Although neural networks have been frequently adopted in data mining applications, their opacity and black box character prevents them from being used to develop white box, comprehensible information systems for decision support in a financial context. In this paper, we introduce a new neural network rule extraction algorithm, Re-RX, that can be efficiently adopted to develop a data mining system for risk management in a Basel II context. The novelty of the algorithm lies in its new way of simultaneously working with discrete and continuous attributes without a need for discretization. Having extracted the Re-RX rules, we discuss how they can be used to build Basel II-compliant ICT systems taking into account the operational and regulatory requirements.
Sun, 01 Jan 2006 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428462006-01-01T00:00:00Z
- From knowledge discovery to implementation: A business intelligence approach using neural network rule extraction and decision tableshttps://scholarbank.nus.edu.sg/handle/10635/42848Title: From knowledge discovery to implementation: A business intelligence approach using neural network rule extraction and decision tables
Authors: Mues, C.; Baesens, B.; Setiono, R.; Vanthienen, J.
Abstract: The advent of knowledge discovery in data (KDD) technology has created new opportunities to analyze huge amounts of data. However, in order for this knowledge to be deployed, it first needs to be validated by the end-users and then implemented and integrated into the existing business and decision support environment. In this paper, we propose a framework for the development of business intelligence (BI) systems which centers on the use of neural network rule extraction and decision tables. Two different types of neural network rule extraction algorithms, viz. Neurolinear and Neurorule, are compared, and subsequent implementation strategies based on decision tables are discussed. © Springer-Verlag Berlin Heidelberg 2005.
Sat, 01 Jan 2005 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428482005-01-01T00:00:00Z
- FERNN: an algorithm for fast extraction of rules from neural networkshttps://scholarbank.nus.edu.sg/handle/10635/42905Title: FERNN: an algorithm for fast extraction of rules from neural networks
Authors: Setiono, R.; Leow, W.K.
Abstract: Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network with a single hidden layer, FERNN first identifies the relevant hidden units by computing their information gains. For each relevant hidden unit, its activation values is divided into two subintervals such that the information gain is maximized. FERNN finds the set of relevant network connections from the input units to this hidden unit by checking the magnitudes of their weights. The connections with large weights are identified as relevant. Finally, FERNN generates rules that distinguish the two subintervals of the hidden activation values in terms of the network inputs. Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.
Sat, 01 Jan 2000 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429052000-01-01T00:00:00Z
- An effective method for generating multiple linear regression rules from artificial neural networkshttps://scholarbank.nus.edu.sg/handle/10635/42916Title: An effective method for generating multiple linear regression rules from artificial neural networks
Authors: Setiono, R.; Azcarraga, A.
Abstract: We describe a method for multivariate function approximation which combines neural network learning, clustering and multiple regression. Neural networks with a single hidden layer are universal function approximators. However, due to the complexity of the network topology and the nonlinear transfer function used in computing the activation of the hidden units, the predictions of a trained network are difficult to comprehend. On the other hand, predictions from a multiple linear regression equation are easy to understand but not accurate when the underlying relationship between the input variables and the output variable is nonlinear. The method presented in this paper generates a set of multiple linear regression equations using neural networks. The number of regression equations is determined by clustering the weighted input variables. The predictions for samples in the same cluster are computed by the same regression equation. Experimental results on real-world data demonstrate that the new method generates relatively few regression equations from the training data samples. The errors in prediction using these equations are comparable to or lower than those achieved by existing function approximation methods.
Mon, 01 Jan 2001 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429162001-01-01T00:00:00Z
- Generating rules from trained network using fast pruninghttps://scholarbank.nus.edu.sg/handle/10635/42915Title: Generating rules from trained network using fast pruning
Authors: Setiono, Rudy; Leow, Wee Kheng
Abstract: Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network, FERNN first identifies the relevant hidden units by computing their information gains. Next, it identifies relevant connections from the input units to the relevant hidden units by checking the magnitudes of their weights. Finally, FERNN generates rules based on the relevant hidden units and weights. Our experimental results show that the size and accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.
Fri, 01 Jan 1999 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429151999-01-01T00:00:00Z
- Pruned neural networks for regressionhttps://scholarbank.nus.edu.sg/handle/10635/42919Title: Pruned neural networks for regression
Authors: Setiono, R.; Leow, W.K.
Abstract: Neural networks have been widely used as a tool for regression. They are capable of approximating any function and they do not require any assumption about the distribution of the data. The most commonly used architectures for regression are the feedforward neural networks with one or more hidden layers. In this paper, we present a network pruning algorithm which determines the number of units in the input and hidden layers of the networks. We compare the performance of the pruned networks to four regression methods namely, linear regression (LR), Naive Bayes (NB), k-nearest-neighbor (kNN), and a decision tree predictor M5. On 32 publicly available data sets tested, the neural network method outperforms NB and kNN if the prediction errors are computed in terms of the root mean squared errors. Under this measurement metric, it also performs as well as LR and M5. On the other hand, using the mean absolute error as the measurement metric, the neural network method outperforms all four other regression methods. © Springer-Verlag Berlin Heidelberg 2000.
Sat, 01 Jan 2000 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429192000-01-01T00:00:00Z
- Applying the conjugate gradient method for text document categorizationhttps://scholarbank.nus.edu.sg/handle/10635/42750Title: Applying the conjugate gradient method for text document categorization
Authors: Tam, V.; Setiono, R.; Santoso, A.
Abstract: In this paper, we investigate the effectiveness of two different methods to solve the linear least squares fit (LLSF) problem for document categorization. The first method is the Singular Value Decomposition (SVD) method that has been previously used to solve the document categorization problem. The second method is the Conjugate Gradient (CG) method that is one of the most effective algorithms for solving a linear equation problem. However, up to our knowledge, the CG method has never been applied to handle the document classification, problem. Therefore, we compare the effectiveness of these two LLSF methods to categorize text documents. In addition, we examine the effect of using different term weighting schemes on their performance for document classification. Lastly, we compare the performance of the LLSF classifiers against the neighborhood-based Dt-kNN classifier, our best variant of the kNN classifier integrated with a dynamic threshold scheme, on the Reuters 21578 dataset. Besides being the first proposal to use the CG method for document classification, our work opens up many exciting directions for future investigation.
Thu, 01 Jan 2004 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/427502004-01-01T00:00:00Z
- A connectionist approach to generating oblique decision treeshttps://scholarbank.nus.edu.sg/handle/10635/42903Title: A connectionist approach to generating oblique decision trees
Authors: Setiono, R.; Liu, H.
Abstract: Neural networks and decision tree methods are two common approaches to pattern classification. While neural networks can achieve high predictive accuracy rates, the decision boundaries they form are highly nonlinear and generally difficult to comprehend. Decision trees, on the other hand, can be readily translated into a set of rules. In this paper, we present a novel algorithm for generating oblique decision trees that capitalizes on the strength of both approaches. Oblique decision trees classify the patterns by testing on linear combinations of the input attributes. As a result, an oblique decision tree is usually much smaller than the univariate tree generated for the same domain. Our algorithm consists of two components: connectionist and symbolic. A three-layer feedforward neural network is constructed and pruned, a decision tree is then built from the hidden unit activation values of the pruned network. An oblique decision tree is obtained by expressing the activation values using the original input attributes. We test our algorithm on a wide range of problems. The oblique decision trees generated by the algorithm preserve the high accuracy of the neural networks, while keeping the explicitness of decision trees. Moreover, they outperform univariate decision trees generated by the symbolic approach and oblique decision trees built by other approaches in accuracy and tree size. © 1999 IEEE.
Fri, 01 Jan 1999 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429031999-01-01T00:00:00Z
- Feedforward neural network construction using cross validationhttps://scholarbank.nus.edu.sg/handle/10635/42526Title: Feedforward neural network construction using cross validation
Authors: Setiono, R.
Abstract: This article presents an algorithm that constructs feedforward neural networks with a single hidden layer for pattern classification. The algorithm starts with a small number of hidden units in the network and adds more hidden units as needed to improve the network's predictive accuracy. To determine when to stop adding new hidden units, the algorithm makes use of a subset of the available training samples for cross validation. New hidden units are added to the network only if they improve the classification accuracy of the network on the training samples and on the cross-validation samples. Extensive experimental results show that the algorithm is effective in obtaining networks with predictive accuracy rates that are better than those obtained by state-of-the-art decision tree methods.
Mon, 01 Jan 2001 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/425262001-01-01T00:00:00Z
- Knowledge acquisition and revision using neural networks: An application to a cross-national study of brand image perceptionhttps://scholarbank.nus.edu.sg/handle/10635/42398Title: Knowledge acquisition and revision using neural networks: An application to a cross-national study of brand image perception
Authors: Setiono, R.; Pan, S.-L.; Hsieh, M.-H.; Azcarraga, A.
Abstract: A three-tier knowledge management approach is proposed in the context of a cross-national study of car brand and corporate image perceptions. The approach consists of knowledge acquisition, transfer and revision using neural networks. We investigate how knowledge acquired by a neural network from one car market can be exploited and applied in another market. This transferred knowledge is subsequently revised for application in the new market. Knowledge revision is achieved by re-training the neural network. Core knowledge common to both markets is retained while some localized knowledge components are introduced during network re-training. Since the knowledge acquired by a neural network can be expressed as an accurate set of simple rules, we are able to compare the knowledge extracted from one network with the knowledge extracted from another. Comparison of the originally acquired knowledge with the revised knowledge provides us with insights into the commonalities and differences in car brand and corporate perceptions across national markets. © 2006 Operational Research Society Ltd. All rights reserved.
Sun, 01 Jan 2006 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/423982006-01-01T00:00:00Z
- Discrete variable generation for improved neural network classificationhttps://scholarbank.nus.edu.sg/handle/10635/78099Title: Discrete variable generation for improved neural network classification
Authors: Setiono, R.; Seret, A.
Abstract: Neural networks are widely used for classification as they achieve good predictive accuracy. When the class labels are determined by complex interactions of the input variables, neural networks can be expected to provide better predictions than methods that test on the values of one variable at a time such as univariate decision tree classifiers. On the other hand, when no or relatively simple interaction between variables determines the class membership, the neural network may over fit the data and the input-to-output relationship in the data is represented by a function that is more complex than it should be. In this paper, we propose adding discretized values of the continuous variables in the data as input when training the neural networks. Finding out whether the discretized values or the original continuous values of the variables are useful is achieved by pruning. By having only the relevant inputs left in the pruned networks, we are able to extract classification rules from these networks that are accurate, concise and interpretable. © 2012 IEEE.
Sun, 01 Jan 2012 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/780992012-01-01T00:00:00Z
- Predicting consumer preference for fast-food franchises: A data mining approachhttps://scholarbank.nus.edu.sg/handle/10635/42575Title: Predicting consumer preference for fast-food franchises: A data mining approach
Authors: Hayashi, Y.; Hsieh, M.-H.; Setiono, R.
Abstract: The objectives of the study reported in this paper are: (1) to evaluate the adequacy of two data mining techniques, decision tree and neural network in analysing consumer preference for a fast-food franchise and (2) to examine the sufficiency of the criteria selected in understanding this preference. We build decision tree and neural network models to fit data samples collected from 800 respondents in Taiwan to understand the factors that determine their brand preference. Classification rules are generated from these models to differentiate between consumers who prefer the brand and those who do not. The generated rules show that while both decision tree and neural network models can achieve predictive accuracy of more than 80% on the training data samples and more that 70% on the cross-validation data samples, the neural network models compare very favourably to a decision tree model in rule complexity and the numbers of relevant input attributes. © 2009 Operational Research Society Ltd.
Thu, 01 Jan 2009 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/425752009-01-01T00:00:00Z
- Recursive neural network rule extraction for data with mixed attributeshttps://scholarbank.nus.edu.sg/handle/10635/42577Title: Recursive neural network rule extraction for data with mixed attributes
Authors: Setiono, R.; Baesens, B.; Mues, C.
Abstract: In this paper, we present a recursive algorithm for extracting classification rules from feedforward neural networks (NNs) that have been trained on data sets having both discrete and continuous attributes. The novelty of this algorithm lies in the conditions of the extracted rules: the rule conditions involving discrete attributes are disjoint from those involving continuous attributes. The algorithm starts by first generating rules with discrete attributes only to explain the classification process of the NN. If the accuracy of a rule with only discrete attributes is not satisfactory, the algorithm refines this rule by recursively generating more rules with discrete attributes not already present in the rule condition, or by generating a hyperplane involving only the continuous attributes. We show that for three real-life credit scoring data sets, the algorithm generates rules that are not only more accurate but also more comprehensible than those generated by other NN rule extraction methods. © 2007 IEEE.
Tue, 01 Jan 2008 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/425772008-01-01T00:00:00Z
- Combining neural network predictions for medical diagnosishttps://scholarbank.nus.edu.sg/handle/10635/42403Title: Combining neural network predictions for medical diagnosis
Authors: Hayashi, Y.; Setiono, R.
Abstract: We present our results from combining the predictions of an ensemble of neural networks for the diagnosis of hepatobiliary disorders. To improve the accuracy of the diagnosis, we train the second level networks using the outputs of the first level networks as input data. The second level networks achieve an accuracy that is higher than that of the individual networks in the first level. Compared to the simple method which averages the outputs of the first level networks, the second level networks are also more accurate. We discuss how the overall predictive accuracy can be improved by introducing bias during the training of the level one networks. © 2002 Elsevier Science Ltd. All rights reserved.
Tue, 01 Jan 2002 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/424032002-01-01T00:00:00Z
- Effective query size estimation using neural networkshttps://scholarbank.nus.edu.sg/handle/10635/42908Title: Effective query size estimation using neural networks
Authors: Lu, H.; Setiono, R.
Abstract: This paper describes a novel approach to estimate the size of database query results using neural networks. Using the proposed approach, three layer neural networks are constructed and trained to learn the cumulative distribution functions of attribute values in relations. With a trained network, the estimation of the query result size could be obtained instantly by simply computing the network output from the given query predicates. The basic computational model using a cumulative distribution function to compute the query result size is described. The network construction and training is discussed. Comprehensive experiments were conducted to study the effectiveness of the proposed approach. The results indicate that the approach produces estimates with accuracies that are comparable with or higher than those reported in the literature.
Tue, 01 Jan 2002 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429082002-01-01T00:00:00Z
- Neural-network feature selectorhttps://scholarbank.nus.edu.sg/handle/10635/99624Title: Neural-network feature selector
Authors: Setiono, R.; Liu, H.
Abstract: Feature selection is an integral part of most learning algorithms. Due to the existence of irrelevant and redundant attributes, by selecting only the relevant attributes of the data, higher predictive accuracy can be expected from a machine learning method. In this paper, we propose the use of a three-layer feedforward neural network to select those input attributes that are most useful for discriminating classes in a given set of input patterns. A network pruning algorithm is the foundation of the proposed algorithm. By adding a penalty term to the error function of the network, redundant network connections can be distinguished from those relevant ones by their small weights when the network training process has been completed. A simple criterion to remove an attribute based on the accuracy rate of the network is developed. The network is retrained after removal of an attribute, and the selection process is repeated until no attribute meets the criterion for removal. Our experimental results suggest that the proposed method works very well on a wide variety of classification problems. © 1997 IEEE.
Wed, 01 Jan 1997 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/996241997-01-01T00:00:00Z
- Explanation of the 'virtual input' phenomenonhttps://scholarbank.nus.edu.sg/handle/10635/42904Title: Explanation of the 'virtual input' phenomenon
Authors: Leow, W.K.; Setiono, R.
Abstract: We write this letter to comment on the 'virtual input' phenomenon reported by Thaler (Neural Networks, 8(1) (1995) 55-65). The author attributed the phenomenon to the network's ability to perform pattern classification and completion, and reported that pruning probability affects the number of virtual inputs observed. Our independent study of Thaler's results, however, reveals a simpler explanation of the 'virtual input' phenomenon.
Fri, 01 Jan 1999 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429041999-01-01T00:00:00Z
- Product-, corporate-, and country-image dimensions and purchase behavior: A multicountry analysishttps://scholarbank.nus.edu.sg/handle/10635/42898Title: Product-, corporate-, and country-image dimensions and purchase behavior: A multicountry analysis
Authors: Hsieh, M.-H.; Pan, S.-L.; Setiono, R.
Abstract: This research focuses on consumer perceptions that are developed on the basis of a firm's advertising appeals as well as other factors. In conceptualizing brand-image perceptions, the authors extend the frequent use of product-related images to include corporate and country images attached to brands. The authors report findings based on secondary economic and cultural data at the macro level and the results of a global brand-image survey conducted in the top 20 international automobile markets at the individual level. The findings suggest that while consumers' attitudes toward corporate image and country image exert main effects on their brand purchase behavior, the effects of certain product-image appeals are moderated by sociodemographics and national cultural characteristics. The empirical results are broadly supportive of the proposed hypotheses and provide a consumer-based extension of Roth's work on global brand image.
Thu, 01 Jan 2004 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/428982004-01-01T00:00:00Z
- Neurolinear: From neural networks to oblique decision ruleshttps://scholarbank.nus.edu.sg/handle/10635/99345Title: Neurolinear: From neural networks to oblique decision rules
Authors: Setiono, R.; Liu, H.
Abstract: We present NeuroLinear, a system for extracting oblique decision rules from neural networks that have been trained for classification of patterns. Each condition of an oblique decision rule corresponds to a partition of the attribute space by a hyperplane that is not necessarily axis-parallel. Allowing a set of such hyperplanes to form the boundaries of the decision regions leads to a significant reduction in the number of rules generated while maintaining the accuracy rates of the networks. We describe the components of NeuroLinear in detail by way of two examples using artificial datasets. Our experimental results on real-world datasets show that the system is effective in extracting compact and comprehensible rules with high predictive accuracy from neural networks.
Tue, 30 Sep 1997 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/993451997-09-30T00:00:00Z
- Generating concise and accurate classification rules for breast cancer diagnosishttps://scholarbank.nus.edu.sg/handle/10635/42371Title: Generating concise and accurate classification rules for breast cancer diagnosis
Authors: Setiono, R.
Abstract: In our previous work, we have presented an algorithm that extracts classification rules from trained neural networks and discussed its application to breast cancer diagnosis. In this paper, we describe how the accuracy of the networks and the accuracy of the rules extracted from them can be improved by a simple pre-processing of the data. Data pre-processing involves selecting the relevant input attributes and removing those samples with missing attribute values. The rules generated by our neural network rule extraction algorithm are more concise and accurate than those generated by other rule generating methods reported in the literature. (C) 2000 Elsevier Science B.V.
Sat, 01 Jan 2000 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/423712000-01-01T00:00:00Z
- Extraction of rules from artificial neural networks for nonlinear regressionhttps://scholarbank.nus.edu.sg/handle/10635/42907Title: Extraction of rules from artificial neural networks for nonlinear regression
Authors: Setiono, R.; Leow, W.K.; Zurada, J.M.
Abstract: Neural networks (NNs) have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how the problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained NNs for regression. This article presents an approach for extracting rules from trained NNs for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.
Tue, 01 Jan 2002 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/429072002-01-01T00:00:00Z
- Market research applications of artificial neural networkshttps://scholarbank.nus.edu.sg/handle/10635/42605Title: Market research applications of artificial neural networks
Authors: Azcarraga, A.P.; Hsieh, M.-H.; Setiono, R.
Abstract: Even in an increasingly globalized market, the knowledge about individual local markets could still be invaluable. In this cross-national study of brand image perception of cars, survey data from buyers in the top 20 automobile markets have been collected, where each respondent has been asked to associate a car brand with individual brand images and corporate brand images. These data can be used as tool for decision making at the enterprise level. We describe an algorithm for constructing auto-associative neural networks which can be used as a tool for knowledge discovery from this data. It automatically determines the network topology by adding hidden units as they are needed to improve accuracy and by removing irrelevant input attributes. Two market research applications are presented, the first is for classification, and the second is for measuring similarities in the perceptions of the respondents from the different markets. In the first application, the constructed networks are shown to be more accurate than a decision tree. In the second application, the constructed networks are able to reproduce the training data very accurately and could be used to identify country-level (i.e. local) markets which share similar perceptions about the car brands being studied. © 2008 IEEE.
Tue, 01 Jan 2008 00:00:00 GMThttps://scholarbank.nus.edu.sg/handle/10635/426052008-01-01T00:00:00Z