Please use this identifier to cite or link to this item:
|Title:||COSAC: A framework for combinatorial statistical analysis on cloud|
Combinatorial statistical analysis
Parallel object combination enumeration
|Citation:||Wang, Z., Agrawal, D., Tan, K.-L. (2013). COSAC: A framework for combinatorial statistical analysis on cloud. IEEE Transactions on Knowledge and Data Engineering 25 (9) : 2010-2023. ScholarBank@NUS Repository. https://doi.org/10.1109/TKDE.2012.113|
|Abstract:||In many scientific applications, it is critical to determine if there is a relationship between a combination of objects. The strength of such an association is typically computed using some statistical measures. In order not to miss any important associations, it is not uncommon to exhaustively enumerate all possible combinations of a certain size. However, discovering significant associations among hundreds of thousands or even millions of objects is a computationally intensive job that typically takes days, if not weeks, to complete. We are, therefore, motivated to provide efficient and practical techniques to speed up the processing exploiting parallelism. In this paper, we propose a framework, COSAC, for such combinatorial statistical analysis for large-scale data sets over a MapReduce-based cloud computing platform. COSAC operates in two key phases: 1) In the distribution phase, a novel load balancing scheme distributes the combination enumeration tasks across the processing units; 2) In the statistical analysis phase, each unit optimizes the processing of the allocated combinations by salvaging computations that can be reused. COSAC also supports a more practical scenario, where only a selected subset of objects need to be analyzed against all the objects. As a representative application, we developed COSAC to find combinations of Single Nucleotide Polymorphisms (SNPs) that may interact to cause diseases. We have evaluated our framework on a cluster of more than 40 nodes. The experimental results show that our framework is computationally practical, efficient, scalable, and flexible. © 1989-2012 IEEE.|
|Source Title:||IEEE Transactions on Knowledge and Data Engineering|
|Appears in Collections:||Staff Publications|
Show full item record
Files in This Item:
There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.