Please use this identifier to cite or link to this item:
|Title:||Predicting the parallel file system performance via machine learning||Authors:||Zhao, T.
Parallel file system
|Issue Date:||2011||Citation:||Zhao, T.,Dong, S.,March, V.,See, S. (2011). Predicting the parallel file system performance via machine learning. Jisuanji Yanjiu yu Fazhan/Computer Research and Development 48 (7) : 1202-1215. ScholarBank@NUS Repository.||Abstract:||Parallel file system can effectively solve the problems of massive data storage and I/O bottleneck. Because the potential impact on the system is not clearly understood, how to evaluate and predict performance of parallel file system becomes the potential challenge and hotspot. In this work, we aim to research the performance evaluation and prediction of parallel file system. After studying the architecture and performance factors of such file system, we design a predictive mode of parallel file system based on machine learning approaches. We use feature selection algorithms to reduce the number of performance factors to be tested in validating the performance. We also mine the particular relationship of system performance and impact factors to predict the performance of a specific file system. We validate and predict the performance of a specific Lustre file system through a series of experiment cases. Our evaluation and experiment results indicate that threads/OST, num of OSSs (Object Storage Server), num of disks and num and type of RAID are the four most important parameters to tune the performance of Lustre file system. The average relative errors of predictive results can be controlled within 25.1%-32.1%, which shows the better prediction accuracy.||Source Title:||Jisuanji Yanjiu yu Fazhan/Computer Research and Development||URI:||http://scholarbank.nus.edu.sg/handle/10635/39761||ISSN:||10001239|
|Appears in Collections:||Staff Publications|
Show full item record
Files in This Item:
There are no files associated with this item.
checked on Apr 19, 2019
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.