Using Map-reduce to scale an empirical database

Abstract

Datasets are crucial for testing in both industrial and academic fields. However, getting a dataset which has a proper size and can reflect the real data properties is not easy. Different from normal domain-specific benchmarks, UpSizeR is a tool that takes an empirical dataset D and a scale factor s as input and generates a synthetic dataset which keeps the properties of the original dataset but s times its size. UpSizeR is implemented using Map-Reduce which guarantees it could efficiently handle large datasets . In order to reduce I/O operations, we optimize our UpSizeR implementation to make it more efficient. We run queries on both the synthetic and the original datasets and compare the results to evaluate the similarity of both datasets.