HARMONY: EXPLORING STAGE EXECUTION AND DATA LOCALITY IN GEO-DISTRIBUTED DATA PROCESSING

Abstract

Big-data application processing is increasingly geo-distributed, a paradigm shift from the traditional cluster-based processing frameworks. As the communication time for data movement across geo-distributed data centers is not a design criterion for traditional cluster-based processing approaches, there are research gaps in the algorithms used for staging and scheduling big-data applications for geo-distributed clusters. We address these gaps by proposing Harmony, an approach consisting of both staging and scheduling strategies to minimize an application's total execution time.