Processing multi-join query in parallel systems

Abstract

In parallel systems, a number of joins from one or more queries can be executed either serially or in parallel. While serial execution assigns all processors to execute each join one after another, the parallel execution distributes the joins to clusters formed by certain number of processors and executes them concurrently. Both approaches employ parallelism to improve system performance. However, data skew may result in load imbalance among processors executing the same join and some clusters may be overloaded with time-consuming joins. As a result, the completion time will be much longer than what is expected. In this paper, we propose an algorithm to further minimize the completion time of concurrently executed multiple joins. For this algorithm, all the joins to be executed concurrently are decomposed into a set of tasks that are ordered according to decreasing task size. These tasks are dynamically allocated to available processors during execution. Our performance study shows that the proposed algorithm outperforms the previously proposed approaches, especially when number of processors increases, high skewness is present in the relations to be joined and relation sizes are large.