WRITE-INTENSIVE DATA MANAGEMENT IN LOG-STRUCTURED STORAGE

Abstract

Due to the rapid development of information technologies, real-world workloads are becoming write-intensive and large-scale. In this thesis, we work towards designing solutions for managing write-intensive workloads with the adoption of log-structured techniques. We first propose a distributed log-structured storage, providing high write-throughput. It removes the write bottleneck by unifying data and log repositories, and supports fast failure recovery. Second, we design a novel indexing method on top of log-storage to support efficient range queries. It works well for observational data, which is write-intensive in nature. It utilizes intrinsic clustering property in data source, and gracefully reduces indexing overhead. Lastly, we provide an extended solution for indexing multi-dimensional observational data. It overcomes the data sparsity in multi-dimensional spaces, and minimizes space over-coverage introduced by conventional methods. We evaluate proposed approaches using real workloads, and observe that though our approaches are optimized for write throughput, they still preserve good read efficiency.