Design and implementation of an object storage system

Abstract

In the information age, data-intensive applications require secure, scalable and highly-available and high performance storage systems. This thesis aims at providing such an intelligent storage solution based on Object-based Storage Device (OSD) concept. This dissertation presents the design of BrainStor, a Fibre Channel OSD prototype. With the separated data path and metadata path, clients directly access storage. With the centralized metadata management and object level abstract, BrainStor supports intelligent storage management and cross-platform data sharing. The thesis also identifies some key challenges in OSD architecture by analyzing the test results of BrainStor.In order to address the intensive metadata access problem in metadata server cluster design, a new method named Hashing Partition (HAP) is proposed to avoid frequent metadata access and metadata movement within the cluster. This thesis demonstrates a design using HAP to achieve good performance of metadata cluster load balancing, failover and scalability.