Performance and Advanced Data Placement Techniques with Ceph's Distributed Storage System

[摘要] The STAR online computing environment is a demanding concentrated multipurpose compute system with the objective to obtain maximum throughput with process concurrency. Motivation for extending the STAR online compute farm from a simple job processing tool for data taking, into a multipurpose resource equipped with a large storage system would lead any dedicated resources to become an extremely efficient and an attractive multi-purpose facility. To achieve this goal, our compute farm is using the Ceph distributed storage system which has proven to be an agile solution due to its effective POSIX interface and excelling its object storage with IO concurrency. With this we have taken our cluster one step further in terms of IO performance by investigating and leveraging new technologies and key features of Ceph. With the acquisition of a 10Gb backbone network we have ensured to eliminate the network as a limitation. With further acquisition of large fast drives (1TB SSDs) we will show how one can customize the data placement options Ceph has to offer such as primary affinity, mounting OSD journals on SSDs, and Cache Tiering along with non-Ceph related local disk caching techniques. We will discuss the latest performance results along with the expected results by using each technique. We hope this paper will serve the community's interest for the Ceph distributed storage solution.

[发布机构] Brookhaven National Laboratory, P.O. Box 5000, Upton; NY; 11973-5000, United States^1

[效力级别] 计算机科学 [学科分类] 计算机科学（综合）

[关键词] Agile solutions;Back-bone network;Distributed storage;Distributed storage system;Maximum through-put;New technologies;Object storages;Online computing