Data services: bringing I/O processing to petascale

[摘要] The increasing size of high performance computing systems and the associatedincrease in the volume of generated data, has resulted in an I/O bottleneck for these applications.This bottleneck is further exacerbated by the imbalance in the growth of processingcapability compared to storage capability, due mainly to the power and cost requirementsof scaling the storage. This thesis introduces data services, a new abstraction which providessignificant benefits for data intensive applications. Data services combine low overheaddata movement with flexible placement of data manipulation operations, to addressthe I/O challenges of leadership class scientific applications. The impact of asynchronousdata movement on application runtime is minimized by utilizing novel server side datamovement schedulers to avoid contention related jitter in application communication. Additionally,the JITStager component is presented. Utilizing dynamic code generation andflexible code placement, the JITStager allows data services to be executed as a pipelineextending from the application to storage. It is shown in this thesis that data services canadd new functionality to the application without having an significant negative impact onperformance.

[发布机构] University:Georgia Institute of Technology;Department:Computing

[效力级别] Scientific computing

[关键词] I/O;Scientific computing;Data services;I/O frameworks;Infiniband;Cray;Supercomputing;Data;HPC