Estimating job runtime for CMS analysis jobs

[摘要] The basic premise of pilot systems is to create an overlay scheduling system on top of leased resources. And by definition, leases have a limited lifetime, so any job that is scheduled on such resources must finish before the lease is over, or it will be killed and all the computation is wasted. In order to effectively schedule jobs to resources, the pilot system thus requires the expected runtime of the users' jobs. Past studies have shown that relying on user provided estimates is not a valid strategy, so the system should try to make an estimate by itself. This paper provides a study of the historical data obtained from the Compact Muon Solenoid (CMS) experiment's Analysis Operations submission system. Clear patterns are observed, suggesting that making prediction of an expected job lifetime range is achievable with high confidence level in this environment.

[发布机构] University of California San Diego, 9500 Gilman Dr, San Diego; CA; 92093, United States^1

[效力级别] 计算机科学 [学科分类] 计算机科学（综合）

[关键词] Compact Muon solenoids;High confidence;Historical data;Leased resources;Pilot system;Runtimes;Scheduling systems