TetriSched: Global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters
TetriSched is a scheduler that works in tandem with a calendaring reservation system to continuously re-evaluate the immediate-term scheduling plan for all pending jobs (including those with reservations and best-effort jobs) on each scheduling cycle. TetriSched leverages information supplied by the reservation system about jobs' deadlines and estimated runtimes to plan ahead in deciding whether to wait for a busy preferred resource type (e.g., machine with a GPU) or fall back to less preferred placement options. Plan-ahead affords significant flexibility in handling mis-estimates in job runtimes specified at reservation time. Integrated with the main reservation system in Hadoop YARN, TetriSched is experimentally shown to achieve significantly higher SLO attainment and cluster utilization than the best-configured YARN reservation and CapacityScheduler stack deployed on a real 256 node cluster.
Files
Metadata
Work Title | TetriSched: Global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters |
---|---|
Access | |
Creators |
|
License | In Copyright (Rights Reserved) |
Work Type | Conference Proceeding |
Publication Date | April 18, 2016 |
Publisher Identifier (DOI) |
|
Source |
|
Deposited | August 03, 2023 |
Versions
Analytics
Collections
This resource is currently not in any collection.