Hierarchical Scheduling for Diverse Datacenter Workloads
Title | Hierarchical Scheduling for Diverse Datacenter Workloads |
Publication Type | Conference Paper |
Year of Publication | 2013 |
Authors | Bhattacharya, A. A., Culler D. E., Friedman E., Ghodsi A., Shenker S. J., & Stoica I. |
Other Numbers | 3796 |
Abstract | There has been a recent industrial effort to develop multi-resource hierarchical schedulers. However, the existing implementations have some shortcomings in that they might leave resources unallocated or starve certain jobs. This is because the multi-resource setting introduces new challenges for hierarchical scheduling policies. We provide an algorithm, which we implement in Hadoop, that generalizes the most commonly used multi-resource scheduler, DRF [1], to support hierarchies. Our evaluation shows that our proposed algorithm, H-DRF, avoids the starvation and resource inefficiencies of the existing open-source schedulers and outperforms slot scheduling. |
URL | https://people.eecs.berkeley.edu/~alig/papers/h-drf.pdf |
Bibliographic Notes | Proceedings of the ACM Symposium on Cloud Computing, Santa Clara, California |
Abbreviated Authors | A. A. Bhattacharya, D. Culler, E. Friedman, A. Ghodsi, S. Shenker, and I. Stoica |
ICSI Research Group | Research Initiatives |
ICSI Publication Type | Article in conference proceedings |