{"data":{"id":"10.48550/arxiv.1907.01484","type":"dois","attributes":{"doi":"10.48550/arxiv.1907.01484","prefix":"10.48550","suffix":"arxiv.1907.01484","identifiers":[{"identifier":"1907.01484","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1907.01484"}],"creators":[{"name":"Mahajan, Kshiteej","nameType":"Personal","givenName":"Kshiteej","familyName":"Mahajan","affiliation":[],"nameIdentifiers":[]},{"name":"Balasubramanian, Arjun","nameType":"Personal","givenName":"Arjun","familyName":"Balasubramanian","affiliation":[],"nameIdentifiers":[]},{"name":"Singhvi, Arjun","nameType":"Personal","givenName":"Arjun","familyName":"Singhvi","affiliation":[],"nameIdentifiers":[]},{"name":"Venkataraman, Shivaram","nameType":"Personal","givenName":"Shivaram","familyName":"Venkataraman","affiliation":[],"nameIdentifiers":[]},{"name":"Akella, Aditya","nameType":"Personal","givenName":"Aditya","familyName":"Akella","affiliation":[],"nameIdentifiers":[]},{"name":"Phanishayee, Amar","nameType":"Personal","givenName":"Amar","familyName":"Phanishayee","affiliation":[],"nameIdentifiers":[]},{"name":"Chawla, Shuchi","nameType":"Personal","givenName":"Shuchi","familyName":"Chawla","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Themis: Fair and Efficient GPU Cluster Scheduling"}],"publisher":"arXiv","container":{},"publicationYear":2019,"subjects":[{"lang":"en","subject":"Distributed, Parallel, and Cluster Computing (cs.DC)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2019-07-02T16:45:22Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2019-07-03T00:22:07Z","dateType":"Updated","dateInformation":"v1"},{"date":"2019-10-29T16:15:01Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2019-10-30T00:21:04Z","dateType":"Updated","dateInformation":"v2"},{"date":"2019-07","dateType":"Available","dateInformation":"v1"},{"date":"2019","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"Modern distributed machine learning (ML) training workloads benefit significantly from leveraging GPUs. However, significant contention ensues when multiple such workloads are run atop a shared cluster of GPUs. A key question is how to fairly apportion GPUs across workloads. We find that established cluster scheduling disciplines are a poor fit because of ML workloads' unique attributes: ML jobs have long-running tasks that need to be gang-scheduled, and their performance is sensitive to tasks' relative placement. We propose Themis, a new scheduling framework for ML training workloads. It's GPU allocation policy enforces that ML workloads complete in a finish-time fair manner, a new notion we introduce. To capture placement sensitivity and ensure efficiency, Themis uses a two-level scheduling architecture where ML workloads bid on available resources that are offered in an auction run by a central arbiter. Our auction design allocates GPUs to winning bids by trading off efficiency for fairness in the short term but ensuring finish-time fairness in the long term. Our evaluation on a production trace shows that Themis can improve fairness by more than 2.25X and is ~5% to 250% more cluster efficient in comparison to state-of-the-art schedulers.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xOTA3LjAxNDg0PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTkwNy4wMTQ4NDwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5NYWhhamFuLCBLc2hpdGVlajwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+S3NoaXRlZWo8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+TWFoYWphbjwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5CYWxhc3VicmFtYW5pYW4sIEFyanVuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5Bcmp1bjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5CYWxhc3VicmFtYW5pYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+U2luZ2h2aSwgQXJqdW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkFyanVuPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlNpbmdodmk8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+VmVua2F0YXJhbWFuLCBTaGl2YXJhbTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+U2hpdmFyYW08L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+VmVua2F0YXJhbWFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkFrZWxsYSwgQWRpdHlhPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5BZGl0eWE8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+QWtlbGxhPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlBoYW5pc2hheWVlLCBBbWFyPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5BbWFyPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlBoYW5pc2hheWVlPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkNoYXdsYSwgU2h1Y2hpPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5TaHVjaGk8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+Q2hhd2xhPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5UaGVtaXM6IEZhaXIgYW5kIEVmZmljaWVudCBHUFUgQ2x1c3RlciBTY2hlZHVsaW5nPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDE5PC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+RGlzdHJpYnV0ZWQsIFBhcmFsbGVsLCBhbmQgQ2x1c3RlciBDb21wdXRpbmcgKGNzLkRDKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IENvbXB1dGVyIGFuZCBpbmZvcm1hdGlvbiBzY2llbmNlczwvc3ViamVjdD4KICA8L3N1YmplY3RzPgogIDxkYXRlcz4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTktMDctMDJUMTY6NDU6MjJaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTktMDctMDNUMDA6MjI6MDdaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MiI+MjAxOS0xMC0yOVQxNjoxNTowMVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MiI+MjAxOS0xMC0zMFQwMDoyMTowNFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDE5LTA3PC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJQcmVwcmludCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDx2ZXJzaW9uPjI8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2FyeGl2Lm9yZy9saWNlbnNlcy9ub25leGNsdXNpdmUtZGlzdHJpYi8xLjAvIj5hclhpdi5vcmcgcGVycGV0dWFsLCBub24tZXhjbHVzaXZlIGxpY2Vuc2U8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5Nb2Rlcm4gZGlzdHJpYnV0ZWQgbWFjaGluZSBsZWFybmluZyAoTUwpIHRyYWluaW5nIHdvcmtsb2FkcyBiZW5lZml0IHNpZ25pZmljYW50bHkgZnJvbSBsZXZlcmFnaW5nIEdQVXMuIEhvd2V2ZXIsIHNpZ25pZmljYW50IGNvbnRlbnRpb24gZW5zdWVzIHdoZW4gbXVsdGlwbGUgc3VjaCB3b3JrbG9hZHMgYXJlIHJ1biBhdG9wIGEgc2hhcmVkIGNsdXN0ZXIgb2YgR1BVcy4gQSBrZXkgcXVlc3Rpb24gaXMgaG93IHRvIGZhaXJseSBhcHBvcnRpb24gR1BVcyBhY3Jvc3Mgd29ya2xvYWRzLiBXZSBmaW5kIHRoYXQgZXN0YWJsaXNoZWQgY2x1c3RlciBzY2hlZHVsaW5nIGRpc2NpcGxpbmVzIGFyZSBhIHBvb3IgZml0IGJlY2F1c2Ugb2YgTUwgd29ya2xvYWRzJyB1bmlxdWUgYXR0cmlidXRlczogTUwgam9icyBoYXZlIGxvbmctcnVubmluZyB0YXNrcyB0aGF0IG5lZWQgdG8gYmUgZ2FuZy1zY2hlZHVsZWQsIGFuZCB0aGVpciBwZXJmb3JtYW5jZSBpcyBzZW5zaXRpdmUgdG8gdGFza3MnIHJlbGF0aXZlIHBsYWNlbWVudC4KICBXZSBwcm9wb3NlIFRoZW1pcywgYSBuZXcgc2NoZWR1bGluZyBmcmFtZXdvcmsgZm9yIE1MIHRyYWluaW5nIHdvcmtsb2Fkcy4gSXQncyBHUFUgYWxsb2NhdGlvbiBwb2xpY3kgZW5mb3JjZXMgdGhhdCBNTCB3b3JrbG9hZHMgY29tcGxldGUgaW4gYSBmaW5pc2gtdGltZSBmYWlyIG1hbm5lciwgYSBuZXcgbm90aW9uIHdlIGludHJvZHVjZS4gVG8gY2FwdHVyZSBwbGFjZW1lbnQgc2Vuc2l0aXZpdHkgYW5kIGVuc3VyZSBlZmZpY2llbmN5LCBUaGVtaXMgdXNlcyBhIHR3by1sZXZlbCBzY2hlZHVsaW5nIGFyY2hpdGVjdHVyZSB3aGVyZSBNTCB3b3JrbG9hZHMgYmlkIG9uIGF2YWlsYWJsZSByZXNvdXJjZXMgdGhhdCBhcmUgb2ZmZXJlZCBpbiBhbiBhdWN0aW9uIHJ1biBieSBhIGNlbnRyYWwgYXJiaXRlci4gT3VyIGF1Y3Rpb24gZGVzaWduIGFsbG9jYXRlcyBHUFVzIHRvIHdpbm5pbmcgYmlkcyBieSB0cmFkaW5nIG9mZiBlZmZpY2llbmN5IGZvciBmYWlybmVzcyBpbiB0aGUgc2hvcnQgdGVybSBidXQgZW5zdXJpbmcgZmluaXNoLXRpbWUgZmFpcm5lc3MgaW4gdGhlIGxvbmcgdGVybS4gT3VyIGV2YWx1YXRpb24gb24gYSBwcm9kdWN0aW9uIHRyYWNlIHNob3dzIHRoYXQgVGhlbWlzIGNhbiBpbXByb3ZlIGZhaXJuZXNzIGJ5IG1vcmUgdGhhbiAyLjI1WCBhbmQgaXMgfjUlIHRvIDI1MCUgbW9yZSBjbHVzdGVyIGVmZmljaWVudCBpbiBjb21wYXJpc29uIHRvIHN0YXRlLW9mLXRoZS1hcnQgc2NoZWR1bGVycy48L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/1907.01484","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-28T08:24:52.000Z","registered":"2022-02-28T08:24:53.000Z","published":"2019","updated":"2022-02-28T08:24:53.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1907.01484","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}