{"data":{"id":"10.48550/arxiv.2001.08002","type":"dois","attributes":{"doi":"10.48550/arxiv.2001.08002","prefix":"10.48550","suffix":"arxiv.2001.08002","identifiers":[{"identifier":"2001.08002","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2001.08002"}],"creators":[{"name":"Fekry, Ayat","nameType":"Personal","givenName":"Ayat","familyName":"Fekry","affiliation":[],"nameIdentifiers":[]},{"name":"Carata, Lucian","nameType":"Personal","givenName":"Lucian","familyName":"Carata","affiliation":[],"nameIdentifiers":[]},{"name":"Pasquier, Thomas","nameType":"Personal","givenName":"Thomas","familyName":"Pasquier","affiliation":[],"nameIdentifiers":[]},{"name":"Rice, Andrew","nameType":"Personal","givenName":"Andrew","familyName":"Rice","affiliation":[],"nameIdentifiers":[]},{"name":"Hopper, Andy","nameType":"Personal","givenName":"Andy","familyName":"Hopper","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Tuneful: An Online Significance-Aware Configuration Tuner for Big Data Analytics"}],"publisher":"arXiv","container":{},"publicationYear":2020,"subjects":[{"lang":"en","subject":"Distributed, Parallel, and Cluster Computing (cs.DC)","subjectScheme":"arXiv"},{"lang":"en","subject":"Systems and Control (eess.SY)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2020-01-22T13:24:41Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2020-01-23T01:10:47Z","dateType":"Updated","dateInformation":"v1"},{"date":"2020-01","dateType":"Available","dateInformation":"v1"},{"date":"2020","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"Distributed analytics engines such as Spark are a common choice for processing extremely large datasets. However, finding good configurations for these systems remains challenging, with each workload potentially requiring a different setup to run optimally. Using suboptimal configurations incurs significant extra runtime costs. %Furthermore, Spark and similar platforms are gaining traction within data-scientists communities where awareness of such issues is relatively low. We propose Tuneful, an approach that efficiently tunes the configuration of in-memory cluster computing systems. Tuneful combines incremental Sensitivity Analysis and Bayesian optimization to identify near-optimal configurations from a high-dimensional search space, using a small number of executions. This setup allows the tuning to be done online, without any previous training. Our experimental results show that Tuneful reduces the search time for finding close-to-optimal configurations by 62\\% (at the median) when compared to existing state-of-the-art techniques. This means that the amortization of the tuning cost happens significantly faster, enabling practical tuning for new classes of workloads.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMDAxLjA4MDAyPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjAwMS4wODAwMjwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5GZWtyeSwgQXlhdDwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+QXlhdDwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5GZWtyeTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5DYXJhdGEsIEx1Y2lhbjwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+THVjaWFuPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkNhcmF0YTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5QYXNxdWllciwgVGhvbWFzPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5UaG9tYXM8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UGFzcXVpZXI8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+UmljZSwgQW5kcmV3PC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5BbmRyZXc8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UmljZTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5Ib3BwZXIsIEFuZHk8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkFuZHk8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+SG9wcGVyPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5UdW5lZnVsOiBBbiBPbmxpbmUgU2lnbmlmaWNhbmNlLUF3YXJlIENvbmZpZ3VyYXRpb24gVHVuZXIgZm9yIEJpZyBEYXRhIEFuYWx5dGljczwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAyMDwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPkRpc3RyaWJ1dGVkLCBQYXJhbGxlbCwgYW5kIENsdXN0ZXIgQ29tcHV0aW5nIChjcy5EQyk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5TeXN0ZW1zIGFuZCBDb250cm9sIChlZXNzLlNZKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IENvbXB1dGVyIGFuZCBpbmZvcm1hdGlvbiBzY2llbmNlczwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IEVsZWN0cmljYWwgZW5naW5lZXJpbmcsIGVsZWN0cm9uaWMgZW5naW5lZXJpbmcsIGluZm9ybWF0aW9uIGVuZ2luZWVyaW5nPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMC0wMS0yMlQxMzoyNDo0MVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMC0wMS0yM1QwMToxMDo0N1o8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIwLTAxPC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJQcmVwcmludCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDx2ZXJzaW9uPjE8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2FyeGl2Lm9yZy9saWNlbnNlcy9ub25leGNsdXNpdmUtZGlzdHJpYi8xLjAvIj5hclhpdi5vcmcgcGVycGV0dWFsLCBub24tZXhjbHVzaXZlIGxpY2Vuc2U8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5EaXN0cmlidXRlZCBhbmFseXRpY3MgZW5naW5lcyBzdWNoIGFzIFNwYXJrIGFyZSBhIGNvbW1vbiBjaG9pY2UgZm9yIHByb2Nlc3NpbmcgZXh0cmVtZWx5IGxhcmdlIGRhdGFzZXRzLiBIb3dldmVyLCBmaW5kaW5nIGdvb2QgY29uZmlndXJhdGlvbnMgZm9yIHRoZXNlIHN5c3RlbXMgcmVtYWlucyBjaGFsbGVuZ2luZywgd2l0aCBlYWNoIHdvcmtsb2FkIHBvdGVudGlhbGx5IHJlcXVpcmluZyBhIGRpZmZlcmVudCBzZXR1cCB0byBydW4gb3B0aW1hbGx5LiBVc2luZyBzdWJvcHRpbWFsIGNvbmZpZ3VyYXRpb25zIGluY3VycyBzaWduaWZpY2FudCBleHRyYSBydW50aW1lIGNvc3RzLiAlRnVydGhlcm1vcmUsIFNwYXJrIGFuZCBzaW1pbGFyIHBsYXRmb3JtcyBhcmUgZ2FpbmluZyB0cmFjdGlvbiB3aXRoaW4gZGF0YS1zY2llbnRpc3RzIGNvbW11bml0aWVzIHdoZXJlIGF3YXJlbmVzcyBvZiBzdWNoIGlzc3VlcyBpcyByZWxhdGl2ZWx5IGxvdy4KICBXZSBwcm9wb3NlIFR1bmVmdWwsIGFuIGFwcHJvYWNoIHRoYXQgZWZmaWNpZW50bHkgdHVuZXMgdGhlIGNvbmZpZ3VyYXRpb24gb2YgaW4tbWVtb3J5IGNsdXN0ZXIgY29tcHV0aW5nIHN5c3RlbXMuIFR1bmVmdWwgY29tYmluZXMgaW5jcmVtZW50YWwgU2Vuc2l0aXZpdHkgQW5hbHlzaXMgYW5kIEJheWVzaWFuIG9wdGltaXphdGlvbiB0byBpZGVudGlmeSBuZWFyLW9wdGltYWwgY29uZmlndXJhdGlvbnMgZnJvbSBhIGhpZ2gtZGltZW5zaW9uYWwgc2VhcmNoIHNwYWNlLCB1c2luZyBhIHNtYWxsIG51bWJlciBvZiBleGVjdXRpb25zLiBUaGlzIHNldHVwIGFsbG93cyB0aGUgdHVuaW5nIHRvIGJlIGRvbmUgb25saW5lLCB3aXRob3V0IGFueSBwcmV2aW91cyB0cmFpbmluZy4gT3VyIGV4cGVyaW1lbnRhbCByZXN1bHRzIHNob3cgdGhhdCBUdW5lZnVsIHJlZHVjZXMgdGhlIHNlYXJjaCB0aW1lIGZvciBmaW5kaW5nIGNsb3NlLXRvLW9wdGltYWwgY29uZmlndXJhdGlvbnMgYnkgNjJcJSAoYXQgdGhlIG1lZGlhbikgd2hlbiBjb21wYXJlZCB0byBleGlzdGluZyBzdGF0ZS1vZi10aGUtYXJ0IHRlY2huaXF1ZXMuIFRoaXMgbWVhbnMgdGhhdCB0aGUgYW1vcnRpemF0aW9uIG9mIHRoZSB0dW5pbmcgY29zdCBoYXBwZW5zIHNpZ25pZmljYW50bHkgZmFzdGVyLCBlbmFibGluZyBwcmFjdGljYWwgdHVuaW5nIGZvciBuZXcgY2xhc3NlcyBvZiB3b3JrbG9hZHMuPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2001.08002","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-26T22:35:54.000Z","registered":"2022-02-26T22:35:55.000Z","published":"2020","updated":"2022-02-26T22:35:55.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2001.08002","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}