{"data":{"id":"10.48550/arxiv.2202.08396","type":"dois","attributes":{"doi":"10.48550/arxiv.2202.08396","prefix":"10.48550","suffix":"arxiv.2202.08396","identifiers":[{"identifier":"2202.08396","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2202.08396"}],"creators":[{"name":"Duan, Haonan","nameType":"Personal","givenName":"Haonan","familyName":"Duan","affiliation":[],"nameIdentifiers":[]},{"name":"Vaezipoor, Pashootan","nameType":"Personal","givenName":"Pashootan","familyName":"Vaezipoor","affiliation":[],"nameIdentifiers":[]},{"name":"Paulus, Max B.","nameType":"Personal","givenName":"Max B.","familyName":"Paulus","affiliation":[],"nameIdentifiers":[]},{"name":"Ruan, Yangjun","nameType":"Personal","givenName":"Yangjun","familyName":"Ruan","affiliation":[],"nameIdentifiers":[]},{"name":"Maddison, Chris J.","nameType":"Personal","givenName":"Chris J.","familyName":"Maddison","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Augment with Care: Contrastive Learning for Combinatorial Problems"}],"publisher":"arXiv","container":{},"publicationYear":2022,"subjects":[{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"lang":"en","subject":"Artificial Intelligence (cs.AI)","subjectScheme":"arXiv"},{"lang":"en","subject":"Logic in Computer Science (cs.LO)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2022-02-17T01:31:32Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-02-18T01:07:04Z","dateType":"Updated","dateInformation":"v1"},{"date":"2022-06-20T19:11:30Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2022-06-22T00:36:50Z","dateType":"Updated","dateInformation":"v2"},{"date":"2022-02","dateType":"Available","dateInformation":"v1"},{"date":"2022","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"Creative Commons Attribution 4.0 International","rightsUri":"https://creativecommons.org/licenses/by/4.0/legalcode","schemeUri":"https://spdx.org/licenses/","rightsIdentifier":"cc-by-4.0","rightsIdentifierScheme":"SPDX"}],"descriptions":[{"description":"Supervised learning can improve the design of state-of-the-art solvers for combinatorial problems, but labelling large numbers of combinatorial instances is often impractical due to exponential worst-case complexity. Inspired by the recent success of contrastive pre-training for images, we conduct a scientific study of the effect of augmentation design on contrastive pre-training for the Boolean satisfiability problem. While typical graph contrastive pre-training uses label-agnostic augmentations, our key insight is that many combinatorial problems have well-studied invariances, which allow for the design of label-preserving augmentations. We find that label-preserving augmentations are critical for the success of contrastive pre-training. We show that our representations are able to achieve comparable test accuracy to fully-supervised learning while using only 1% of the labels. We also demonstrate that our representations are more transferable to larger problems from unseen domains. Our code is available at https://github.com/h4duan/contrastive-sat.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMjAyLjA4Mzk2PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjIwMi4wODM5NjwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5EdWFuLCBIYW9uYW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkhhb25hbjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5EdWFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlZhZXppcG9vciwgUGFzaG9vdGFuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5QYXNob290YW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+VmFlemlwb29yPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlBhdWx1cywgTWF4IEIuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5NYXggQi48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UGF1bHVzPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlJ1YW4sIFlhbmdqdW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPllhbmdqdW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UnVhbjwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5NYWRkaXNvbiwgQ2hyaXMgSi48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkNocmlzIEouPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPk1hZGRpc29uPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5BdWdtZW50IHdpdGggQ2FyZTogQ29udHJhc3RpdmUgTGVhcm5pbmcgZm9yIENvbWJpbmF0b3JpYWwgUHJvYmxlbXM8L3RpdGxlPgogIDwvdGl0bGVzPgogIDxwdWJsaXNoZXI+YXJYaXY8L3B1Ymxpc2hlcj4KICA8cHVibGljYXRpb25ZZWFyPjIwMjI8L3B1YmxpY2F0aW9uWWVhcj4KICA8c3ViamVjdHM+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5NYWNoaW5lIExlYXJuaW5nIChjcy5MRyk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5BcnRpZmljaWFsIEludGVsbGlnZW5jZSAoY3MuQUkpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TG9naWMgaW4gQ29tcHV0ZXIgU2NpZW5jZSAoY3MuTE8pPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0wMi0xN1QwMTozMTozMlo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0wMi0xOFQwMTowNzowNFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIyLTA2LTIwVDE5OjExOjMwWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIyLTA2LTIyVDAwOjM2OjUwWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjItMDI8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MjwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vY3JlYXRpdmVjb21tb25zLm9yZy9saWNlbnNlcy9ieS80LjAvIiByaWdodHNJZGVudGlmaWVyU2NoZW1lPSJTUERYIiByaWdodHNJZGVudGlmaWVyPSJDQy1CWS00LjAiPkNyZWF0aXZlIENvbW1vbnMgQXR0cmlidXRpb24gNC4wIEludGVybmF0aW9uYWw8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5TdXBlcnZpc2VkIGxlYXJuaW5nIGNhbiBpbXByb3ZlIHRoZSBkZXNpZ24gb2Ygc3RhdGUtb2YtdGhlLWFydCBzb2x2ZXJzIGZvciBjb21iaW5hdG9yaWFsIHByb2JsZW1zLCBidXQgbGFiZWxsaW5nIGxhcmdlIG51bWJlcnMgb2YgY29tYmluYXRvcmlhbCBpbnN0YW5jZXMgaXMgb2Z0ZW4gaW1wcmFjdGljYWwgZHVlIHRvIGV4cG9uZW50aWFsIHdvcnN0LWNhc2UgY29tcGxleGl0eS4gSW5zcGlyZWQgYnkgdGhlIHJlY2VudCBzdWNjZXNzIG9mIGNvbnRyYXN0aXZlIHByZS10cmFpbmluZyBmb3IgaW1hZ2VzLCB3ZSBjb25kdWN0IGEgc2NpZW50aWZpYyBzdHVkeSBvZiB0aGUgZWZmZWN0IG9mIGF1Z21lbnRhdGlvbiBkZXNpZ24gb24gY29udHJhc3RpdmUgcHJlLXRyYWluaW5nIGZvciB0aGUgQm9vbGVhbiBzYXRpc2ZpYWJpbGl0eSBwcm9ibGVtLiBXaGlsZSB0eXBpY2FsIGdyYXBoIGNvbnRyYXN0aXZlIHByZS10cmFpbmluZyB1c2VzIGxhYmVsLWFnbm9zdGljIGF1Z21lbnRhdGlvbnMsIG91ciBrZXkgaW5zaWdodCBpcyB0aGF0IG1hbnkgY29tYmluYXRvcmlhbCBwcm9ibGVtcyBoYXZlIHdlbGwtc3R1ZGllZCBpbnZhcmlhbmNlcywgd2hpY2ggYWxsb3cgZm9yIHRoZSBkZXNpZ24gb2YgbGFiZWwtcHJlc2VydmluZyBhdWdtZW50YXRpb25zLiBXZSBmaW5kIHRoYXQgbGFiZWwtcHJlc2VydmluZyBhdWdtZW50YXRpb25zIGFyZSBjcml0aWNhbCBmb3IgdGhlIHN1Y2Nlc3Mgb2YgY29udHJhc3RpdmUgcHJlLXRyYWluaW5nLiBXZSBzaG93IHRoYXQgb3VyIHJlcHJlc2VudGF0aW9ucyBhcmUgYWJsZSB0byBhY2hpZXZlIGNvbXBhcmFibGUgdGVzdCBhY2N1cmFjeSB0byBmdWxseS1zdXBlcnZpc2VkIGxlYXJuaW5nIHdoaWxlIHVzaW5nIG9ubHkgMSUgb2YgdGhlIGxhYmVscy4gV2UgYWxzbyBkZW1vbnN0cmF0ZSB0aGF0IG91ciByZXByZXNlbnRhdGlvbnMgYXJlIG1vcmUgdHJhbnNmZXJhYmxlIHRvIGxhcmdlciBwcm9ibGVtcyBmcm9tIHVuc2VlbiBkb21haW5zLiBPdXIgY29kZSBpcyBhdmFpbGFibGUgYXQgaHR0cHM6Ly9naXRodWIuY29tL2g0ZHVhbi9jb250cmFzdGl2ZS1zYXQuPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2202.08396","contentUrl":null,"metadataVersion":1,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-18T01:59:58.000Z","registered":"2022-02-18T01:59:59.000Z","published":"2022","updated":"2022-06-22T03:07:10.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2202.08396","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}