{"data":{"id":"10.48550/arxiv.1906.00910","type":"dois","attributes":{"doi":"10.48550/arxiv.1906.00910","prefix":"10.48550","suffix":"arxiv.1906.00910","identifiers":[{"identifier":"1906.00910","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1906.00910"}],"creators":[{"name":"Bachman, Philip","nameType":"Personal","givenName":"Philip","familyName":"Bachman","affiliation":[],"nameIdentifiers":[]},{"name":"Hjelm, R Devon","nameType":"Personal","givenName":"R Devon","familyName":"Hjelm","affiliation":[],"nameIdentifiers":[]},{"name":"Buchwalter, William","nameType":"Personal","givenName":"William","familyName":"Buchwalter","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Learning Representations by Maximizing Mutual Information Across Views"}],"publisher":"arXiv","container":{},"publicationYear":2019,"subjects":[{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2019-06-03T16:24:57Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2019-06-04T00:31:44Z","dateType":"Updated","dateInformation":"v1"},{"date":"2019-07-08T16:41:31Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2019-07-09T00:28:02Z","dateType":"Updated","dateInformation":"v2"},{"date":"2019-06","dateType":"Available","dateInformation":"v1"},{"date":"2019","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"We propose an approach to self-supervised representation learning based on maximizing mutual information between features extracted from multiple views of a shared context. For example, one could produce multiple views of a local spatio-temporal context by observing it from different locations (e.g., camera positions within a scene), and via different modalities (e.g., tactile, auditory, or visual). Or, an ImageNet image could provide a context from which one produces multiple views by repeatedly applying data augmentation. Maximizing mutual information between features extracted from these views requires capturing information about high-level factors whose influence spans multiple views -- e.g., presence of certain objects or occurrence of certain events. Following our proposed approach, we develop a model which learns image representations that significantly outperform prior methods on the tasks we consider. Most notably, using self-supervised learning, our model learns representations which achieve 68.1% accuracy on ImageNet using standard linear evaluation. This beats prior results by over 12% and concurrent results by 7%. When we extend our model to use mixture-based representations, segmentation behaviour emerges as a natural side-effect. Our code is available online: https://github.com/Philip-Bachman/amdim-public.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xOTA2LjAwOTEwPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTkwNi4wMDkxMDwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5CYWNobWFuLCBQaGlsaXA8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlBoaWxpcDwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5CYWNobWFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkhqZWxtLCBSIERldm9uPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5SIERldm9uPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkhqZWxtPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkJ1Y2h3YWx0ZXIsIFdpbGxpYW08L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPldpbGxpYW08L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+QnVjaHdhbHRlcjwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICA8L2NyZWF0b3JzPgogIDx0aXRsZXM+CiAgICA8dGl0bGU+TGVhcm5pbmcgUmVwcmVzZW50YXRpb25zIGJ5IE1heGltaXppbmcgTXV0dWFsIEluZm9ybWF0aW9uIEFjcm9zcyBWaWV3czwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAxOTwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPk1hY2hpbmUgTGVhcm5pbmcgKGNzLkxHKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPk1hY2hpbmUgTGVhcm5pbmcgKHN0YXQuTUwpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxOS0wNi0wM1QxNjoyNDo1N1o8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxOS0wNi0wNFQwMDozMTo0NFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE5LTA3LTA4VDE2OjQxOjMxWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE5LTA3LTA5VDAwOjI4OjAyWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTktMDY8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MjwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vYXJ4aXYub3JnL2xpY2Vuc2VzL25vbmV4Y2x1c2l2ZS1kaXN0cmliLzEuMC8iPmFyWGl2Lm9yZyBwZXJwZXR1YWwsIG5vbi1leGNsdXNpdmUgbGljZW5zZTwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPldlIHByb3Bvc2UgYW4gYXBwcm9hY2ggdG8gc2VsZi1zdXBlcnZpc2VkIHJlcHJlc2VudGF0aW9uIGxlYXJuaW5nIGJhc2VkIG9uIG1heGltaXppbmcgbXV0dWFsIGluZm9ybWF0aW9uIGJldHdlZW4gZmVhdHVyZXMgZXh0cmFjdGVkIGZyb20gbXVsdGlwbGUgdmlld3Mgb2YgYSBzaGFyZWQgY29udGV4dC4gRm9yIGV4YW1wbGUsIG9uZSBjb3VsZCBwcm9kdWNlIG11bHRpcGxlIHZpZXdzIG9mIGEgbG9jYWwgc3BhdGlvLXRlbXBvcmFsIGNvbnRleHQgYnkgb2JzZXJ2aW5nIGl0IGZyb20gZGlmZmVyZW50IGxvY2F0aW9ucyAoZS5nLiwgY2FtZXJhIHBvc2l0aW9ucyB3aXRoaW4gYSBzY2VuZSksIGFuZCB2aWEgZGlmZmVyZW50IG1vZGFsaXRpZXMgKGUuZy4sIHRhY3RpbGUsIGF1ZGl0b3J5LCBvciB2aXN1YWwpLiBPciwgYW4gSW1hZ2VOZXQgaW1hZ2UgY291bGQgcHJvdmlkZSBhIGNvbnRleHQgZnJvbSB3aGljaCBvbmUgcHJvZHVjZXMgbXVsdGlwbGUgdmlld3MgYnkgcmVwZWF0ZWRseSBhcHBseWluZyBkYXRhIGF1Z21lbnRhdGlvbi4gTWF4aW1pemluZyBtdXR1YWwgaW5mb3JtYXRpb24gYmV0d2VlbiBmZWF0dXJlcyBleHRyYWN0ZWQgZnJvbSB0aGVzZSB2aWV3cyByZXF1aXJlcyBjYXB0dXJpbmcgaW5mb3JtYXRpb24gYWJvdXQgaGlnaC1sZXZlbCBmYWN0b3JzIHdob3NlIGluZmx1ZW5jZSBzcGFucyBtdWx0aXBsZSB2aWV3cyAtLSBlLmcuLCBwcmVzZW5jZSBvZiBjZXJ0YWluIG9iamVjdHMgb3Igb2NjdXJyZW5jZSBvZiBjZXJ0YWluIGV2ZW50cy4KICBGb2xsb3dpbmcgb3VyIHByb3Bvc2VkIGFwcHJvYWNoLCB3ZSBkZXZlbG9wIGEgbW9kZWwgd2hpY2ggbGVhcm5zIGltYWdlIHJlcHJlc2VudGF0aW9ucyB0aGF0IHNpZ25pZmljYW50bHkgb3V0cGVyZm9ybSBwcmlvciBtZXRob2RzIG9uIHRoZSB0YXNrcyB3ZSBjb25zaWRlci4gTW9zdCBub3RhYmx5LCB1c2luZyBzZWxmLXN1cGVydmlzZWQgbGVhcm5pbmcsIG91ciBtb2RlbCBsZWFybnMgcmVwcmVzZW50YXRpb25zIHdoaWNoIGFjaGlldmUgNjguMSUgYWNjdXJhY3kgb24gSW1hZ2VOZXQgdXNpbmcgc3RhbmRhcmQgbGluZWFyIGV2YWx1YXRpb24uIFRoaXMgYmVhdHMgcHJpb3IgcmVzdWx0cyBieSBvdmVyIDEyJSBhbmQgY29uY3VycmVudCByZXN1bHRzIGJ5IDclLiBXaGVuIHdlIGV4dGVuZCBvdXIgbW9kZWwgdG8gdXNlIG1peHR1cmUtYmFzZWQgcmVwcmVzZW50YXRpb25zLCBzZWdtZW50YXRpb24gYmVoYXZpb3VyIGVtZXJnZXMgYXMgYSBuYXR1cmFsIHNpZGUtZWZmZWN0LiBPdXIgY29kZSBpcyBhdmFpbGFibGUgb25saW5lOiBodHRwczovL2dpdGh1Yi5jb20vUGhpbGlwLUJhY2htYW4vYW1kaW0tcHVibGljLjwvZGVzY3JpcHRpb24+CiAgPC9kZXNjcmlwdGlvbnM+CjwvcmVzb3VyY2U+","url":"https://arxiv.org/abs/1906.00910","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-28T04:15:10.000Z","registered":"2022-02-28T04:15:12.000Z","published":"2019","updated":"2022-02-28T04:15:12.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1906.00910","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}