{"data":{"id":"10.48550/arxiv.2006.16405","type":"dois","attributes":{"doi":"10.48550/arxiv.2006.16405","prefix":"10.48550","suffix":"arxiv.2006.16405","identifiers":[{"identifier":"2006.16405","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2006.16405"}],"creators":[{"name":"Pampari, Anusri","nameType":"Personal","givenName":"Anusri","familyName":"Pampari","affiliation":[],"nameIdentifiers":[]},{"name":"Ermon, Stefano","nameType":"Personal","givenName":"Stefano","familyName":"Ermon","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Unsupervised Calibration under Covariate Shift"}],"publisher":"arXiv","container":{},"publicationYear":2020,"subjects":[{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2020-06-29T21:50:07Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2020-07-01T00:05:18Z","dateType":"Updated","dateInformation":"v1"},{"date":"2020-06","dateType":"Available","dateInformation":"v1"},{"date":"2020","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"A probabilistic model is said to be calibrated if its predicted probabilities match the corresponding empirical frequencies. Calibration is important for uncertainty quantification and decision making in safety-critical applications. While calibration of classifiers has been widely studied, we find that calibration is brittle and can be easily lost under minimal covariate shifts. Existing techniques, including domain adaptation ones, primarily focus on prediction accuracy and do not guarantee calibration neither in theory nor in practice. In this work, we formally introduce the problem of calibration under domain shift, and propose an importance sampling based approach to address it. We evaluate and discuss the efficacy of our method on both real-world datasets and synthetic datasets.","descriptionType":"Abstract"},{"description":"Submitted to Conference on Uncertainty in Artificial Intelligence (UAI 2020)","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMDA2LjE2NDA1PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjAwNi4xNjQwNTwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5QYW1wYXJpLCBBbnVzcmk8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkFudXNyaTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5QYW1wYXJpPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkVybW9uLCBTdGVmYW5vPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5TdGVmYW5vPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkVybW9uPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5VbnN1cGVydmlzZWQgQ2FsaWJyYXRpb24gdW5kZXIgQ292YXJpYXRlIFNoaWZ0PC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDIwPC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoY3MuTEcpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoc3RhdC5NTCk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBDb21wdXRlciBhbmQgaW5mb3JtYXRpb24gc2NpZW5jZXM8L3N1YmplY3Q+CiAgPC9zdWJqZWN0cz4KICA8ZGF0ZXM+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIwLTA2LTI5VDIxOjUwOjA3WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIwLTA3LTAxVDAwOjA1OjE4WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjAtMDY8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MTwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vYXJ4aXYub3JnL2xpY2Vuc2VzL25vbmV4Y2x1c2l2ZS1kaXN0cmliLzEuMC8iPmFyWGl2Lm9yZyBwZXJwZXR1YWwsIG5vbi1leGNsdXNpdmUgbGljZW5zZTwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPkEgcHJvYmFiaWxpc3RpYyBtb2RlbCBpcyBzYWlkIHRvIGJlIGNhbGlicmF0ZWQgaWYgaXRzIHByZWRpY3RlZCBwcm9iYWJpbGl0aWVzIG1hdGNoIHRoZSBjb3JyZXNwb25kaW5nIGVtcGlyaWNhbCBmcmVxdWVuY2llcy4gQ2FsaWJyYXRpb24gaXMgaW1wb3J0YW50IGZvciB1bmNlcnRhaW50eSBxdWFudGlmaWNhdGlvbiBhbmQgZGVjaXNpb24gbWFraW5nIGluIHNhZmV0eS1jcml0aWNhbCBhcHBsaWNhdGlvbnMuIFdoaWxlIGNhbGlicmF0aW9uIG9mIGNsYXNzaWZpZXJzIGhhcyBiZWVuIHdpZGVseSBzdHVkaWVkLCB3ZSBmaW5kIHRoYXQgY2FsaWJyYXRpb24gaXMgYnJpdHRsZSBhbmQgY2FuIGJlIGVhc2lseSBsb3N0IHVuZGVyIG1pbmltYWwgY292YXJpYXRlIHNoaWZ0cy4gRXhpc3RpbmcgdGVjaG5pcXVlcywgaW5jbHVkaW5nIGRvbWFpbiBhZGFwdGF0aW9uIG9uZXMsIHByaW1hcmlseSBmb2N1cyBvbiBwcmVkaWN0aW9uIGFjY3VyYWN5IGFuZCBkbyBub3QgZ3VhcmFudGVlIGNhbGlicmF0aW9uIG5laXRoZXIgaW4gdGhlb3J5IG5vciBpbiBwcmFjdGljZS4gSW4gdGhpcyB3b3JrLCB3ZSBmb3JtYWxseSBpbnRyb2R1Y2UgdGhlIHByb2JsZW0gb2YgY2FsaWJyYXRpb24gdW5kZXIgZG9tYWluIHNoaWZ0LCBhbmQgcHJvcG9zZSBhbiBpbXBvcnRhbmNlIHNhbXBsaW5nIGJhc2VkIGFwcHJvYWNoIHRvIGFkZHJlc3MgaXQuIFdlIGV2YWx1YXRlIGFuZCBkaXNjdXNzIHRoZSBlZmZpY2FjeSBvZiBvdXIgbWV0aG9kIG9uIGJvdGggcmVhbC13b3JsZCBkYXRhc2V0cyBhbmQgc3ludGhldGljIGRhdGFzZXRzLjwvZGVzY3JpcHRpb24+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJPdGhlciI+U3VibWl0dGVkIHRvIENvbmZlcmVuY2Ugb24gVW5jZXJ0YWludHkgaW4gQXJ0aWZpY2lhbCBJbnRlbGxpZ2VuY2UgKFVBSSAyMDIwKTwvZGVzY3JpcHRpb24+CiAgPC9kZXNjcmlwdGlvbnM+CjwvcmVzb3VyY2U+","url":"https://arxiv.org/abs/2006.16405","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-25T02:51:11.000Z","registered":"2022-02-25T02:51:12.000Z","published":"2020","updated":"2022-02-25T02:51:12.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2006.16405","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}