{"data":{"id":"10.48550/arxiv.2210.01408","type":"dois","attributes":{"doi":"10.48550/arxiv.2210.01408","prefix":"10.48550","suffix":"arxiv.2210.01408","identifiers":[{"identifier":"2210.01408","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2210.01408"}],"creators":[{"name":"Jin, Ying","nameType":"Personal","givenName":"Ying","familyName":"Jin","affiliation":[],"nameIdentifiers":[]},{"name":"Candès, Emmanuel J.","nameType":"Personal","givenName":"Emmanuel J.","familyName":"Candès","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Selection by Prediction with Conformal p-values"}],"publisher":"arXiv","container":{},"publicationYear":2022,"subjects":[{"lang":"en","subject":"Methodology (stat.ME)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2022-10-04T06:34:49Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-10-05T00:08:39Z","dateType":"Updated","dateInformation":"v1"},{"date":"2022-10-13T22:56:10Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2022-10-17T00:05:40Z","dateType":"Updated","dateInformation":"v2"},{"date":"2023-05-27T01:24:14Z","dateType":"Submitted","dateInformation":"v3"},{"date":"2023-05-30T00:06:43Z","dateType":"Updated","dateInformation":"v3"},{"date":"2022-10","dateType":"Available","dateInformation":"v1"},{"date":"2022","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"3","rightsList":[{"rights":"Creative Commons Attribution 4.0 International","rightsUri":"https://creativecommons.org/licenses/by/4.0/legalcode","schemeUri":"https://spdx.org/licenses/","rightsIdentifier":"cc-by-4.0","rightsIdentifierScheme":"SPDX"}],"descriptions":[{"description":"Decision making or scientific discovery pipelines such as job hiring and drug discovery often involve multiple stages: before any resource-intensive step, there is often an initial screening that uses predictions from a machine learning model to shortlist a few candidates from a large pool. We study screening procedures that aim to select candidates whose unobserved outcomes exceed user-specified values. We develop a method that wraps around any prediction model to produce a subset of candidates while controlling the proportion of falsely selected units. Building upon the conformal inference framework, our method first constructs p-values that quantify the statistical evidence for large outcomes; it then determines the shortlist by comparing the p-values to a threshold introduced in the multiple testing literature. In many cases, the procedure selects candidates whose predictions are above a data-dependent threshold. Our theoretical guarantee holds under mild exchangeability conditions on the samples, generalizing existing results on multiple conformal p-values. We demonstrate the empirical performance of our method via simulations, and apply it to job hiring and drug discovery datasets.","descriptionType":"Abstract"},{"description":"Journal of Machine Learning Research","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMjEwLjAxNDA4PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjIxMC4wMTQwODwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5KaW4sIFlpbmc8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPllpbmc8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+SmluPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkNhbmTDqHMsIEVtbWFudWVsIEouPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5FbW1hbnVlbCBKLjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5DYW5kw6hzPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5TZWxlY3Rpb24gYnkgUHJlZGljdGlvbiB3aXRoIENvbmZvcm1hbCBwLXZhbHVlczwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAyMjwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPk1ldGhvZG9sb2d5IChzdGF0Lk1FKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPk1hY2hpbmUgTGVhcm5pbmcgKHN0YXQuTUwpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0xMC0wNFQwNjozNDo0OVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0xMC0wNVQwMDowODozOVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIyLTEwLTEzVDIyOjU2OjEwWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIyLTEwLTE3VDAwOjA1OjQwWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjMiPjIwMjMtMDUtMjdUMDE6MjQ6MTRaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjMiPjIwMjMtMDUtMzBUMDA6MDY6NDNaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IkF2YWlsYWJsZSIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0xMDwvZGF0ZT4KICA8L2RhdGVzPgogIDxyZXNvdXJjZVR5cGUgcmVzb3VyY2VUeXBlR2VuZXJhbD0iUHJlcHJpbnQiPkFydGljbGU8L3Jlc291cmNlVHlwZT4KICA8dmVyc2lvbj4zPC92ZXJzaW9uPgogIDxyaWdodHNMaXN0PgogICAgPHJpZ2h0cyByaWdodHNVUkk9Imh0dHA6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LzQuMC8iIHJpZ2h0c0lkZW50aWZpZXJTY2hlbWU9IlNQRFgiIHJpZ2h0c0lkZW50aWZpZXI9IkNDLUJZLTQuMCI+Q3JlYXRpdmUgQ29tbW9ucyBBdHRyaWJ1dGlvbiA0LjAgSW50ZXJuYXRpb25hbDwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPkRlY2lzaW9uIG1ha2luZyBvciBzY2llbnRpZmljIGRpc2NvdmVyeSBwaXBlbGluZXMgc3VjaCBhcyBqb2IgaGlyaW5nIGFuZCBkcnVnIGRpc2NvdmVyeSBvZnRlbiBpbnZvbHZlIG11bHRpcGxlIHN0YWdlczogYmVmb3JlIGFueSByZXNvdXJjZS1pbnRlbnNpdmUgc3RlcCwgdGhlcmUgaXMgb2Z0ZW4gYW4gaW5pdGlhbCBzY3JlZW5pbmcgdGhhdCB1c2VzIHByZWRpY3Rpb25zIGZyb20gYSBtYWNoaW5lIGxlYXJuaW5nIG1vZGVsIHRvIHNob3J0bGlzdCBhIGZldyBjYW5kaWRhdGVzIGZyb20gYSBsYXJnZSBwb29sLiBXZSBzdHVkeSBzY3JlZW5pbmcgcHJvY2VkdXJlcyB0aGF0IGFpbSB0byBzZWxlY3QgY2FuZGlkYXRlcyB3aG9zZSB1bm9ic2VydmVkIG91dGNvbWVzIGV4Y2VlZCB1c2VyLXNwZWNpZmllZCB2YWx1ZXMuIFdlIGRldmVsb3AgYSBtZXRob2QgdGhhdCB3cmFwcyBhcm91bmQgYW55IHByZWRpY3Rpb24gbW9kZWwgdG8gcHJvZHVjZSBhIHN1YnNldCBvZiBjYW5kaWRhdGVzIHdoaWxlIGNvbnRyb2xsaW5nIHRoZSBwcm9wb3J0aW9uIG9mIGZhbHNlbHkgc2VsZWN0ZWQgdW5pdHMuIEJ1aWxkaW5nIHVwb24gdGhlIGNvbmZvcm1hbCBpbmZlcmVuY2UgZnJhbWV3b3JrLCBvdXIgbWV0aG9kIGZpcnN0IGNvbnN0cnVjdHMgcC12YWx1ZXMgdGhhdCBxdWFudGlmeSB0aGUgc3RhdGlzdGljYWwgZXZpZGVuY2UgZm9yIGxhcmdlIG91dGNvbWVzOyBpdCB0aGVuIGRldGVybWluZXMgdGhlIHNob3J0bGlzdCBieSBjb21wYXJpbmcgdGhlIHAtdmFsdWVzIHRvIGEgdGhyZXNob2xkIGludHJvZHVjZWQgaW4gdGhlIG11bHRpcGxlIHRlc3RpbmcgbGl0ZXJhdHVyZS4gSW4gbWFueSBjYXNlcywgdGhlIHByb2NlZHVyZSBzZWxlY3RzIGNhbmRpZGF0ZXMgd2hvc2UgcHJlZGljdGlvbnMgYXJlIGFib3ZlIGEgZGF0YS1kZXBlbmRlbnQgdGhyZXNob2xkLiBPdXIgdGhlb3JldGljYWwgZ3VhcmFudGVlIGhvbGRzIHVuZGVyIG1pbGQgZXhjaGFuZ2VhYmlsaXR5IGNvbmRpdGlvbnMgb24gdGhlIHNhbXBsZXMsIGdlbmVyYWxpemluZyBleGlzdGluZyByZXN1bHRzIG9uIG11bHRpcGxlIGNvbmZvcm1hbCBwLXZhbHVlcy4gV2UgZGVtb25zdHJhdGUgdGhlIGVtcGlyaWNhbCBwZXJmb3JtYW5jZSBvZiBvdXIgbWV0aG9kIHZpYSBzaW11bGF0aW9ucywgYW5kIGFwcGx5IGl0IHRvIGpvYiBoaXJpbmcgYW5kIGRydWcgZGlzY292ZXJ5IGRhdGFzZXRzLjwvZGVzY3JpcHRpb24+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJPdGhlciI+Sm91cm5hbCBvZiBNYWNoaW5lIExlYXJuaW5nIFJlc2VhcmNoPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2210.01408","contentUrl":null,"metadataVersion":2,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-10-05T00:56:43.000Z","registered":"2022-10-05T00:56:44.000Z","published":"2022","updated":"2023-05-30T01:47:01.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2210.01408","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}