{"data":{"id":"10.48550/arxiv.2306.08749","type":"dois","attributes":{"doi":"10.48550/arxiv.2306.08749","prefix":"10.48550","suffix":"arxiv.2306.08749","identifiers":[{"identifier":"2306.08749","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2306.08749"}],"creators":[{"name":"Zhu, Qingqing","nameType":"Personal","givenName":"Qingqing","familyName":"Zhu","affiliation":[],"nameIdentifiers":[]},{"name":"Mathai, Tejas Sudharshan","nameType":"Personal","givenName":"Tejas Sudharshan","familyName":"Mathai","affiliation":[],"nameIdentifiers":[]},{"name":"Mukherjee, Pritam","nameType":"Personal","givenName":"Pritam","familyName":"Mukherjee","affiliation":[],"nameIdentifiers":[]},{"name":"Peng, Yifan","nameType":"Personal","givenName":"Yifan","familyName":"Peng","affiliation":[],"nameIdentifiers":[]},{"name":"Summers, Ronald M.","nameType":"Personal","givenName":"Ronald M.","familyName":"Summers","affiliation":[],"nameIdentifiers":[]},{"name":"Lu, Zhiyong","nameType":"Personal","givenName":"Zhiyong","familyName":"Lu","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports"}],"publisher":"arXiv","container":{},"publicationYear":2023,"subjects":[{"lang":"en","subject":"Computation and Language (cs.CL)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2023-06-14T21:17:31Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2023-06-16T00:30:52Z","dateType":"Updated","dateInformation":"v1"},{"date":"2023-10-10T17:29:52Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2023-10-11T00:35:25Z","dateType":"Updated","dateInformation":"v2"},{"date":"2023-06","dateType":"Available","dateInformation":"v1"},{"date":"2023","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"Creative Commons Attribution 4.0 International","rightsUri":"https://creativecommons.org/licenses/by/4.0/legalcode","schemeUri":"https://spdx.org/licenses/","rightsIdentifier":"cc-by-4.0","rightsIdentifierScheme":"SPDX"}],"descriptions":[{"description":"Despite the reduction in turn-around times in radiology reports with the use of speech recognition software, persistent communication errors can significantly impact the interpretation of the radiology report. Pre-filling a radiology report holds promise in mitigating reporting errors, and despite efforts in the literature to generate medical reports, there exists a lack of approaches that exploit the longitudinal nature of patient visit records in the MIMIC-CXR dataset. To address this gap, we propose to use longitudinal multi-modal data, i.e., previous patient visit CXR, current visit CXR, and previous visit report, to pre-fill the 'findings' section of a current patient visit report. We first gathered the longitudinal visit information for 26,625 patients from the MIMIC-CXR dataset and created a new dataset called Longitudinal-MIMIC. With this new dataset, a transformer-based model was trained to capture the information from longitudinal patient visit records containing multi-modal data (CXR images + reports) via a cross-attention-based multi-modal fusion module and a hierarchical memory-driven decoder. In contrast to previous work that only uses current visit data as input to train a model, our work exploits the longitudinal information available to pre-fill the 'findings' section of radiology reports. Experiments show that our approach outperforms several recent approaches. Code will be published at https://github.com/CelestialShine/Longitudinal-Chest-X-Ray.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMzA2LjA4NzQ5PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjMwNi4wODc0OTwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5aaHUsIFFpbmdxaW5nPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5RaW5ncWluZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5aaHU8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+TWF0aGFpLCBUZWphcyBTdWRoYXJzaGFuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5UZWphcyBTdWRoYXJzaGFuPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPk1hdGhhaTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5NdWtoZXJqZWUsIFByaXRhbTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+UHJpdGFtPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPk11a2hlcmplZTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5QZW5nLCBZaWZhbjwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WWlmYW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UGVuZzwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5TdW1tZXJzLCBSb25hbGQgTS48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlJvbmFsZCBNLjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5TdW1tZXJzPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkx1LCBaaGl5b25nPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5aaGl5b25nPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkx1PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5VdGlsaXppbmcgTG9uZ2l0dWRpbmFsIENoZXN0IFgtUmF5cyBhbmQgUmVwb3J0cyB0byBQcmUtRmlsbCBSYWRpb2xvZ3kgUmVwb3J0czwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAyMzwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPkNvbXB1dGF0aW9uIGFuZCBMYW5ndWFnZSAoY3MuQ0wpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoY3MuTEcpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMy0wNi0xNFQyMToxNzozMVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMy0wNi0xNlQwMDozMDo1Mlo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIzLTEwLTEwVDE3OjI5OjUyWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDIzLTEwLTExVDAwOjM1OjI1WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjMtMDY8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MjwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vY3JlYXRpdmVjb21tb25zLm9yZy9saWNlbnNlcy9ieS80LjAvIiByaWdodHNJZGVudGlmaWVyU2NoZW1lPSJTUERYIiByaWdodHNJZGVudGlmaWVyPSJDQy1CWS00LjAiPkNyZWF0aXZlIENvbW1vbnMgQXR0cmlidXRpb24gNC4wIEludGVybmF0aW9uYWw8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5EZXNwaXRlIHRoZSByZWR1Y3Rpb24gaW4gdHVybi1hcm91bmQgdGltZXMgaW4gcmFkaW9sb2d5IHJlcG9ydHMgd2l0aCB0aGUgdXNlIG9mIHNwZWVjaCByZWNvZ25pdGlvbiBzb2Z0d2FyZSwgcGVyc2lzdGVudCBjb21tdW5pY2F0aW9uIGVycm9ycyBjYW4gc2lnbmlmaWNhbnRseSBpbXBhY3QgdGhlIGludGVycHJldGF0aW9uIG9mIHRoZSByYWRpb2xvZ3kgcmVwb3J0LiBQcmUtZmlsbGluZyBhIHJhZGlvbG9neSByZXBvcnQgaG9sZHMgcHJvbWlzZSBpbiBtaXRpZ2F0aW5nIHJlcG9ydGluZyBlcnJvcnMsIGFuZCBkZXNwaXRlIGVmZm9ydHMgaW4gdGhlIGxpdGVyYXR1cmUgdG8gZ2VuZXJhdGUgbWVkaWNhbCByZXBvcnRzLCB0aGVyZSBleGlzdHMgYSBsYWNrIG9mIGFwcHJvYWNoZXMgdGhhdCBleHBsb2l0IHRoZSBsb25naXR1ZGluYWwgbmF0dXJlIG9mIHBhdGllbnQgdmlzaXQgcmVjb3JkcyBpbiB0aGUgTUlNSUMtQ1hSIGRhdGFzZXQuIFRvIGFkZHJlc3MgdGhpcyBnYXAsIHdlIHByb3Bvc2UgdG8gdXNlIGxvbmdpdHVkaW5hbCBtdWx0aS1tb2RhbCBkYXRhLCBpLmUuLCBwcmV2aW91cyBwYXRpZW50IHZpc2l0IENYUiwgY3VycmVudCB2aXNpdCBDWFIsIGFuZCBwcmV2aW91cyB2aXNpdCByZXBvcnQsIHRvIHByZS1maWxsIHRoZSAnZmluZGluZ3MnIHNlY3Rpb24gb2YgYSBjdXJyZW50IHBhdGllbnQgdmlzaXQgcmVwb3J0LiBXZSBmaXJzdCBnYXRoZXJlZCB0aGUgbG9uZ2l0dWRpbmFsIHZpc2l0IGluZm9ybWF0aW9uIGZvciAyNiw2MjUgcGF0aWVudHMgZnJvbSB0aGUgTUlNSUMtQ1hSIGRhdGFzZXQgYW5kIGNyZWF0ZWQgYSBuZXcgZGF0YXNldCBjYWxsZWQgTG9uZ2l0dWRpbmFsLU1JTUlDLiBXaXRoIHRoaXMgbmV3IGRhdGFzZXQsIGEgdHJhbnNmb3JtZXItYmFzZWQgbW9kZWwgd2FzIHRyYWluZWQgdG8gY2FwdHVyZSB0aGUgaW5mb3JtYXRpb24gZnJvbSBsb25naXR1ZGluYWwgcGF0aWVudCB2aXNpdCByZWNvcmRzIGNvbnRhaW5pbmcgbXVsdGktbW9kYWwgZGF0YSAoQ1hSIGltYWdlcyArIHJlcG9ydHMpIHZpYSBhIGNyb3NzLWF0dGVudGlvbi1iYXNlZCBtdWx0aS1tb2RhbCBmdXNpb24gbW9kdWxlIGFuZCBhIGhpZXJhcmNoaWNhbCBtZW1vcnktZHJpdmVuIGRlY29kZXIuIEluIGNvbnRyYXN0IHRvIHByZXZpb3VzIHdvcmsgdGhhdCBvbmx5IHVzZXMgY3VycmVudCB2aXNpdCBkYXRhIGFzIGlucHV0IHRvIHRyYWluIGEgbW9kZWwsIG91ciB3b3JrIGV4cGxvaXRzIHRoZSBsb25naXR1ZGluYWwgaW5mb3JtYXRpb24gYXZhaWxhYmxlIHRvIHByZS1maWxsIHRoZSAnZmluZGluZ3MnIHNlY3Rpb24gb2YgcmFkaW9sb2d5IHJlcG9ydHMuIEV4cGVyaW1lbnRzIHNob3cgdGhhdCBvdXIgYXBwcm9hY2ggb3V0cGVyZm9ybXMgc2V2ZXJhbCByZWNlbnQgYXBwcm9hY2hlcy4gQ29kZSB3aWxsIGJlIHB1Ymxpc2hlZCBhdCBodHRwczovL2dpdGh1Yi5jb20vQ2VsZXN0aWFsU2hpbmUvTG9uZ2l0dWRpbmFsLUNoZXN0LVgtUmF5LjwvZGVzY3JpcHRpb24+CiAgPC9kZXNjcmlwdGlvbnM+CjwvcmVzb3VyY2U+","url":"https://arxiv.org/abs/2306.08749","contentUrl":null,"metadataVersion":1,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2023-06-16T17:15:21.000Z","registered":"2023-06-16T17:15:22.000Z","published":"2023","updated":"2023-10-11T01:46:48.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2306.08749","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}