{"data":{"id":"10.48550/arxiv.2203.13963","type":"dois","attributes":{"doi":"10.48550/arxiv.2203.13963","prefix":"10.48550","suffix":"arxiv.2203.13963","identifiers":[{"identifier":"2203.13963","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2203.13963"}],"creators":[{"name":"Li, Guangyuan","nameType":"Personal","givenName":"Guangyuan","familyName":"Li","affiliation":[],"nameIdentifiers":[]},{"name":"Lv, Jun","nameType":"Personal","givenName":"Jun","familyName":"Lv","affiliation":[],"nameIdentifiers":[]},{"name":"Tian, Yapeng","nameType":"Personal","givenName":"Yapeng","familyName":"Tian","affiliation":[],"nameIdentifiers":[]},{"name":"Dou, Qi","nameType":"Personal","givenName":"Qi","familyName":"Dou","affiliation":[],"nameIdentifiers":[]},{"name":"Wang, Chengyan","nameType":"Personal","givenName":"Chengyan","familyName":"Wang","affiliation":[],"nameIdentifiers":[]},{"name":"Xu, Chenliang","nameType":"Personal","givenName":"Chenliang","familyName":"Xu","affiliation":[],"nameIdentifiers":[]},{"name":"Qin, Jing","nameType":"Personal","givenName":"Jing","familyName":"Qin","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution"}],"publisher":"arXiv","container":{},"publicationYear":2022,"subjects":[{"lang":"en","subject":"Image and Video Processing (eess.IV)","subjectScheme":"arXiv"},{"lang":"en","subject":"Computer Vision and Pattern Recognition (cs.CV)","subjectScheme":"arXiv"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2022-03-26T01:42:59Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-03-29T00:32:48Z","dateType":"Updated","dateInformation":"v1"},{"date":"2022-03","dateType":"Available","dateInformation":"v1"},{"date":"2022","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"Creative Commons Attribution 4.0 International","rightsUri":"https://creativecommons.org/licenses/by/4.0/legalcode","schemeUri":"https://spdx.org/licenses/","rightsIdentifier":"cc-by-4.0","rightsIdentifierScheme":"SPDX"}],"descriptions":[{"description":"Magnetic resonance imaging (MRI) can present multi-contrast images of the same anatomical structures, enabling multi-contrast super-resolution (SR) techniques. Compared with SR reconstruction using a single-contrast, multi-contrast SR reconstruction is promising to yield SR images with higher quality by leveraging diverse yet complementary information embedded in different imaging modalities. However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures. We propose a novel network to comprehensively address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques; we call it McMRSR. Firstly, we tame transformers to model long-range dependencies in both reference and target images. Then, a new multi-scale contextual matching method is proposed to capture corresponding contexts from reference features at different scales. Furthermore, we introduce a multi-scale aggregation mechanism to gradually and interactively aggregate multi-scale matched features for reconstructing the target SR MR image. Extensive experiments demonstrate that our network outperforms state-of-the-art approaches and has great potential to be applied in clinical practice. Codes are available at https://github.com/XAIMI-Lab/McMRSR.","descriptionType":"Abstract"},{"description":"CVPR 2022 accepted","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMjAzLjEzOTYzPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjIwMy4xMzk2MzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5MaSwgR3Vhbmd5dWFuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5HdWFuZ3l1YW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+TGk8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+THYsIEp1bjwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+SnVuPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkx2PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlRpYW4sIFlhcGVuZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WWFwZW5nPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlRpYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+RG91LCBRaTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+UWk8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+RG91PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPldhbmcsIENoZW5neWFuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5DaGVuZ3lhbjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5XYW5nPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlh1LCBDaGVubGlhbmc8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkNoZW5saWFuZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5YdTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5RaW4sIEppbmc8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkppbmc8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+UWluPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5UcmFuc2Zvcm1lci1lbXBvd2VyZWQgTXVsdGktc2NhbGUgQ29udGV4dHVhbCBNYXRjaGluZyBhbmQgQWdncmVnYXRpb24gZm9yIE11bHRpLWNvbnRyYXN0IE1SSSBTdXBlci1yZXNvbHV0aW9uPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDIyPC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+SW1hZ2UgYW5kIFZpZGVvIFByb2Nlc3NpbmcgKGVlc3MuSVYpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+Q29tcHV0ZXIgVmlzaW9uIGFuZCBQYXR0ZXJuIFJlY29nbml0aW9uIChjcy5DVik8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBFbGVjdHJpY2FsIGVuZ2luZWVyaW5nLCBlbGVjdHJvbmljIGVuZ2luZWVyaW5nLCBpbmZvcm1hdGlvbiBlbmdpbmVlcmluZzwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IENvbXB1dGVyIGFuZCBpbmZvcm1hdGlvbiBzY2llbmNlczwvc3ViamVjdD4KICA8L3N1YmplY3RzPgogIDxkYXRlcz4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjItMDMtMjZUMDE6NDI6NTlaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjItMDMtMjlUMDA6MzI6NDhaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IkF2YWlsYWJsZSIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0wMzwvZGF0ZT4KICA8L2RhdGVzPgogIDxyZXNvdXJjZVR5cGUgcmVzb3VyY2VUeXBlR2VuZXJhbD0iUHJlcHJpbnQiPkFydGljbGU8L3Jlc291cmNlVHlwZT4KICA8dmVyc2lvbj4xPC92ZXJzaW9uPgogIDxyaWdodHNMaXN0PgogICAgPHJpZ2h0cyByaWdodHNVUkk9Imh0dHA6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LzQuMC8iIHJpZ2h0c0lkZW50aWZpZXJTY2hlbWU9IlNQRFgiIHJpZ2h0c0lkZW50aWZpZXI9IkNDLUJZLTQuMCI+Q3JlYXRpdmUgQ29tbW9ucyBBdHRyaWJ1dGlvbiA0LjAgSW50ZXJuYXRpb25hbDwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPk1hZ25ldGljIHJlc29uYW5jZSBpbWFnaW5nIChNUkkpIGNhbiBwcmVzZW50IG11bHRpLWNvbnRyYXN0IGltYWdlcyBvZiB0aGUgc2FtZSBhbmF0b21pY2FsIHN0cnVjdHVyZXMsIGVuYWJsaW5nIG11bHRpLWNvbnRyYXN0IHN1cGVyLXJlc29sdXRpb24gKFNSKSB0ZWNobmlxdWVzLiBDb21wYXJlZCB3aXRoIFNSIHJlY29uc3RydWN0aW9uIHVzaW5nIGEgc2luZ2xlLWNvbnRyYXN0LCBtdWx0aS1jb250cmFzdCBTUiByZWNvbnN0cnVjdGlvbiBpcyBwcm9taXNpbmcgdG8geWllbGQgU1IgaW1hZ2VzIHdpdGggaGlnaGVyIHF1YWxpdHkgYnkgbGV2ZXJhZ2luZyBkaXZlcnNlIHlldCBjb21wbGVtZW50YXJ5IGluZm9ybWF0aW9uIGVtYmVkZGVkIGluIGRpZmZlcmVudCBpbWFnaW5nIG1vZGFsaXRpZXMuIEhvd2V2ZXIsIGV4aXN0aW5nIG1ldGhvZHMgc3RpbGwgaGF2ZSB0d28gc2hvcnRjb21pbmdzOiAoMSkgdGhleSBuZWdsZWN0IHRoYXQgdGhlIG11bHRpLWNvbnRyYXN0IGZlYXR1cmVzIGF0IGRpZmZlcmVudCBzY2FsZXMgY29udGFpbiBkaWZmZXJlbnQgYW5hdG9taWNhbCBkZXRhaWxzIGFuZCBoZW5jZSBsYWNrIGVmZmVjdGl2ZSBtZWNoYW5pc21zIHRvIG1hdGNoIGFuZCBmdXNlIHRoZXNlIGZlYXR1cmVzIGZvciBiZXR0ZXIgcmVjb25zdHJ1Y3Rpb247IGFuZCAoMikgdGhleSBhcmUgc3RpbGwgZGVmaWNpZW50IGluIGNhcHR1cmluZyBsb25nLXJhbmdlIGRlcGVuZGVuY2llcywgd2hpY2ggYXJlIGVzc2VudGlhbCBmb3IgdGhlIHJlZ2lvbnMgd2l0aCBjb21wbGljYXRlZCBhbmF0b21pY2FsIHN0cnVjdHVyZXMuIFdlIHByb3Bvc2UgYSBub3ZlbCBuZXR3b3JrIHRvIGNvbXByZWhlbnNpdmVseSBhZGRyZXNzIHRoZXNlIHByb2JsZW1zIGJ5IGRldmVsb3BpbmcgYSBzZXQgb2YgaW5ub3ZhdGl2ZSBUcmFuc2Zvcm1lci1lbXBvd2VyZWQgbXVsdGktc2NhbGUgY29udGV4dHVhbCBtYXRjaGluZyBhbmQgYWdncmVnYXRpb24gdGVjaG5pcXVlczsgd2UgY2FsbCBpdCBNY01SU1IuIEZpcnN0bHksIHdlIHRhbWUgdHJhbnNmb3JtZXJzIHRvIG1vZGVsIGxvbmctcmFuZ2UgZGVwZW5kZW5jaWVzIGluIGJvdGggcmVmZXJlbmNlIGFuZCB0YXJnZXQgaW1hZ2VzLiBUaGVuLCBhIG5ldyBtdWx0aS1zY2FsZSBjb250ZXh0dWFsIG1hdGNoaW5nIG1ldGhvZCBpcyBwcm9wb3NlZCB0byBjYXB0dXJlIGNvcnJlc3BvbmRpbmcgY29udGV4dHMgZnJvbSByZWZlcmVuY2UgZmVhdHVyZXMgYXQgZGlmZmVyZW50IHNjYWxlcy4gRnVydGhlcm1vcmUsIHdlIGludHJvZHVjZSBhIG11bHRpLXNjYWxlIGFnZ3JlZ2F0aW9uIG1lY2hhbmlzbSB0byBncmFkdWFsbHkgYW5kIGludGVyYWN0aXZlbHkgYWdncmVnYXRlIG11bHRpLXNjYWxlIG1hdGNoZWQgZmVhdHVyZXMgZm9yIHJlY29uc3RydWN0aW5nIHRoZSB0YXJnZXQgU1IgTVIgaW1hZ2UuIEV4dGVuc2l2ZSBleHBlcmltZW50cyBkZW1vbnN0cmF0ZSB0aGF0IG91ciBuZXR3b3JrIG91dHBlcmZvcm1zIHN0YXRlLW9mLXRoZS1hcnQgYXBwcm9hY2hlcyBhbmQgaGFzIGdyZWF0IHBvdGVudGlhbCB0byBiZSBhcHBsaWVkIGluIGNsaW5pY2FsIHByYWN0aWNlLiBDb2RlcyBhcmUgYXZhaWxhYmxlIGF0IGh0dHBzOi8vZ2l0aHViLmNvbS9YQUlNSS1MYWIvTWNNUlNSLjwvZGVzY3JpcHRpb24+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJPdGhlciI+Q1ZQUiAyMDIyIGFjY2VwdGVkPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2203.13963","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-29T02:14:35.000Z","registered":"2022-03-29T02:14:38.000Z","published":"2022","updated":"2022-03-29T02:14:38.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2203.13963","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}