{"data":{"id":"10.48550/arxiv.2111.08685","type":"dois","attributes":{"doi":"10.48550/arxiv.2111.08685","prefix":"10.48550","suffix":"arxiv.2111.08685","identifiers":[{"identifier":"2111.08685","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2111.08685"}],"creators":[{"name":"Shi, Yue","nameType":"Personal","givenName":"Yue","familyName":"Shi","affiliation":[],"nameIdentifiers":[]},{"name":"Han, Liangxiu","nameType":"Personal","givenName":"Liangxiu","familyName":"Han","affiliation":[],"nameIdentifiers":[]},{"name":"Han, Lianghao","nameType":"Personal","givenName":"Lianghao","familyName":"Han","affiliation":[],"nameIdentifiers":[]},{"name":"Chang, Sheng","nameType":"Personal","givenName":"Sheng","familyName":"Chang","affiliation":[],"nameIdentifiers":[]},{"name":"Hu, Tongle","nameType":"Personal","givenName":"Tongle","familyName":"Hu","affiliation":[],"nameIdentifiers":[]},{"name":"Dancey, Darren","nameType":"Personal","givenName":"Darren","familyName":"Dancey","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution"}],"publisher":"arXiv","container":{},"publicationYear":2021,"subjects":[{"lang":"en","subject":"Image and Video Processing (eess.IV)","subjectScheme":"arXiv"},{"lang":"en","subject":"Computer Vision and Pattern Recognition (cs.CV)","subjectScheme":"arXiv"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Electrical engineering, electronic engineering, information engineering","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2021-11-16T18:40:19Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-10-05T13:35:07Z","dateType":"Updated","dateInformation":"v1"},{"date":"2021-11","dateType":"Available","dateInformation":"v1"},{"date":"2021","dateType":"Issued"}],"language":null,"types":{"ris":"RPRT","bibtex":"article","citeproc":"article-journal","schemaOrg":"ScholarlyArticle","resourceType":"Article","resourceTypeGeneral":"Text"},"relatedIdentifiers":[{"relationType":"IsVersionOf","relatedIdentifier":"10.1109/tgrs.2022.3193441","relatedIdentifierType":"DOI"}],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"Creative Commons Zero v1.0 Universal","rightsUri":"https://creativecommons.org/publicdomain/zero/1.0/legalcode","schemeUri":"https://spdx.org/licenses/","rightsIdentifier":"cc0-1.0","rightsIdentifierScheme":"SPDX"}],"descriptions":[{"description":"Realistic hyperspectral image (HSI) super-resolution (SR) techniques aim to generate a high-resolution (HR) HSI with higher spectral and spatial fidelity from its low-resolution (LR) counterpart. The generative adversarial network (GAN) has proven to be an effective deep learning framework for image super-resolution. However, the optimisation process of existing GAN-based models frequently suffers from the problem of mode collapse, leading to the limited capacity of spectral-spatial invariant reconstruction. This may cause the spectral-spatial distortion on the generated HSI, especially with a large upscaling factor. To alleviate the problem of mode collapse, this work has proposed a novel GAN model coupled with a latent encoder (LE-GAN), which can map the generated spectral-spatial features from the image space to the latent space and produce a coupling component to regularise the generated samples. Essentially, we treat an HSI as a high-dimensional manifold embedded in a latent space. Thus, the optimisation of GAN models is converted to the problem of learning the distributions of high-resolution HSI samples in the latent space, making the distributions of the generated super-resolution HSIs closer to those of their original high-resolution counterparts. We have conducted experimental evaluations on the model performance of super-resolution and its capability in alleviating mode collapse. The proposed approach has been tested and validated based on two real HSI datasets with different sensors (i.e. AVIRIS and UHD-185) for various upscaling factors and added noise levels, and compared with the state-of-the-art super-resolution models (i.e. HyCoNet, LTTR, BAGAN, SR- GAN, WGAN).","descriptionType":"Abstract"},{"description":"18 pages, 10 figures","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMTExLjA4Njg1PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjExMS4wODY4NTwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5TaGksIFl1ZTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WXVlPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlNoaTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5IYW4sIExpYW5neGl1PC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5MaWFuZ3hpdTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5IYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+SGFuLCBMaWFuZ2hhbzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+TGlhbmdoYW88L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+SGFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkNoYW5nLCBTaGVuZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+U2hlbmc8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+Q2hhbmc8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+SHUsIFRvbmdsZTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+VG9uZ2xlPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkh1PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkRhbmNleSwgRGFycmVuPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5EYXJyZW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+RGFuY2V5PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5BIExhdGVudCBFbmNvZGVyIENvdXBsZWQgR2VuZXJhdGl2ZSBBZHZlcnNhcmlhbCBOZXR3b3JrIChMRS1HQU4pIGZvciBFZmZpY2llbnQgSHlwZXJzcGVjdHJhbCBJbWFnZSBTdXBlci1yZXNvbHV0aW9uPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDIxPC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+SW1hZ2UgYW5kIFZpZGVvIFByb2Nlc3NpbmcgKGVlc3MuSVYpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+Q29tcHV0ZXIgVmlzaW9uIGFuZCBQYXR0ZXJuIFJlY29nbml0aW9uIChjcy5DVik8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBFbGVjdHJpY2FsIGVuZ2luZWVyaW5nLCBlbGVjdHJvbmljIGVuZ2luZWVyaW5nLCBpbmZvcm1hdGlvbiBlbmdpbmVlcmluZzwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IENvbXB1dGVyIGFuZCBpbmZvcm1hdGlvbiBzY2llbmNlczwvc3ViamVjdD4KICA8L3N1YmplY3RzPgogIDxkYXRlcz4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjEtMTEtMTZUMTg6NDA6MTlaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMjItMTAtMDVUMTM6MzU6MDdaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IkF2YWlsYWJsZSIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMS0xMTwvZGF0ZT4KICA8L2RhdGVzPgogIDxyZXNvdXJjZVR5cGUgcmVzb3VyY2VUeXBlR2VuZXJhbD0iVGV4dCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDxyZWxhdGVkSWRlbnRpZmllcnM+CiAgICA8cmVsYXRlZElkZW50aWZpZXIgcmVsYXRlZElkZW50aWZpZXJUeXBlPSJET0kiIHJlbGF0aW9uVHlwZT0iSXNWZXJzaW9uT2YiPjEwLjExMDkvVEdSUy4yMDIyLjMxOTM0NDE8L3JlbGF0ZWRJZGVudGlmaWVyPgogIDwvcmVsYXRlZElkZW50aWZpZXJzPgogIDx2ZXJzaW9uPjE8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2NyZWF0aXZlY29tbW9ucy5vcmcvcHVibGljZG9tYWluL3plcm8vMS4wLyIgcmlnaHRzSWRlbnRpZmllclNjaGVtZT0iU1BEWCIgcmlnaHRzSWRlbnRpZmllcj0iQ0MwLTEuMCI+Q3JlYXRpdmUgQ29tbW9ucyBaZXJvIHYxLjAgVW5pdmVyc2FsPC9yaWdodHM+CiAgPC9yaWdodHNMaXN0PgogIDxkZXNjcmlwdGlvbnM+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJBYnN0cmFjdCI+UmVhbGlzdGljIGh5cGVyc3BlY3RyYWwgaW1hZ2UgKEhTSSkgc3VwZXItcmVzb2x1dGlvbiAoU1IpIHRlY2huaXF1ZXMgYWltIHRvIGdlbmVyYXRlIGEgaGlnaC1yZXNvbHV0aW9uIChIUikgSFNJIHdpdGggaGlnaGVyIHNwZWN0cmFsIGFuZCBzcGF0aWFsIGZpZGVsaXR5IGZyb20gaXRzIGxvdy1yZXNvbHV0aW9uIChMUikgY291bnRlcnBhcnQuIFRoZSBnZW5lcmF0aXZlIGFkdmVyc2FyaWFsIG5ldHdvcmsgKEdBTikgaGFzIHByb3ZlbiB0byBiZSBhbiBlZmZlY3RpdmUgZGVlcCBsZWFybmluZyBmcmFtZXdvcmsgZm9yIGltYWdlIHN1cGVyLXJlc29sdXRpb24uIEhvd2V2ZXIsIHRoZSBvcHRpbWlzYXRpb24gcHJvY2VzcyBvZiBleGlzdGluZyBHQU4tYmFzZWQgbW9kZWxzIGZyZXF1ZW50bHkgc3VmZmVycyBmcm9tIHRoZSBwcm9ibGVtIG9mIG1vZGUgY29sbGFwc2UsIGxlYWRpbmcgdG8gdGhlIGxpbWl0ZWQgY2FwYWNpdHkgb2Ygc3BlY3RyYWwtc3BhdGlhbCBpbnZhcmlhbnQgcmVjb25zdHJ1Y3Rpb24uIFRoaXMgbWF5IGNhdXNlIHRoZSBzcGVjdHJhbC1zcGF0aWFsIGRpc3RvcnRpb24gb24gdGhlIGdlbmVyYXRlZCBIU0ksIGVzcGVjaWFsbHkgd2l0aCBhIGxhcmdlIHVwc2NhbGluZyBmYWN0b3IuIFRvIGFsbGV2aWF0ZSB0aGUgcHJvYmxlbSBvZiBtb2RlIGNvbGxhcHNlLCB0aGlzIHdvcmsgaGFzIHByb3Bvc2VkIGEgbm92ZWwgR0FOIG1vZGVsIGNvdXBsZWQgd2l0aCBhIGxhdGVudCBlbmNvZGVyIChMRS1HQU4pLCB3aGljaCBjYW4gbWFwIHRoZSBnZW5lcmF0ZWQgc3BlY3RyYWwtc3BhdGlhbCBmZWF0dXJlcyBmcm9tIHRoZSBpbWFnZSBzcGFjZSB0byB0aGUgbGF0ZW50IHNwYWNlIGFuZCBwcm9kdWNlIGEgY291cGxpbmcgY29tcG9uZW50IHRvIHJlZ3VsYXJpc2UgdGhlIGdlbmVyYXRlZCBzYW1wbGVzLiBFc3NlbnRpYWxseSwgd2UgdHJlYXQgYW4gSFNJIGFzIGEgaGlnaC1kaW1lbnNpb25hbCBtYW5pZm9sZCBlbWJlZGRlZCBpbiBhIGxhdGVudCBzcGFjZS4gVGh1cywgdGhlIG9wdGltaXNhdGlvbiBvZiBHQU4gbW9kZWxzIGlzIGNvbnZlcnRlZCB0byB0aGUgcHJvYmxlbSBvZiBsZWFybmluZyB0aGUgZGlzdHJpYnV0aW9ucyBvZiBoaWdoLXJlc29sdXRpb24gSFNJIHNhbXBsZXMgaW4gdGhlIGxhdGVudCBzcGFjZSwgbWFraW5nIHRoZSBkaXN0cmlidXRpb25zIG9mIHRoZSBnZW5lcmF0ZWQgc3VwZXItcmVzb2x1dGlvbiBIU0lzIGNsb3NlciB0byB0aG9zZSBvZiB0aGVpciBvcmlnaW5hbCBoaWdoLXJlc29sdXRpb24gY291bnRlcnBhcnRzLiBXZSBoYXZlIGNvbmR1Y3RlZCBleHBlcmltZW50YWwgZXZhbHVhdGlvbnMgb24gdGhlIG1vZGVsIHBlcmZvcm1hbmNlIG9mIHN1cGVyLXJlc29sdXRpb24gYW5kIGl0cyBjYXBhYmlsaXR5IGluIGFsbGV2aWF0aW5nIG1vZGUgY29sbGFwc2UuIFRoZSBwcm9wb3NlZCBhcHByb2FjaCBoYXMgYmVlbiB0ZXN0ZWQgYW5kIHZhbGlkYXRlZCBiYXNlZCBvbiB0d28gcmVhbCBIU0kgZGF0YXNldHMgd2l0aCBkaWZmZXJlbnQgc2Vuc29ycyAoaS5lLiBBVklSSVMgYW5kIFVIRC0xODUpIGZvciB2YXJpb3VzIHVwc2NhbGluZyBmYWN0b3JzIGFuZCBhZGRlZCBub2lzZSBsZXZlbHMsIGFuZCBjb21wYXJlZCB3aXRoIHRoZSBzdGF0ZS1vZi10aGUtYXJ0IHN1cGVyLXJlc29sdXRpb24gbW9kZWxzIChpLmUuIEh5Q29OZXQsIExUVFIsIEJBR0FOLCBTUi0gR0FOLCBXR0FOKS48L2Rlc2NyaXB0aW9uPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iT3RoZXIiPjE4IHBhZ2VzLCAxMCBmaWd1cmVzPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2111.08685","contentUrl":null,"metadataVersion":1,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":1,"created":"2022-02-20T00:25:24.000Z","registered":"2022-02-20T00:25:25.000Z","published":"2021","updated":"2022-10-06T01:10:58.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2111.08685","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[{"id":"10.1109/tgrs.2022.3193441","type":"dois"}]}}}}