{"data":{"id":"10.48550/arxiv.2204.12109","type":"dois","attributes":{"doi":"10.48550/arxiv.2204.12109","prefix":"10.48550","suffix":"arxiv.2204.12109","identifiers":[{"identifier":"2204.12109","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2204.12109"}],"creators":[{"name":"Liu, Chang","nameType":"Personal","givenName":"Chang","familyName":"Liu","affiliation":[],"nameIdentifiers":[]},{"name":"Jiang, Xudong","nameType":"Personal","givenName":"Xudong","familyName":"Jiang","affiliation":[],"nameIdentifiers":[]},{"name":"Ding, Henghui","nameType":"Personal","givenName":"Henghui","familyName":"Ding","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Instance-Specific Feature Propagation for Referring Segmentation"}],"publisher":"arXiv","container":{},"publicationYear":2022,"subjects":[{"lang":"en","subject":"Computer Vision and Pattern Recognition (cs.CV)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2022-04-26T07:08:14Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-04-27T00:16:09Z","dateType":"Updated","dateInformation":"v1"},{"date":"2022-04","dateType":"Available","dateInformation":"v1"},{"date":"2022","dateType":"Issued"}],"language":null,"types":{"ris":"RPRT","bibtex":"article","citeproc":"article-journal","schemaOrg":"ScholarlyArticle","resourceType":"Article","resourceTypeGeneral":"Text"},"relatedIdentifiers":[{"relationType":"IsVersionOf","relatedIdentifier":"10.1109/tmm.2022.3163578","relatedIdentifierType":"DOI"}],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"Referring segmentation aims to generate a segmentation mask for the target instance indicated by a natural language expression. There are typically two kinds of existing methods: one-stage methods that directly perform segmentation on the fused vision and language features; and two-stage methods that first utilize an instance segmentation model for instance proposal and then select one of these instances via matching them with language features. In this work, we propose a novel framework that simultaneously detects the target-of-interest via feature propagation and generates a fine-grained segmentation mask. In our framework, each instance is represented by an Instance-Specific Feature (ISF), and the target-of-referring is identified by exchanging information among all ISFs using our proposed Feature Propagation Module (FPM). Our instance-aware approach learns the relationship among all objects, which helps to better locate the target-of-interest than one-stage methods. Comparing to two-stage methods, our approach collaboratively and interactively utilizes both vision and language information for synchronous identification and segmentation. In the experimental tests, our method outperforms previous state-of-the-art methods on all three RefCOCO series datasets.","descriptionType":"Abstract"},{"description":"TMM","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMjA0LjEyMTA5PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjIwNC4xMjEwOTwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5MaXUsIENoYW5nPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5DaGFuZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5MaXU8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+SmlhbmcsIFh1ZG9uZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WHVkb25nPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkppYW5nPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkRpbmcsIEhlbmdodWk8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkhlbmdodWk8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+RGluZzwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICA8L2NyZWF0b3JzPgogIDx0aXRsZXM+CiAgICA8dGl0bGU+SW5zdGFuY2UtU3BlY2lmaWMgRmVhdHVyZSBQcm9wYWdhdGlvbiBmb3IgUmVmZXJyaW5nIFNlZ21lbnRhdGlvbjwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAyMjwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPkNvbXB1dGVyIFZpc2lvbiBhbmQgUGF0dGVybiBSZWNvZ25pdGlvbiAoY3MuQ1YpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0wNC0yNlQwNzowODoxNFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMi0wNC0yN1QwMDoxNjowOVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIyLTA0PC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJUZXh0Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHJlbGF0ZWRJZGVudGlmaWVycz4KICAgIDxyZWxhdGVkSWRlbnRpZmllciByZWxhdGVkSWRlbnRpZmllclR5cGU9IkRPSSIgcmVsYXRpb25UeXBlPSJJc1ZlcnNpb25PZiI+MTAuMTEwOS9UTU0uMjAyMi4zMTYzNTc4PC9yZWxhdGVkSWRlbnRpZmllcj4KICA8L3JlbGF0ZWRJZGVudGlmaWVycz4KICA8dmVyc2lvbj4xPC92ZXJzaW9uPgogIDxyaWdodHNMaXN0PgogICAgPHJpZ2h0cyByaWdodHNVUkk9Imh0dHA6Ly9hcnhpdi5vcmcvbGljZW5zZXMvbm9uZXhjbHVzaXZlLWRpc3RyaWIvMS4wLyI+YXJYaXYub3JnIHBlcnBldHVhbCwgbm9uLWV4Y2x1c2l2ZSBsaWNlbnNlPC9yaWdodHM+CiAgPC9yaWdodHNMaXN0PgogIDxkZXNjcmlwdGlvbnM+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJBYnN0cmFjdCI+UmVmZXJyaW5nIHNlZ21lbnRhdGlvbiBhaW1zIHRvIGdlbmVyYXRlIGEgc2VnbWVudGF0aW9uIG1hc2sgZm9yIHRoZSB0YXJnZXQgaW5zdGFuY2UgaW5kaWNhdGVkIGJ5IGEgbmF0dXJhbCBsYW5ndWFnZSBleHByZXNzaW9uLiBUaGVyZSBhcmUgdHlwaWNhbGx5IHR3byBraW5kcyBvZiBleGlzdGluZyBtZXRob2RzOiBvbmUtc3RhZ2UgbWV0aG9kcyB0aGF0IGRpcmVjdGx5IHBlcmZvcm0gc2VnbWVudGF0aW9uIG9uIHRoZSBmdXNlZCB2aXNpb24gYW5kIGxhbmd1YWdlIGZlYXR1cmVzOyBhbmQgdHdvLXN0YWdlIG1ldGhvZHMgdGhhdCBmaXJzdCB1dGlsaXplIGFuIGluc3RhbmNlIHNlZ21lbnRhdGlvbiBtb2RlbCBmb3IgaW5zdGFuY2UgcHJvcG9zYWwgYW5kIHRoZW4gc2VsZWN0IG9uZSBvZiB0aGVzZSBpbnN0YW5jZXMgdmlhIG1hdGNoaW5nIHRoZW0gd2l0aCBsYW5ndWFnZSBmZWF0dXJlcy4gSW4gdGhpcyB3b3JrLCB3ZSBwcm9wb3NlIGEgbm92ZWwgZnJhbWV3b3JrIHRoYXQgc2ltdWx0YW5lb3VzbHkgZGV0ZWN0cyB0aGUgdGFyZ2V0LW9mLWludGVyZXN0IHZpYSBmZWF0dXJlIHByb3BhZ2F0aW9uIGFuZCBnZW5lcmF0ZXMgYSBmaW5lLWdyYWluZWQgc2VnbWVudGF0aW9uIG1hc2suIEluIG91ciBmcmFtZXdvcmssIGVhY2ggaW5zdGFuY2UgaXMgcmVwcmVzZW50ZWQgYnkgYW4gSW5zdGFuY2UtU3BlY2lmaWMgRmVhdHVyZSAoSVNGKSwgYW5kIHRoZSB0YXJnZXQtb2YtcmVmZXJyaW5nIGlzIGlkZW50aWZpZWQgYnkgZXhjaGFuZ2luZyBpbmZvcm1hdGlvbiBhbW9uZyBhbGwgSVNGcyB1c2luZyBvdXIgcHJvcG9zZWQgRmVhdHVyZSBQcm9wYWdhdGlvbiBNb2R1bGUgKEZQTSkuIE91ciBpbnN0YW5jZS1hd2FyZSBhcHByb2FjaCBsZWFybnMgdGhlIHJlbGF0aW9uc2hpcCBhbW9uZyBhbGwgb2JqZWN0cywgd2hpY2ggaGVscHMgdG8gYmV0dGVyIGxvY2F0ZSB0aGUgdGFyZ2V0LW9mLWludGVyZXN0IHRoYW4gb25lLXN0YWdlIG1ldGhvZHMuIENvbXBhcmluZyB0byB0d28tc3RhZ2UgbWV0aG9kcywgb3VyIGFwcHJvYWNoIGNvbGxhYm9yYXRpdmVseSBhbmQgaW50ZXJhY3RpdmVseSB1dGlsaXplcyBib3RoIHZpc2lvbiBhbmQgbGFuZ3VhZ2UgaW5mb3JtYXRpb24gZm9yIHN5bmNocm9ub3VzIGlkZW50aWZpY2F0aW9uIGFuZCBzZWdtZW50YXRpb24uIEluIHRoZSBleHBlcmltZW50YWwgdGVzdHMsIG91ciBtZXRob2Qgb3V0cGVyZm9ybXMgcHJldmlvdXMgc3RhdGUtb2YtdGhlLWFydCBtZXRob2RzIG9uIGFsbCB0aHJlZSBSZWZDT0NPIHNlcmllcyBkYXRhc2V0cy48L2Rlc2NyaXB0aW9uPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iT3RoZXIiPlRNTTwvZGVzY3JpcHRpb24+CiAgPC9kZXNjcmlwdGlvbnM+CjwvcmVzb3VyY2U+","url":"https://arxiv.org/abs/2204.12109","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-04-27T01:13:37.000Z","registered":"2022-04-27T01:13:38.000Z","published":"2022","updated":"2022-04-27T01:13:38.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2204.12109","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}