{"data":{"id":"10.48550/arxiv.2209.00383","type":"dois","attributes":{"doi":"10.48550/arxiv.2209.00383","prefix":"10.48550","suffix":"arxiv.2209.00383","identifiers":[{"identifier":"2209.00383","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2209.00383"}],"creators":[{"name":"Wang, Yangtao","nameType":"Personal","givenName":"Yangtao","familyName":"Wang","affiliation":["M-PSI"],"nameIdentifiers":[]},{"name":"Shen, Xi","nameType":"Personal","givenName":"Xi","familyName":"Shen","affiliation":["MIT CSAIL"],"nameIdentifiers":[]},{"name":"Yuan, Yuan","nameType":"Personal","givenName":"Yuan","familyName":"Yuan","affiliation":["MIT CSAIL"],"nameIdentifiers":[]},{"name":"Du, Yuming","nameType":"Personal","givenName":"Yuming","familyName":"Du","affiliation":["M-PSI"],"nameIdentifiers":[]},{"name":"Li, Maomao","nameType":"Personal","givenName":"Maomao","familyName":"Li","affiliation":["M-PSI"],"nameIdentifiers":[]},{"name":"Hu, Shell Xu","nameType":"Personal","givenName":"Shell Xu","familyName":"Hu","affiliation":["M-PSI"],"nameIdentifiers":[]},{"name":"Crowley, James L","nameType":"Personal","givenName":"James L","familyName":"Crowley","affiliation":["M-PSI"],"nameIdentifiers":[]},{"name":"Vaufreydaz, Dominique","nameType":"Personal","givenName":"Dominique","familyName":"Vaufreydaz","affiliation":["M-PSI"],"nameIdentifiers":[]}],"titles":[{"title":"TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cut"}],"publisher":"arXiv","container":{},"publicationYear":2022,"subjects":[{"lang":"en","subject":"Computer Vision and Pattern Recognition (cs.CV)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2022-09-01T11:52:26Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2022-09-02T00:14:04Z","dateType":"Updated","dateInformation":"v1"},{"date":"2022-09-13T12:33:17Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2022-09-14T00:21:25Z","dateType":"Updated","dateInformation":"v2"},{"date":"2023-12-05T09:01:49Z","dateType":"Submitted","dateInformation":"v3"},{"date":"2023-12-06T01:18:11Z","dateType":"Updated","dateInformation":"v3"},{"date":"2022-09","dateType":"Available","dateInformation":"v1"},{"date":"2022","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"3","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"In this paper, we describe a graph-based algorithm that uses the features obtained by a self-supervised transformer to detect and segment salient objects in images and videos. With this approach, the image patches that compose an image or video are organised into a fully connected graph, where the edge between each pair of patches is labeled with a similarity score between patches using features learned by the transformer. Detection and segmentation of salient objects is then formulated as a graph-cut problem and solved using the classical Normalized Cut algorithm. Despite the simplicity of this approach, it achieves state-of-the-art results on several common image and video detection and segmentation tasks. For unsupervised object discovery, this approach outperforms the competing approaches by a margin of 6.1%, 5.7%, and 2.6%, respectively, when tested with the VOC07, VOC12, and COCO20K datasets. For the unsupervised saliency detection task in images, this method improves the score for Intersection over Union (IoU) by 4.4%, 5.6% and 5.2%. When tested with the ECSSD, DUTS, and DUT-OMRON datasets, respectively, compared to current state-of-the-art techniques. This method also achieves competitive results for unsupervised video object segmentation tasks with the DAVIS, SegTV2, and FBMS datasets.","descriptionType":"Abstract"},{"description":"arXiv admin note: text overlap with arXiv:2202.11539","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMjA5LjAwMzgzPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjIwOS4wMDM4MzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5XYW5nLCBZYW5ndGFvPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5ZYW5ndGFvPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPldhbmc8L2ZhbWlseU5hbWU+CiAgICAgIDxhZmZpbGlhdGlvbj5NLVBTSTwvYWZmaWxpYXRpb24+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+U2hlbiwgWGk8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlhpPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlNoZW48L2ZhbWlseU5hbWU+CiAgICAgIDxhZmZpbGlhdGlvbj5NSVQgQ1NBSUw8L2FmZmlsaWF0aW9uPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPll1YW4sIFl1YW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPll1YW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+WXVhbjwvZmFtaWx5TmFtZT4KICAgICAgPGFmZmlsaWF0aW9uPk1JVCBDU0FJTDwvYWZmaWxpYXRpb24+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+RHUsIFl1bWluZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WXVtaW5nPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkR1PC9mYW1pbHlOYW1lPgogICAgICA8YWZmaWxpYXRpb24+TS1QU0k8L2FmZmlsaWF0aW9uPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkxpLCBNYW9tYW88L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPk1hb21hbzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5MaTwvZmFtaWx5TmFtZT4KICAgICAgPGFmZmlsaWF0aW9uPk0tUFNJPC9hZmZpbGlhdGlvbj4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5IdSwgU2hlbGwgWHU8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlNoZWxsIFh1PC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkh1PC9mYW1pbHlOYW1lPgogICAgICA8YWZmaWxpYXRpb24+TS1QU0k8L2FmZmlsaWF0aW9uPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkNyb3dsZXksIEphbWVzIEw8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkphbWVzIEw8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+Q3Jvd2xleTwvZmFtaWx5TmFtZT4KICAgICAgPGFmZmlsaWF0aW9uPk0tUFNJPC9hZmZpbGlhdGlvbj4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5WYXVmcmV5ZGF6LCBEb21pbmlxdWU8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkRvbWluaXF1ZTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5WYXVmcmV5ZGF6PC9mYW1pbHlOYW1lPgogICAgICA8YWZmaWxpYXRpb24+TS1QU0k8L2FmZmlsaWF0aW9uPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5Ub2tlbkN1dDogU2VnbWVudGluZyBPYmplY3RzIGluIEltYWdlcyBhbmQgVmlkZW9zIHdpdGggU2VsZi1zdXBlcnZpc2VkIFRyYW5zZm9ybWVyIGFuZCBOb3JtYWxpemVkIEN1dDwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAyMjwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPkNvbXB1dGVyIFZpc2lvbiBhbmQgUGF0dGVybiBSZWNvZ25pdGlvbiAoY3MuQ1YpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoc3RhdC5NTCk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBDb21wdXRlciBhbmQgaW5mb3JtYXRpb24gc2NpZW5jZXM8L3N1YmplY3Q+CiAgPC9zdWJqZWN0cz4KICA8ZGF0ZXM+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIyLTA5LTAxVDExOjUyOjI2WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIyLTA5LTAyVDAwOjE0OjA0WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjIiPjIwMjItMDktMTNUMTI6MzM6MTdaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjIiPjIwMjItMDktMTRUMDA6MjE6MjVaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MyI+MjAyMy0xMi0wNVQwOTowMTo0OVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MyI+MjAyMy0xMi0wNlQwMToxODoxMVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIyLTA5PC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJQcmVwcmludCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDx2ZXJzaW9uPjM8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2FyeGl2Lm9yZy9saWNlbnNlcy9ub25leGNsdXNpdmUtZGlzdHJpYi8xLjAvIj5hclhpdi5vcmcgcGVycGV0dWFsLCBub24tZXhjbHVzaXZlIGxpY2Vuc2U8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5JbiB0aGlzIHBhcGVyLCB3ZSBkZXNjcmliZSBhIGdyYXBoLWJhc2VkIGFsZ29yaXRobSB0aGF0IHVzZXMgdGhlIGZlYXR1cmVzIG9idGFpbmVkIGJ5IGEgc2VsZi1zdXBlcnZpc2VkIHRyYW5zZm9ybWVyIHRvIGRldGVjdCBhbmQgc2VnbWVudCBzYWxpZW50IG9iamVjdHMgaW4gaW1hZ2VzIGFuZCB2aWRlb3MuIFdpdGggdGhpcyBhcHByb2FjaCwgdGhlIGltYWdlIHBhdGNoZXMgdGhhdCBjb21wb3NlIGFuIGltYWdlIG9yIHZpZGVvIGFyZSBvcmdhbmlzZWQgaW50byBhIGZ1bGx5IGNvbm5lY3RlZCBncmFwaCwgd2hlcmUgdGhlIGVkZ2UgYmV0d2VlbiBlYWNoIHBhaXIgb2YgcGF0Y2hlcyBpcyBsYWJlbGVkIHdpdGggYSBzaW1pbGFyaXR5IHNjb3JlIGJldHdlZW4gcGF0Y2hlcyB1c2luZyBmZWF0dXJlcyBsZWFybmVkIGJ5IHRoZSB0cmFuc2Zvcm1lci4gRGV0ZWN0aW9uIGFuZCBzZWdtZW50YXRpb24gb2Ygc2FsaWVudCBvYmplY3RzIGlzIHRoZW4gZm9ybXVsYXRlZCBhcyBhIGdyYXBoLWN1dCBwcm9ibGVtIGFuZCBzb2x2ZWQgdXNpbmcgdGhlIGNsYXNzaWNhbCBOb3JtYWxpemVkIEN1dCBhbGdvcml0aG0uIERlc3BpdGUgdGhlIHNpbXBsaWNpdHkgb2YgdGhpcyBhcHByb2FjaCwgaXQgYWNoaWV2ZXMgc3RhdGUtb2YtdGhlLWFydCByZXN1bHRzIG9uIHNldmVyYWwgY29tbW9uIGltYWdlIGFuZCB2aWRlbyBkZXRlY3Rpb24gYW5kIHNlZ21lbnRhdGlvbiB0YXNrcy4gRm9yIHVuc3VwZXJ2aXNlZCBvYmplY3QgZGlzY292ZXJ5LCB0aGlzIGFwcHJvYWNoIG91dHBlcmZvcm1zIHRoZSBjb21wZXRpbmcgYXBwcm9hY2hlcyBieSBhIG1hcmdpbiBvZiA2LjElLCA1LjclLCBhbmQgMi42JSwgcmVzcGVjdGl2ZWx5LCB3aGVuIHRlc3RlZCB3aXRoIHRoZSBWT0MwNywgVk9DMTIsIGFuZCBDT0NPMjBLIGRhdGFzZXRzLiBGb3IgdGhlIHVuc3VwZXJ2aXNlZCBzYWxpZW5jeSBkZXRlY3Rpb24gdGFzayBpbiBpbWFnZXMsIHRoaXMgbWV0aG9kIGltcHJvdmVzIHRoZSBzY29yZSBmb3IgSW50ZXJzZWN0aW9uIG92ZXIgVW5pb24gKElvVSkgYnkgNC40JSwgNS42JSBhbmQgNS4yJS4gV2hlbiB0ZXN0ZWQgd2l0aCB0aGUgRUNTU0QsIERVVFMsIGFuZCBEVVQtT01ST04gZGF0YXNldHMsIHJlc3BlY3RpdmVseSwgY29tcGFyZWQgdG8gY3VycmVudCBzdGF0ZS1vZi10aGUtYXJ0IHRlY2huaXF1ZXMuIFRoaXMgbWV0aG9kIGFsc28gYWNoaWV2ZXMgY29tcGV0aXRpdmUgcmVzdWx0cyBmb3IgdW5zdXBlcnZpc2VkIHZpZGVvIG9iamVjdCBzZWdtZW50YXRpb24gdGFza3Mgd2l0aCB0aGUgREFWSVMsIFNlZ1RWMiwgYW5kIEZCTVMgZGF0YXNldHMuPC9kZXNjcmlwdGlvbj4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9Ik90aGVyIj5hclhpdiBhZG1pbiBub3RlOiB0ZXh0IG92ZXJsYXAgd2l0aCBhclhpdjoyMjAyLjExNTM5PC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/2209.00383","contentUrl":null,"metadataVersion":2,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-09-02T00:57:16.000Z","registered":"2022-09-02T00:57:16.000Z","published":"2022","updated":"2023-12-06T02:34:32.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2209.00383","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}