{"data":{"id":"10.48550/arxiv.2106.03193","type":"dois","attributes":{"doi":"10.48550/arxiv.2106.03193","prefix":"10.48550","suffix":"arxiv.2106.03193","identifiers":[{"identifier":"2106.03193","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2106.03193"}],"creators":[{"name":"Goyal, Naman","nameType":"Personal","givenName":"Naman","familyName":"Goyal","affiliation":[],"nameIdentifiers":[]},{"name":"Gao, Cynthia","nameType":"Personal","givenName":"Cynthia","familyName":"Gao","affiliation":[],"nameIdentifiers":[]},{"name":"Chaudhary, Vishrav","nameType":"Personal","givenName":"Vishrav","familyName":"Chaudhary","affiliation":[],"nameIdentifiers":[]},{"name":"Chen, Peng-Jen","nameType":"Personal","givenName":"Peng-Jen","familyName":"Chen","affiliation":[],"nameIdentifiers":[]},{"name":"Wenzek, Guillaume","nameType":"Personal","givenName":"Guillaume","familyName":"Wenzek","affiliation":[],"nameIdentifiers":[]},{"name":"Ju, Da","nameType":"Personal","givenName":"Da","familyName":"Ju","affiliation":[],"nameIdentifiers":[]},{"name":"Krishnan, Sanjana","nameType":"Personal","givenName":"Sanjana","familyName":"Krishnan","affiliation":[],"nameIdentifiers":[]},{"name":"Ranzato, Marc'Aurelio","nameType":"Personal","givenName":"Marc'Aurelio","familyName":"Ranzato","affiliation":[],"nameIdentifiers":[]},{"name":"Guzman, Francisco","nameType":"Personal","givenName":"Francisco","familyName":"Guzman","affiliation":[],"nameIdentifiers":[]},{"name":"Fan, Angela","nameType":"Personal","givenName":"Angela","familyName":"Fan","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation"}],"publisher":"arXiv","container":{},"publicationYear":2021,"subjects":[{"lang":"en","subject":"Computation and Language (cs.CL)","subjectScheme":"arXiv"},{"lang":"en","subject":"Artificial Intelligence (cs.AI)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2021-06-06T17:58:12Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2021-06-08T00:27:54Z","dateType":"Updated","dateInformation":"v1"},{"date":"2021-06","dateType":"Available","dateInformation":"v1"},{"date":"2021","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES-101 evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yMTA2LjAzMTkzPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjEwNi4wMzE5MzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5Hb3lhbCwgTmFtYW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPk5hbWFuPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkdveWFsPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkdhbywgQ3ludGhpYTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+Q3ludGhpYTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5HYW88L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+Q2hhdWRoYXJ5LCBWaXNocmF2PC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5WaXNocmF2PC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkNoYXVkaGFyeTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5DaGVuLCBQZW5nLUplbjwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+UGVuZy1KZW48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+Q2hlbjwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5XZW56ZWssIEd1aWxsYXVtZTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+R3VpbGxhdW1lPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPldlbnplazwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5KdSwgRGE8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkRhPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkp1PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPktyaXNobmFuLCBTYW5qYW5hPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5TYW5qYW5hPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPktyaXNobmFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlJhbnphdG8sIE1hcmMnQXVyZWxpbzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+TWFyYydBdXJlbGlvPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPlJhbnphdG88L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+R3V6bWFuLCBGcmFuY2lzY288L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkZyYW5jaXNjbzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5HdXptYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+RmFuLCBBbmdlbGE8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkFuZ2VsYTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5GYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgPC9jcmVhdG9ycz4KICA8dGl0bGVzPgogICAgPHRpdGxlPlRoZSBGTE9SRVMtMTAxIEV2YWx1YXRpb24gQmVuY2htYXJrIGZvciBMb3ctUmVzb3VyY2UgYW5kIE11bHRpbGluZ3VhbCBNYWNoaW5lIFRyYW5zbGF0aW9uPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDIxPC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+Q29tcHV0YXRpb24gYW5kIExhbmd1YWdlIChjcy5DTCk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5BcnRpZmljaWFsIEludGVsbGlnZW5jZSAoY3MuQUkpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMS0wNi0wNlQxNzo1ODoxMlo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyMS0wNi0wOFQwMDoyNzo1NFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDIxLTA2PC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJQcmVwcmludCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDx2ZXJzaW9uPjE8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2FyeGl2Lm9yZy9saWNlbnNlcy9ub25leGNsdXNpdmUtZGlzdHJpYi8xLjAvIj5hclhpdi5vcmcgcGVycGV0dWFsLCBub24tZXhjbHVzaXZlIGxpY2Vuc2U8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5PbmUgb2YgdGhlIGJpZ2dlc3QgY2hhbGxlbmdlcyBoaW5kZXJpbmcgcHJvZ3Jlc3MgaW4gbG93LXJlc291cmNlIGFuZCBtdWx0aWxpbmd1YWwgbWFjaGluZSB0cmFuc2xhdGlvbiBpcyB0aGUgbGFjayBvZiBnb29kIGV2YWx1YXRpb24gYmVuY2htYXJrcy4gQ3VycmVudCBldmFsdWF0aW9uIGJlbmNobWFya3MgZWl0aGVyIGxhY2sgZ29vZCBjb3ZlcmFnZSBvZiBsb3ctcmVzb3VyY2UgbGFuZ3VhZ2VzLCBjb25zaWRlciBvbmx5IHJlc3RyaWN0ZWQgZG9tYWlucywgb3IgYXJlIGxvdyBxdWFsaXR5IGJlY2F1c2UgdGhleSBhcmUgY29uc3RydWN0ZWQgdXNpbmcgc2VtaS1hdXRvbWF0aWMgcHJvY2VkdXJlcy4gSW4gdGhpcyB3b3JrLCB3ZSBpbnRyb2R1Y2UgdGhlIEZMT1JFUy0xMDEgZXZhbHVhdGlvbiBiZW5jaG1hcmssIGNvbnNpc3Rpbmcgb2YgMzAwMSBzZW50ZW5jZXMgZXh0cmFjdGVkIGZyb20gRW5nbGlzaCBXaWtpcGVkaWEgYW5kIGNvdmVyaW5nIGEgdmFyaWV0eSBvZiBkaWZmZXJlbnQgdG9waWNzIGFuZCBkb21haW5zLiBUaGVzZSBzZW50ZW5jZXMgaGF2ZSBiZWVuIHRyYW5zbGF0ZWQgaW4gMTAxIGxhbmd1YWdlcyBieSBwcm9mZXNzaW9uYWwgdHJhbnNsYXRvcnMgdGhyb3VnaCBhIGNhcmVmdWxseSBjb250cm9sbGVkIHByb2Nlc3MuIFRoZSByZXN1bHRpbmcgZGF0YXNldCBlbmFibGVzIGJldHRlciBhc3Nlc3NtZW50IG9mIG1vZGVsIHF1YWxpdHkgb24gdGhlIGxvbmcgdGFpbCBvZiBsb3ctcmVzb3VyY2UgbGFuZ3VhZ2VzLCBpbmNsdWRpbmcgdGhlIGV2YWx1YXRpb24gb2YgbWFueS10by1tYW55IG11bHRpbGluZ3VhbCB0cmFuc2xhdGlvbiBzeXN0ZW1zLCBhcyBhbGwgdHJhbnNsYXRpb25zIGFyZSBtdWx0aWxpbmd1YWxseSBhbGlnbmVkLiBCeSBwdWJsaWNseSByZWxlYXNpbmcgc3VjaCBhIGhpZ2gtcXVhbGl0eSBhbmQgaGlnaC1jb3ZlcmFnZSBkYXRhc2V0LCB3ZSBob3BlIHRvIGZvc3RlciBwcm9ncmVzcyBpbiB0aGUgbWFjaGluZSB0cmFuc2xhdGlvbiBjb21tdW5pdHkgYW5kIGJleW9uZC48L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/2106.03193","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-02-21T05:34:11.000Z","registered":"2022-02-21T05:34:13.000Z","published":"2021","updated":"2022-02-21T05:34:13.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2106.03193","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}