{"data":{"id":"10.48550/arxiv.1607.05427","type":"dois","attributes":{"doi":"10.48550/arxiv.1607.05427","prefix":"10.48550","suffix":"arxiv.1607.05427","identifiers":[{"identifier":"1607.05427","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1607.05427"}],"creators":[{"name":"Ding, Changxing","nameType":"Personal","givenName":"Changxing","familyName":"Ding","affiliation":[],"nameIdentifiers":[]},{"name":"Tao, Dacheng","nameType":"Personal","givenName":"Dacheng","familyName":"Tao","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Trunk-Branch Ensemble Convolutional Neural Networks for Video-based Face Recognition"}],"publisher":"arXiv","container":{},"publicationYear":2016,"subjects":[{"lang":"en","subject":"Computer Vision and Pattern Recognition (cs.CV)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2016-07-19T07:14:28Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2016-07-20T00:06:23Z","dateType":"Updated","dateInformation":"v1"},{"date":"2017-05-17T09:12:19Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2017-05-18T00:05:51Z","dateType":"Updated","dateInformation":"v2"},{"date":"2016-07","dateType":"Available","dateInformation":"v1"},{"date":"2016","dateType":"Issued"}],"language":null,"types":{"ris":"RPRT","bibtex":"article","citeproc":"article-journal","schemaOrg":"ScholarlyArticle","resourceType":"Article","resourceTypeGeneral":"Text"},"relatedIdentifiers":[{"relationType":"IsVersionOf","relatedIdentifier":"10.1109/tpami.2017.2700390","relatedIdentifierType":"DOI"}],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"Human faces in surveillance videos often suffer from severe image blur, dramatic pose variations, and occlusion. In this paper, we propose a comprehensive framework based on Convolutional Neural Networks (CNN) to overcome challenges in video-based face recognition (VFR). First, to learn blur-robust face representations, we artificially blur training data composed of clear still images to account for a shortfall in real-world video training data. Using training data composed of both still images and artificially blurred data, CNN is encouraged to learn blur-insensitive features automatically. Second, to enhance robustness of CNN features to pose variations and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which extracts complementary information from holistic face images and patches cropped around facial components. TBE-CNN is an end-to-end model that extracts features efficiently by sharing the low- and middle-level convolutional layers between the trunk and branch networks. Third, to further promote the discriminative power of the representations learnt by TBE-CNN, we propose an improved triplet loss function. Systematic experiments justify the effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves state-of-the-art performance on three popular video face databases: PaSC, COX Face, and YouTube Faces. With the proposed techniques, we also obtain the first place in the BTAS 2016 Video Person Recognition Evaluation.","descriptionType":"Abstract"},{"description":"Accepted Version to IEEE T-PAMI","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xNjA3LjA1NDI3PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTYwNy4wNTQyNzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5EaW5nLCBDaGFuZ3hpbmc8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkNoYW5neGluZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5EaW5nPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlRhbywgRGFjaGVuZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+RGFjaGVuZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5UYW88L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgPC9jcmVhdG9ycz4KICA8dGl0bGVzPgogICAgPHRpdGxlPlRydW5rLUJyYW5jaCBFbnNlbWJsZSBDb252b2x1dGlvbmFsIE5ldXJhbCBOZXR3b3JrcyBmb3IgVmlkZW8tYmFzZWQgRmFjZSBSZWNvZ25pdGlvbjwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAxNjwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPkNvbXB1dGVyIFZpc2lvbiBhbmQgUGF0dGVybiBSZWNvZ25pdGlvbiAoY3MuQ1YpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNi0wNy0xOVQwNzoxNDoyOFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNi0wNy0yMFQwMDowNjoyM1o8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE3LTA1LTE3VDA5OjEyOjE5WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE3LTA1LTE4VDAwOjA1OjUxWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTYtMDc8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlRleHQiPkFydGljbGU8L3Jlc291cmNlVHlwZT4KICA8cmVsYXRlZElkZW50aWZpZXJzPgogICAgPHJlbGF0ZWRJZGVudGlmaWVyIHJlbGF0ZWRJZGVudGlmaWVyVHlwZT0iRE9JIiByZWxhdGlvblR5cGU9IklzVmVyc2lvbk9mIj4xMC4xMTA5L1RQQU1JLjIwMTcuMjcwMDM5MDwvcmVsYXRlZElkZW50aWZpZXI+CiAgPC9yZWxhdGVkSWRlbnRpZmllcnM+CiAgPHZlcnNpb24+MjwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vYXJ4aXYub3JnL2xpY2Vuc2VzL25vbmV4Y2x1c2l2ZS1kaXN0cmliLzEuMC8iPmFyWGl2Lm9yZyBwZXJwZXR1YWwsIG5vbi1leGNsdXNpdmUgbGljZW5zZTwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPkh1bWFuIGZhY2VzIGluIHN1cnZlaWxsYW5jZSB2aWRlb3Mgb2Z0ZW4gc3VmZmVyIGZyb20gc2V2ZXJlIGltYWdlIGJsdXIsIGRyYW1hdGljIHBvc2UgdmFyaWF0aW9ucywgYW5kIG9jY2x1c2lvbi4gSW4gdGhpcyBwYXBlciwgd2UgcHJvcG9zZSBhIGNvbXByZWhlbnNpdmUgZnJhbWV3b3JrIGJhc2VkIG9uIENvbnZvbHV0aW9uYWwgTmV1cmFsIE5ldHdvcmtzIChDTk4pIHRvIG92ZXJjb21lIGNoYWxsZW5nZXMgaW4gdmlkZW8tYmFzZWQgZmFjZSByZWNvZ25pdGlvbiAoVkZSKS4gRmlyc3QsIHRvIGxlYXJuIGJsdXItcm9idXN0IGZhY2UgcmVwcmVzZW50YXRpb25zLCB3ZSBhcnRpZmljaWFsbHkgYmx1ciB0cmFpbmluZyBkYXRhIGNvbXBvc2VkIG9mIGNsZWFyIHN0aWxsIGltYWdlcyB0byBhY2NvdW50IGZvciBhIHNob3J0ZmFsbCBpbiByZWFsLXdvcmxkIHZpZGVvIHRyYWluaW5nIGRhdGEuIFVzaW5nIHRyYWluaW5nIGRhdGEgY29tcG9zZWQgb2YgYm90aCBzdGlsbCBpbWFnZXMgYW5kIGFydGlmaWNpYWxseSBibHVycmVkIGRhdGEsIENOTiBpcyBlbmNvdXJhZ2VkIHRvIGxlYXJuIGJsdXItaW5zZW5zaXRpdmUgZmVhdHVyZXMgYXV0b21hdGljYWxseS4gU2Vjb25kLCB0byBlbmhhbmNlIHJvYnVzdG5lc3Mgb2YgQ05OIGZlYXR1cmVzIHRvIHBvc2UgdmFyaWF0aW9ucyBhbmQgb2NjbHVzaW9uLCB3ZSBwcm9wb3NlIGEgVHJ1bmstQnJhbmNoIEVuc2VtYmxlIENOTiBtb2RlbCAoVEJFLUNOTiksIHdoaWNoIGV4dHJhY3RzIGNvbXBsZW1lbnRhcnkgaW5mb3JtYXRpb24gZnJvbSBob2xpc3RpYyBmYWNlIGltYWdlcyBhbmQgcGF0Y2hlcyBjcm9wcGVkIGFyb3VuZCBmYWNpYWwgY29tcG9uZW50cy4gVEJFLUNOTiBpcyBhbiBlbmQtdG8tZW5kIG1vZGVsIHRoYXQgZXh0cmFjdHMgZmVhdHVyZXMgZWZmaWNpZW50bHkgYnkgc2hhcmluZyB0aGUgbG93LSBhbmQgbWlkZGxlLWxldmVsIGNvbnZvbHV0aW9uYWwgbGF5ZXJzIGJldHdlZW4gdGhlIHRydW5rIGFuZCBicmFuY2ggbmV0d29ya3MuIFRoaXJkLCB0byBmdXJ0aGVyIHByb21vdGUgdGhlIGRpc2NyaW1pbmF0aXZlIHBvd2VyIG9mIHRoZSByZXByZXNlbnRhdGlvbnMgbGVhcm50IGJ5IFRCRS1DTk4sIHdlIHByb3Bvc2UgYW4gaW1wcm92ZWQgdHJpcGxldCBsb3NzIGZ1bmN0aW9uLiBTeXN0ZW1hdGljIGV4cGVyaW1lbnRzIGp1c3RpZnkgdGhlIGVmZmVjdGl2ZW5lc3Mgb2YgdGhlIHByb3Bvc2VkIHRlY2huaXF1ZXMuIE1vc3QgaW1wcmVzc2l2ZWx5LCBUQkUtQ05OIGFjaGlldmVzIHN0YXRlLW9mLXRoZS1hcnQgcGVyZm9ybWFuY2Ugb24gdGhyZWUgcG9wdWxhciB2aWRlbyBmYWNlIGRhdGFiYXNlczogUGFTQywgQ09YIEZhY2UsIGFuZCBZb3VUdWJlIEZhY2VzLiBXaXRoIHRoZSBwcm9wb3NlZCB0ZWNobmlxdWVzLCB3ZSBhbHNvIG9idGFpbiB0aGUgZmlyc3QgcGxhY2UgaW4gdGhlIEJUQVMgMjAxNiBWaWRlbyBQZXJzb24gUmVjb2duaXRpb24gRXZhbHVhdGlvbi48L2Rlc2NyaXB0aW9uPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iT3RoZXIiPkFjY2VwdGVkIFZlcnNpb24gdG8gSUVFRSBULVBBTUk8L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/1607.05427","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-06T01:20:28.000Z","registered":"2022-03-06T01:20:29.000Z","published":"2016","updated":"2022-03-06T01:20:29.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1607.05427","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}