{"data":{"id":"10.48550/arxiv.1712.01038","type":"dois","attributes":{"doi":"10.48550/arxiv.1712.01038","prefix":"10.48550","suffix":"arxiv.1712.01038","identifiers":[{"identifier":"1712.01038","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1712.01038"}],"creators":[{"name":"Khan, Mohammad Emtiyaz","nameType":"Personal","givenName":"Mohammad Emtiyaz","familyName":"Khan","affiliation":[],"nameIdentifiers":[]},{"name":"Liu, Zuozhu","nameType":"Personal","givenName":"Zuozhu","familyName":"Liu","affiliation":[],"nameIdentifiers":[]},{"name":"Tangkaratt, Voot","nameType":"Personal","givenName":"Voot","familyName":"Tangkaratt","affiliation":[],"nameIdentifiers":[]},{"name":"Gal, Yarin","nameType":"Personal","givenName":"Yarin","familyName":"Gal","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Vprop: Variational Inference using RMSprop"}],"publisher":"arXiv","container":{},"publicationYear":2017,"subjects":[{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2017-12-04T12:31:36Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2017-12-05T01:16:47Z","dateType":"Updated","dateInformation":"v1"},{"date":"2017-12","dateType":"Available","dateInformation":"v1"},{"date":"2017","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"Many computationally-efficient methods for Bayesian deep learning rely on continuous optimization algorithms, but the implementation of these methods requires significant changes to existing code-bases. In this paper, we propose Vprop, a method for Gaussian variational inference that can be implemented with two minor changes to the off-the-shelf RMSprop optimizer. Vprop also reduces the memory requirements of Black-Box Variational Inference by half. We derive Vprop using the conjugate-computation variational inference method, and establish its connections to Newton's method, natural-gradient methods, and extended Kalman filters. Overall, this paper presents Vprop as a principled, computationally-efficient, and easy-to-implement method for Bayesian deep learning.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xNzEyLjAxMDM4PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTcxMi4wMTAzODwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5LaGFuLCBNb2hhbW1hZCBFbXRpeWF6PC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5Nb2hhbW1hZCBFbXRpeWF6PC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPktoYW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+TGl1LCBadW96aHU8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlp1b3podTwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5MaXU8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+VGFuZ2thcmF0dCwgVm9vdDwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+Vm9vdDwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5UYW5na2FyYXR0PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkdhbCwgWWFyaW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPllhcmluPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkdhbDwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICA8L2NyZWF0b3JzPgogIDx0aXRsZXM+CiAgICA8dGl0bGU+VnByb3A6IFZhcmlhdGlvbmFsIEluZmVyZW5jZSB1c2luZyBSTVNwcm9wPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDE3PC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoc3RhdC5NTCk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5NYWNoaW5lIExlYXJuaW5nIChjcy5MRyk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBDb21wdXRlciBhbmQgaW5mb3JtYXRpb24gc2NpZW5jZXM8L3N1YmplY3Q+CiAgPC9zdWJqZWN0cz4KICA8ZGF0ZXM+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDE3LTEyLTA0VDEyOjMxOjM2WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDE3LTEyLTA1VDAxOjE2OjQ3WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTctMTI8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MTwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vYXJ4aXYub3JnL2xpY2Vuc2VzL25vbmV4Y2x1c2l2ZS1kaXN0cmliLzEuMC8iPmFyWGl2Lm9yZyBwZXJwZXR1YWwsIG5vbi1leGNsdXNpdmUgbGljZW5zZTwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPk1hbnkgY29tcHV0YXRpb25hbGx5LWVmZmljaWVudCBtZXRob2RzIGZvciBCYXllc2lhbiBkZWVwIGxlYXJuaW5nIHJlbHkgb24gY29udGludW91cyBvcHRpbWl6YXRpb24gYWxnb3JpdGhtcywgYnV0IHRoZSBpbXBsZW1lbnRhdGlvbiBvZiB0aGVzZSBtZXRob2RzIHJlcXVpcmVzIHNpZ25pZmljYW50IGNoYW5nZXMgdG8gZXhpc3RpbmcgY29kZS1iYXNlcy4gSW4gdGhpcyBwYXBlciwgd2UgcHJvcG9zZSBWcHJvcCwgYSBtZXRob2QgZm9yIEdhdXNzaWFuIHZhcmlhdGlvbmFsIGluZmVyZW5jZSB0aGF0IGNhbiBiZSBpbXBsZW1lbnRlZCB3aXRoIHR3byBtaW5vciBjaGFuZ2VzIHRvIHRoZSBvZmYtdGhlLXNoZWxmIFJNU3Byb3Agb3B0aW1pemVyLiBWcHJvcCBhbHNvIHJlZHVjZXMgdGhlIG1lbW9yeSByZXF1aXJlbWVudHMgb2YgQmxhY2stQm94IFZhcmlhdGlvbmFsIEluZmVyZW5jZSBieSBoYWxmLiBXZSBkZXJpdmUgVnByb3AgdXNpbmcgdGhlIGNvbmp1Z2F0ZS1jb21wdXRhdGlvbiB2YXJpYXRpb25hbCBpbmZlcmVuY2UgbWV0aG9kLCBhbmQgZXN0YWJsaXNoIGl0cyBjb25uZWN0aW9ucyB0byBOZXd0b24ncyBtZXRob2QsIG5hdHVyYWwtZ3JhZGllbnQgbWV0aG9kcywgYW5kIGV4dGVuZGVkIEthbG1hbiBmaWx0ZXJzLiBPdmVyYWxsLCB0aGlzIHBhcGVyIHByZXNlbnRzIFZwcm9wIGFzIGEgcHJpbmNpcGxlZCwgY29tcHV0YXRpb25hbGx5LWVmZmljaWVudCwgYW5kIGVhc3ktdG8taW1wbGVtZW50IG1ldGhvZCBmb3IgQmF5ZXNpYW4gZGVlcCBsZWFybmluZy48L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/1712.01038","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-03T02:17:18.000Z","registered":"2022-03-03T02:17:19.000Z","published":"2017","updated":"2022-03-03T02:17:19.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1712.01038","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}