{"data":{"id":"10.48550/arxiv.1408.0361","type":"dois","attributes":{"doi":"10.48550/arxiv.1408.0361","prefix":"10.48550","suffix":"arxiv.1408.0361","identifiers":[{"identifier":"1408.0361","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1408.0361"}],"creators":[{"name":"Dieuleveut, Aymeric","nameType":"Personal","givenName":"Aymeric","familyName":"Dieuleveut","affiliation":[],"nameIdentifiers":[]},{"name":"Bach, Francis","nameType":"Personal","givenName":"Francis","familyName":"Bach","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Non-parametric Stochastic Approximation with Large Step sizes"}],"publisher":"arXiv","container":{},"publicationYear":2014,"subjects":[{"lang":"en","subject":"Statistics Theory (math.ST)","subjectScheme":"arXiv"},{"subject":"FOS: Mathematics","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Mathematics","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2014-08-02T11:22:52Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2014-08-05T00:04:40Z","dateType":"Updated","dateInformation":"v1"},{"date":"2015-07-24T07:05:28Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2015-07-27T00:05:42Z","dateType":"Updated","dateInformation":"v2"},{"date":"2016-03-29T00:03:47Z","dateType":"Submitted","dateInformation":"v3"},{"date":"2016-03-30T00:02:40Z","dateType":"Updated","dateInformation":"v3"},{"date":"2014-08","dateType":"Available","dateInformation":"v1"},{"date":"2014","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"3","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"We consider the random-design least-squares regression problem within the reproducing kernel Hilbert space (RKHS) framework. Given a stream of independent and identically distributed input/output data, we aim to learn a regression function within an RKHS $\\mathcal{H}$, even if the optimal predictor (i.e., the conditional expectation) is not in $\\mathcal{H}$. In a stochastic approximation framework where the estimator is updated after each observation, we show that the averaged unregularized least-mean-square algorithm (a form of stochastic gradient), given a sufficient large step-size, attains optimal rates of convergence for a variety of regimes for the smoothnesses of the optimal prediction function and the functions in $\\mathcal{H}$.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xNDA4LjAzNjE8L2lkZW50aWZpZXI+CiAgPGFsdGVybmF0ZUlkZW50aWZpZXJzPgogICAgPGFsdGVybmF0ZUlkZW50aWZpZXIgYWx0ZXJuYXRlSWRlbnRpZmllclR5cGU9ImFyWGl2Ij4xNDA4LjAzNjE8L2FsdGVybmF0ZUlkZW50aWZpZXI+CiAgPC9hbHRlcm5hdGVJZGVudGlmaWVycz4KICA8Y3JlYXRvcnM+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+RGlldWxldmV1dCwgQXltZXJpYzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+QXltZXJpYzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5EaWV1bGV2ZXV0PC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkJhY2gsIEZyYW5jaXM8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkZyYW5jaXM8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+QmFjaDwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICA8L2NyZWF0b3JzPgogIDx0aXRsZXM+CiAgICA8dGl0bGU+Tm9uLXBhcmFtZXRyaWMgU3RvY2hhc3RpYyBBcHByb3hpbWF0aW9uIHdpdGggTGFyZ2UgU3RlcCBzaXplczwvdGl0bGU+CiAgPC90aXRsZXM+CiAgPHB1Ymxpc2hlcj5hclhpdjwvcHVibGlzaGVyPgogIDxwdWJsaWNhdGlvblllYXI+MjAxNDwvcHVibGljYXRpb25ZZWFyPgogIDxzdWJqZWN0cz4KICAgIDxzdWJqZWN0IHhtbDpsYW5nPSJlbiIgc3ViamVjdFNjaGVtZT0iYXJYaXYiPlN0YXRpc3RpY3MgVGhlb3J5IChtYXRoLlNUKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IE1hdGhlbWF0aWNzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNC0wOC0wMlQxMToyMjo1Mlo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNC0wOC0wNVQwMDowNDo0MFo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE1LTA3LTI0VDA3OjA1OjI4WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE1LTA3LTI3VDAwOjA1OjQyWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjMiPjIwMTYtMDMtMjlUMDA6MDM6NDdaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjMiPjIwMTYtMDMtMzBUMDA6MDI6NDBaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IkF2YWlsYWJsZSIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNC0wODwvZGF0ZT4KICA8L2RhdGVzPgogIDxyZXNvdXJjZVR5cGUgcmVzb3VyY2VUeXBlR2VuZXJhbD0iUHJlcHJpbnQiPkFydGljbGU8L3Jlc291cmNlVHlwZT4KICA8dmVyc2lvbj4zPC92ZXJzaW9uPgogIDxyaWdodHNMaXN0PgogICAgPHJpZ2h0cyByaWdodHNVUkk9Imh0dHA6Ly9hcnhpdi5vcmcvbGljZW5zZXMvbm9uZXhjbHVzaXZlLWRpc3RyaWIvMS4wLyI+YXJYaXYub3JnIHBlcnBldHVhbCwgbm9uLWV4Y2x1c2l2ZSBsaWNlbnNlPC9yaWdodHM+CiAgPC9yaWdodHNMaXN0PgogIDxkZXNjcmlwdGlvbnM+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJBYnN0cmFjdCI+V2UgY29uc2lkZXIgdGhlIHJhbmRvbS1kZXNpZ24gbGVhc3Qtc3F1YXJlcyByZWdyZXNzaW9uIHByb2JsZW0gd2l0aGluIHRoZSByZXByb2R1Y2luZyBrZXJuZWwgSGlsYmVydCBzcGFjZSAoUktIUykgZnJhbWV3b3JrLiBHaXZlbiBhIHN0cmVhbSBvZiBpbmRlcGVuZGVudCBhbmQgaWRlbnRpY2FsbHkgZGlzdHJpYnV0ZWQgaW5wdXQvb3V0cHV0IGRhdGEsIHdlIGFpbSB0byBsZWFybiBhIHJlZ3Jlc3Npb24gZnVuY3Rpb24gd2l0aGluIGFuIFJLSFMgJFxtYXRoY2Fse0h9JCwgZXZlbiBpZiB0aGUgb3B0aW1hbCBwcmVkaWN0b3IgKGkuZS4sIHRoZSBjb25kaXRpb25hbCBleHBlY3RhdGlvbikgaXMgbm90IGluICRcbWF0aGNhbHtIfSQuIEluIGEgc3RvY2hhc3RpYyBhcHByb3hpbWF0aW9uIGZyYW1ld29yayB3aGVyZSB0aGUgZXN0aW1hdG9yIGlzIHVwZGF0ZWQgYWZ0ZXIgZWFjaCBvYnNlcnZhdGlvbiwgd2Ugc2hvdyB0aGF0IHRoZSBhdmVyYWdlZCB1bnJlZ3VsYXJpemVkIGxlYXN0LW1lYW4tc3F1YXJlIGFsZ29yaXRobSAoYSBmb3JtIG9mIHN0b2NoYXN0aWMgZ3JhZGllbnQpLCBnaXZlbiBhIHN1ZmZpY2llbnQgbGFyZ2Ugc3RlcC1zaXplLCBhdHRhaW5zIG9wdGltYWwgcmF0ZXMgb2YgY29udmVyZ2VuY2UgZm9yIGEgdmFyaWV0eSBvZiByZWdpbWVzIGZvciB0aGUgc21vb3RobmVzc2VzIG9mIHRoZSBvcHRpbWFsIHByZWRpY3Rpb24gZnVuY3Rpb24gYW5kIHRoZSBmdW5jdGlvbnMgaW4gJFxtYXRoY2Fse0h9JC48L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/1408.0361","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-09T21:06:45.000Z","registered":"2022-03-09T21:06:46.000Z","published":"2014","updated":"2025-05-27T21:58:10.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1408.0361","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}