{"data":{"id":"10.48550/arxiv.1510.08896","type":"dois","attributes":{"doi":"10.48550/arxiv.1510.08896","prefix":"10.48550","suffix":"arxiv.1510.08896","identifiers":[{"identifier":"1510.08896","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1510.08896"}],"creators":[{"name":"Jin, Chi","nameType":"Personal","givenName":"Chi","familyName":"Jin","affiliation":[],"nameIdentifiers":[]},{"name":"Kakade, Sham M.","nameType":"Personal","givenName":"Sham M.","familyName":"Kakade","affiliation":[],"nameIdentifiers":[]},{"name":"Musco, Cameron","nameType":"Personal","givenName":"Cameron","familyName":"Musco","affiliation":[],"nameIdentifiers":[]},{"name":"Netrapalli, Praneeth","nameType":"Personal","givenName":"Praneeth","familyName":"Netrapalli","affiliation":[],"nameIdentifiers":[]},{"name":"Sidford, Aaron","nameType":"Personal","givenName":"Aaron","familyName":"Sidford","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Robust Shift-and-Invert Preconditioning: Faster and More Sample Efficient Algorithms for Eigenvector Computation"}],"publisher":"arXiv","container":{},"publicationYear":2015,"subjects":[{"lang":"en","subject":"Data Structures and Algorithms (cs.DS)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"lang":"en","subject":"Numerical Analysis (math.NA)","subjectScheme":"arXiv"},{"lang":"en","subject":"Optimization and Control (math.OC)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Mathematics","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Mathematics","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2015-10-29T20:47:27Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2015-11-02T01:01:46Z","dateType":"Updated","dateInformation":"v1"},{"date":"2016-05-30T02:40:00Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2016-05-31T00:12:11Z","dateType":"Updated","dateInformation":"v2"},{"date":"2015-10","dateType":"Available","dateInformation":"v1"},{"date":"2015","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"We provide faster algorithms and improved sample complexities for approximating the top eigenvector of a matrix. Offline Setting: Given an $n \\times d$ matrix $A$, we show how to compute an $ε$ approximate top eigenvector in time $\\tilde O ( [nnz(A) + \\frac{d \\cdot sr(A)}{gap^2}]\\cdot \\log 1/ε)$ and $\\tilde O([\\frac{nnz(A)^{3/4} (d \\cdot sr(A))^{1/4}}{\\sqrt{gap}}]\\cdot \\log1/ε)$. Here $sr(A)$ is the stable rank and $gap$ is the multiplicative eigenvalue gap. By separating the $gap$ dependence from $nnz(A)$ we improve on the classic power and Lanczos methods. We also improve prior work using fast subspace embeddings and stochastic optimization, giving significantly improved dependencies on $sr(A)$ and $ε$. Our second running time improves this further when $nnz(A) \\le \\frac{d\\cdot sr(A)}{gap^2}$. Online Setting: Given a distribution $D$ with covariance matrix $Σ$ and a vector $x_0$ which is an $O(gap)$ approximate top eigenvector for $Σ$, we show how to refine to an $ε$ approximation using $\\tilde O(\\frac{v(D)}{gap^2} + \\frac{v(D)}{gap \\cdot ε})$ samples from $D$. Here $v(D)$ is a natural variance measure. Combining our algorithm with previous work to initialize $x_0$, we obtain a number of improved sample complexity and runtime results. For general distributions, we achieve asymptotically optimal accuracy as a function of sample size as the number of samples grows large. Our results center around a robust analysis of the classic method of shift-and-invert preconditioning to reduce eigenvector computation to approximately solving a sequence of linear systems. We then apply fast SVRG based approximate system solvers to achieve our claims. We believe our results suggest the general effectiveness of shift-and-invert based approaches and imply that further computational gains may be reaped in practice.","descriptionType":"Abstract"},{"description":"Manuscript outdated. Updated version at arxiv:1605.08754","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xNTEwLjA4ODk2PC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTUxMC4wODg5NjwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5KaW4sIENoaTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+Q2hpPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPkppbjwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5LYWthZGUsIFNoYW0gTS48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlNoYW0gTS48L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+S2FrYWRlPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPk11c2NvLCBDYW1lcm9uPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5DYW1lcm9uPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPk11c2NvPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPk5ldHJhcGFsbGksIFByYW5lZXRoPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5QcmFuZWV0aDwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5OZXRyYXBhbGxpPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPlNpZGZvcmQsIEFhcm9uPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5BYXJvbjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5TaWRmb3JkPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5Sb2J1c3QgU2hpZnQtYW5kLUludmVydCBQcmVjb25kaXRpb25pbmc6IEZhc3RlciBhbmQgTW9yZSBTYW1wbGUgRWZmaWNpZW50IEFsZ29yaXRobXMgZm9yIEVpZ2VudmVjdG9yIENvbXB1dGF0aW9uPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDE1PC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+RGF0YSBTdHJ1Y3R1cmVzIGFuZCBBbGdvcml0aG1zIChjcy5EUyk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5NYWNoaW5lIExlYXJuaW5nIChjcy5MRyk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5OdW1lcmljYWwgQW5hbHlzaXMgKG1hdGguTkEpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+T3B0aW1pemF0aW9uIGFuZCBDb250cm9sIChtYXRoLk9DKTwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IENvbXB1dGVyIGFuZCBpbmZvcm1hdGlvbiBzY2llbmNlczwvc3ViamVjdD4KICAgIDxzdWJqZWN0IHN1YmplY3RTY2hlbWU9IkZpZWxkcyBvZiBTY2llbmNlIGFuZCBUZWNobm9sb2d5IChGT1MpIj5GT1M6IE1hdGhlbWF0aWNzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNS0xMC0yOVQyMDo0NzoyN1o8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxNS0xMS0wMlQwMTowMTo0Nlo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE2LTA1LTMwVDAyOjQwOjAwWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYyIj4yMDE2LTA1LTMxVDAwOjEyOjExWjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJBdmFpbGFibGUiIGRhdGVJbmZvcm1hdGlvbj0idjEiPjIwMTUtMTA8L2RhdGU+CiAgPC9kYXRlcz4KICA8cmVzb3VyY2VUeXBlIHJlc291cmNlVHlwZUdlbmVyYWw9IlByZXByaW50Ij5BcnRpY2xlPC9yZXNvdXJjZVR5cGU+CiAgPHZlcnNpb24+MjwvdmVyc2lvbj4KICA8cmlnaHRzTGlzdD4KICAgIDxyaWdodHMgcmlnaHRzVVJJPSJodHRwOi8vYXJ4aXYub3JnL2xpY2Vuc2VzL25vbmV4Y2x1c2l2ZS1kaXN0cmliLzEuMC8iPmFyWGl2Lm9yZyBwZXJwZXR1YWwsIG5vbi1leGNsdXNpdmUgbGljZW5zZTwvcmlnaHRzPgogIDwvcmlnaHRzTGlzdD4KICA8ZGVzY3JpcHRpb25zPgogICAgPGRlc2NyaXB0aW9uIGRlc2NyaXB0aW9uVHlwZT0iQWJzdHJhY3QiPldlIHByb3ZpZGUgZmFzdGVyIGFsZ29yaXRobXMgYW5kIGltcHJvdmVkIHNhbXBsZSBjb21wbGV4aXRpZXMgZm9yIGFwcHJveGltYXRpbmcgdGhlIHRvcCBlaWdlbnZlY3RvciBvZiBhIG1hdHJpeC4KICBPZmZsaW5lIFNldHRpbmc6IEdpdmVuIGFuICRuIFx0aW1lcyBkJCBtYXRyaXggJEEkLCB3ZSBzaG93IGhvdyB0byBjb21wdXRlIGFuICTOtSQgYXBwcm94aW1hdGUgdG9wIGVpZ2VudmVjdG9yIGluIHRpbWUgJFx0aWxkZSBPICggW25ueihBKSArIFxmcmFje2QgXGNkb3Qgc3IoQSl9e2dhcF4yfV1cY2RvdCBcbG9nIDEvzrUpJCBhbmQgJFx0aWxkZSBPKFtcZnJhY3tubnooQSleezMvNH0gKGQgXGNkb3Qgc3IoQSkpXnsxLzR9fXtcc3FydHtnYXB9fV1cY2RvdCBcbG9nMS/OtSkkLiBIZXJlICRzcihBKSQgaXMgdGhlIHN0YWJsZSByYW5rIGFuZCAkZ2FwJCBpcyB0aGUgbXVsdGlwbGljYXRpdmUgZWlnZW52YWx1ZSBnYXAuIEJ5IHNlcGFyYXRpbmcgdGhlICRnYXAkIGRlcGVuZGVuY2UgZnJvbSAkbm56KEEpJCB3ZSBpbXByb3ZlIG9uIHRoZSBjbGFzc2ljIHBvd2VyIGFuZCBMYW5jem9zIG1ldGhvZHMuIFdlIGFsc28gaW1wcm92ZSBwcmlvciB3b3JrIHVzaW5nIGZhc3Qgc3Vic3BhY2UgZW1iZWRkaW5ncyBhbmQgc3RvY2hhc3RpYyBvcHRpbWl6YXRpb24sIGdpdmluZyBzaWduaWZpY2FudGx5IGltcHJvdmVkIGRlcGVuZGVuY2llcyBvbiAkc3IoQSkkIGFuZCAkzrUkLiBPdXIgc2Vjb25kIHJ1bm5pbmcgdGltZSBpbXByb3ZlcyB0aGlzIGZ1cnRoZXIgd2hlbiAkbm56KEEpIFxsZSBcZnJhY3tkXGNkb3Qgc3IoQSl9e2dhcF4yfSQuCiAgT25saW5lIFNldHRpbmc6IEdpdmVuIGEgZGlzdHJpYnV0aW9uICREJCB3aXRoIGNvdmFyaWFuY2UgbWF0cml4ICTOoyQgYW5kIGEgdmVjdG9yICR4XzAkIHdoaWNoIGlzIGFuICRPKGdhcCkkIGFwcHJveGltYXRlIHRvcCBlaWdlbnZlY3RvciBmb3IgJM6jJCwgd2Ugc2hvdyBob3cgdG8gcmVmaW5lIHRvIGFuICTOtSQgYXBwcm94aW1hdGlvbiB1c2luZyAkXHRpbGRlIE8oXGZyYWN7dihEKX17Z2FwXjJ9ICsgXGZyYWN7dihEKX17Z2FwIFxjZG90IM61fSkkIHNhbXBsZXMgZnJvbSAkRCQuIEhlcmUgJHYoRCkkIGlzIGEgbmF0dXJhbCB2YXJpYW5jZSBtZWFzdXJlLiBDb21iaW5pbmcgb3VyIGFsZ29yaXRobSB3aXRoIHByZXZpb3VzIHdvcmsgdG8gaW5pdGlhbGl6ZSAkeF8wJCwgd2Ugb2J0YWluIGEgbnVtYmVyIG9mIGltcHJvdmVkIHNhbXBsZSBjb21wbGV4aXR5IGFuZCBydW50aW1lIHJlc3VsdHMuIEZvciBnZW5lcmFsIGRpc3RyaWJ1dGlvbnMsIHdlIGFjaGlldmUgYXN5bXB0b3RpY2FsbHkgb3B0aW1hbCBhY2N1cmFjeSBhcyBhIGZ1bmN0aW9uIG9mIHNhbXBsZSBzaXplIGFzIHRoZSBudW1iZXIgb2Ygc2FtcGxlcyBncm93cyBsYXJnZS4KICBPdXIgcmVzdWx0cyBjZW50ZXIgYXJvdW5kIGEgcm9idXN0IGFuYWx5c2lzIG9mIHRoZSBjbGFzc2ljIG1ldGhvZCBvZiBzaGlmdC1hbmQtaW52ZXJ0IHByZWNvbmRpdGlvbmluZyB0byByZWR1Y2UgZWlnZW52ZWN0b3IgY29tcHV0YXRpb24gdG8gYXBwcm94aW1hdGVseSBzb2x2aW5nIGEgc2VxdWVuY2Ugb2YgbGluZWFyIHN5c3RlbXMuIFdlIHRoZW4gYXBwbHkgZmFzdCBTVlJHIGJhc2VkIGFwcHJveGltYXRlIHN5c3RlbSBzb2x2ZXJzIHRvIGFjaGlldmUgb3VyIGNsYWltcy4gV2UgYmVsaWV2ZSBvdXIgcmVzdWx0cyBzdWdnZXN0IHRoZSBnZW5lcmFsIGVmZmVjdGl2ZW5lc3Mgb2Ygc2hpZnQtYW5kLWludmVydCBiYXNlZCBhcHByb2FjaGVzIGFuZCBpbXBseSB0aGF0IGZ1cnRoZXIgY29tcHV0YXRpb25hbCBnYWlucyBtYXkgYmUgcmVhcGVkIGluIHByYWN0aWNlLjwvZGVzY3JpcHRpb24+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJPdGhlciI+TWFudXNjcmlwdCBvdXRkYXRlZC4gVXBkYXRlZCB2ZXJzaW9uIGF0IGFyeGl2OjE2MDUuMDg3NTQ8L2Rlc2NyaXB0aW9uPgogIDwvZGVzY3JpcHRpb25zPgo8L3Jlc291cmNlPg==","url":"https://arxiv.org/abs/1510.08896","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-07T20:40:40.000Z","registered":"2022-03-07T20:40:42.000Z","published":"2015","updated":"2022-03-07T20:40:42.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1510.08896","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}