{"data":{"id":"10.48550/arxiv.1810.06793","type":"dois","attributes":{"doi":"10.48550/arxiv.1810.06793","prefix":"10.48550","suffix":"arxiv.1810.06793","identifiers":[{"identifier":"1810.06793","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"1810.06793"}],"creators":[{"name":"Ge, Rong","nameType":"Personal","givenName":"Rong","familyName":"Ge","affiliation":[],"nameIdentifiers":[]},{"name":"Kuditipudi, Rohith","nameType":"Personal","givenName":"Rohith","familyName":"Kuditipudi","affiliation":[],"nameIdentifiers":[]},{"name":"Li, Zhize","nameType":"Personal","givenName":"Zhize","familyName":"Li","affiliation":[],"nameIdentifiers":[]},{"name":"Wang, Xiang","nameType":"Personal","givenName":"Xiang","familyName":"Wang","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"Learning Two-layer Neural Networks with Symmetric Inputs"}],"publisher":"arXiv","container":{},"publicationYear":2018,"subjects":[{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (stat.ML)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"},{"subject":"FOS: Computer and information sciences","schemeUri":"http://www.oecd.org/science/inno/38235147.pdf","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2018-10-16T02:26:55Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2018-10-17T00:06:24Z","dateType":"Updated","dateInformation":"v1"},{"date":"2019-02-03T19:46:44Z","dateType":"Submitted","dateInformation":"v2"},{"date":"2019-02-05T01:20:04Z","dateType":"Updated","dateInformation":"v2"},{"date":"2018-10","dateType":"Available","dateInformation":"v1"},{"date":"2018","dateType":"Issued"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"2","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network $$ y = A σ(Wx) + ξ, $$ where $A,W$ are weight matrices, $ξ$ represents noise, and the number of neurons in the hidden layer is no larger than the input or output, our algorithm is guaranteed to recover the parameters $A,W$ of the ground-truth network. The only requirement on the input $x$ is that it is symmetric, which still allows highly complicated and structured input. Our algorithm is based on the method-of-moments framework and extends several results in tensor decompositions. We use spectral algorithms to avoid the complicated non-convex optimization in learning neural networks. Experiments show that our algorithm can robustly learn the ground-truth neural network with a small number of samples for many symmetric input distributions.","descriptionType":"Abstract"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4xODEwLjA2NzkzPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MTgxMC4wNjc5MzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5HZSwgUm9uZzwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+Um9uZzwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5HZTwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5LdWRpdGlwdWRpLCBSb2hpdGg8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlJvaGl0aDwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5LdWRpdGlwdWRpPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogICAgPGNyZWF0b3I+CiAgICAgIDxjcmVhdG9yTmFtZSBuYW1lVHlwZT0iUGVyc29uYWwiPkxpLCBaaGl6ZTwvY3JlYXRvck5hbWU+CiAgICAgIDxnaXZlbk5hbWU+WmhpemU8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+TGk8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+V2FuZywgWGlhbmc8L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPlhpYW5nPC9naXZlbk5hbWU+CiAgICAgIDxmYW1pbHlOYW1lPldhbmc8L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgPC9jcmVhdG9ycz4KICA8dGl0bGVzPgogICAgPHRpdGxlPkxlYXJuaW5nIFR3by1sYXllciBOZXVyYWwgTmV0d29ya3Mgd2l0aCBTeW1tZXRyaWMgSW5wdXRzPC90aXRsZT4KICA8L3RpdGxlcz4KICA8cHVibGlzaGVyPmFyWGl2PC9wdWJsaXNoZXI+CiAgPHB1YmxpY2F0aW9uWWVhcj4yMDE4PC9wdWJsaWNhdGlvblllYXI+CiAgPHN1YmplY3RzPgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoY3MuTEcpPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoc3RhdC5NTCk8L3N1YmplY3Q+CiAgICA8c3ViamVjdCBzdWJqZWN0U2NoZW1lPSJGaWVsZHMgb2YgU2NpZW5jZSBhbmQgVGVjaG5vbG9neSAoRk9TKSI+Rk9TOiBDb21wdXRlciBhbmQgaW5mb3JtYXRpb24gc2NpZW5jZXM8L3N1YmplY3Q+CiAgPC9zdWJqZWN0cz4KICA8ZGF0ZXM+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iU3VibWl0dGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDE4LTEwLTE2VDAyOjI2OjU1WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJVcGRhdGVkIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDE4LTEwLTE3VDAwOjA2OjI0WjwvZGF0ZT4KICAgIDxkYXRlIGRhdGVUeXBlPSJTdWJtaXR0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjIiPjIwMTktMDItMDNUMTk6NDY6NDRaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IlVwZGF0ZWQiIGRhdGVJbmZvcm1hdGlvbj0idjIiPjIwMTktMDItMDVUMDE6MjA6MDRaPC9kYXRlPgogICAgPGRhdGUgZGF0ZVR5cGU9IkF2YWlsYWJsZSIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAxOC0xMDwvZGF0ZT4KICA8L2RhdGVzPgogIDxyZXNvdXJjZVR5cGUgcmVzb3VyY2VUeXBlR2VuZXJhbD0iUHJlcHJpbnQiPkFydGljbGU8L3Jlc291cmNlVHlwZT4KICA8dmVyc2lvbj4yPC92ZXJzaW9uPgogIDxyaWdodHNMaXN0PgogICAgPHJpZ2h0cyByaWdodHNVUkk9Imh0dHA6Ly9hcnhpdi5vcmcvbGljZW5zZXMvbm9uZXhjbHVzaXZlLWRpc3RyaWIvMS4wLyI+YXJYaXYub3JnIHBlcnBldHVhbCwgbm9uLWV4Y2x1c2l2ZSBsaWNlbnNlPC9yaWdodHM+CiAgPC9yaWdodHNMaXN0PgogIDxkZXNjcmlwdGlvbnM+CiAgICA8ZGVzY3JpcHRpb24gZGVzY3JpcHRpb25UeXBlPSJBYnN0cmFjdCI+V2UgZ2l2ZSBhIG5ldyBhbGdvcml0aG0gZm9yIGxlYXJuaW5nIGEgdHdvLWxheWVyIG5ldXJhbCBuZXR3b3JrIHVuZGVyIGEgZ2VuZXJhbCBjbGFzcyBvZiBpbnB1dCBkaXN0cmlidXRpb25zLiBBc3N1bWluZyB0aGVyZSBpcyBhIGdyb3VuZC10cnV0aCB0d28tbGF5ZXIgbmV0d29yayAkJCB5ID0gQSDPgyhXeCkgKyDOviwgJCQgd2hlcmUgJEEsVyQgYXJlIHdlaWdodCBtYXRyaWNlcywgJM6+JCByZXByZXNlbnRzIG5vaXNlLCBhbmQgdGhlIG51bWJlciBvZiBuZXVyb25zIGluIHRoZSBoaWRkZW4gbGF5ZXIgaXMgbm8gbGFyZ2VyIHRoYW4gdGhlIGlucHV0IG9yIG91dHB1dCwgb3VyIGFsZ29yaXRobSBpcyBndWFyYW50ZWVkIHRvIHJlY292ZXIgdGhlIHBhcmFtZXRlcnMgJEEsVyQgb2YgdGhlIGdyb3VuZC10cnV0aCBuZXR3b3JrLiBUaGUgb25seSByZXF1aXJlbWVudCBvbiB0aGUgaW5wdXQgJHgkIGlzIHRoYXQgaXQgaXMgc3ltbWV0cmljLCB3aGljaCBzdGlsbCBhbGxvd3MgaGlnaGx5IGNvbXBsaWNhdGVkIGFuZCBzdHJ1Y3R1cmVkIGlucHV0LgogIE91ciBhbGdvcml0aG0gaXMgYmFzZWQgb24gdGhlIG1ldGhvZC1vZi1tb21lbnRzIGZyYW1ld29yayBhbmQgZXh0ZW5kcyBzZXZlcmFsIHJlc3VsdHMgaW4gdGVuc29yIGRlY29tcG9zaXRpb25zLiBXZSB1c2Ugc3BlY3RyYWwgYWxnb3JpdGhtcyB0byBhdm9pZCB0aGUgY29tcGxpY2F0ZWQgbm9uLWNvbnZleCBvcHRpbWl6YXRpb24gaW4gbGVhcm5pbmcgbmV1cmFsIG5ldHdvcmtzLiBFeHBlcmltZW50cyBzaG93IHRoYXQgb3VyIGFsZ29yaXRobSBjYW4gcm9idXN0bHkgbGVhcm4gdGhlIGdyb3VuZC10cnV0aCBuZXVyYWwgbmV0d29yayB3aXRoIGEgc21hbGwgbnVtYmVyIG9mIHNhbXBsZXMgZm9yIG1hbnkgc3ltbWV0cmljIGlucHV0IGRpc3RyaWJ1dGlvbnMuPC9kZXNjcmlwdGlvbj4KICA8L2Rlc2NyaXB0aW9ucz4KPC9yZXNvdXJjZT4=","url":"https://arxiv.org/abs/1810.06793","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2022-03-01T14:52:24.000Z","registered":"2022-03-01T14:52:27.000Z","published":"2018","updated":"2022-03-01T14:52:27.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.1810.06793","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}