{"data":{"id":"10.48550/arxiv.2609.13243","type":"dois","attributes":{"doi":"10.48550/arxiv.2609.13243","prefix":"10.48550","suffix":"arxiv.2609.13243","identifiers":[{"identifier":"2609.13243","identifierType":"arXiv"}],"alternateIdentifiers":[{"alternateIdentifierType":"arXiv","alternateIdentifier":"2609.13243"}],"creators":[{"name":"Haridevan, Amal Dev","nameType":"Personal","givenName":"Amal Dev","familyName":"Haridevan","affiliation":[],"nameIdentifiers":[]},{"name":"Kang, Junjie","nameType":"Personal","givenName":"Junjie","familyName":"Kang","affiliation":[],"nameIdentifiers":[]},{"name":"Shan, Jinjun","nameType":"Personal","givenName":"Jinjun","familyName":"Shan","affiliation":[],"nameIdentifiers":[]}],"titles":[{"title":"GzDRL: Reproducible and Scalable Deep Reinforcement Learning with Gazebo"}],"publisher":"arXiv","container":{},"publicationYear":2026,"subjects":[{"lang":"en","subject":"Robotics (cs.RO)","subjectScheme":"arXiv"},{"lang":"en","subject":"Machine Learning (cs.LG)","subjectScheme":"arXiv"},{"subject":"FOS: Computer and information sciences","subjectScheme":"Fields of Science and Technology (FOS)"}],"contributors":[],"dates":[{"date":"2026-09-03T17:35:13Z","dateType":"Submitted","dateInformation":"v1"},{"date":"2026-09-15T00:02:15Z","dateType":"Updated","dateInformation":"v1"},{"date":"2026-09","dateType":"Available","dateInformation":"v1"}],"language":null,"types":{"ris":"GEN","bibtex":"misc","citeproc":"article","schemaOrg":"CreativeWork","resourceType":"Article","resourceTypeGeneral":"Preprint"},"relatedIdentifiers":[],"relatedItems":[],"sizes":[],"formats":[],"version":"1","rightsList":[{"rights":"arXiv.org perpetual, non-exclusive license","rightsUri":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/"}],"descriptions":[{"description":"We present GzDRL, a novel single-process reinforcement learning (RL) framework for Gazebo that overcomes longstanding bottlenecks in scalable, reproducible robotics experimentation. Unlike conventional middleware-based RL-Gazebo integrations that suffer from nondeterminism and irreproducibility, GzDRL introduces a systematic, middleware-free environment-stepping mechanism that directly synchronizes agent actions and physics updates. This design enables deterministic, high-throughput data collection, efficient vectorization, and reproducible RL training and evaluation. Comprehensive benchmarks demonstrate that GzDRL achieves the highest workstation throughput among the evaluated frameworks while remaining competitive with GPU-accelerated simulators on laptop hardware, and maintains precise agent-environment synchronization, multi-agent scalability, and experiment-level reproducibility. We further validate sim-to-real transfer by deploying learned policies directly onto a physical quadrotor, without fine-tuning. Our results establish GzDRL as an accessible and reproducible platform for advancing RL in robotics and automation.","descriptionType":"Abstract"},{"description":"This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible","descriptionType":"Other"}],"geoLocations":[],"fundingReferences":[],"xml":"PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPHJlc291cmNlIHhtbG5zPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSIgeHNpOnNjaGVtYUxvY2F0aW9uPSJodHRwOi8vZGF0YWNpdGUub3JnL3NjaGVtYS9rZXJuZWwtNCBodHRwOi8vc2NoZW1hLmRhdGFjaXRlLm9yZy9tZXRhL2tlcm5lbC00LjMvbWV0YWRhdGEueHNkIj4KICA8aWRlbnRpZmllciBpZGVudGlmaWVyVHlwZT0iRE9JIj4xMC40ODU1MC9BUlhJVi4yNjA5LjEzMjQzPC9pZGVudGlmaWVyPgogIDxhbHRlcm5hdGVJZGVudGlmaWVycz4KICAgIDxhbHRlcm5hdGVJZGVudGlmaWVyIGFsdGVybmF0ZUlkZW50aWZpZXJUeXBlPSJhclhpdiI+MjYwOS4xMzI0MzwvYWx0ZXJuYXRlSWRlbnRpZmllcj4KICA8L2FsdGVybmF0ZUlkZW50aWZpZXJzPgogIDxjcmVhdG9ycz4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5IYXJpZGV2YW4sIEFtYWwgRGV2PC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5BbWFsIERldjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5IYXJpZGV2YW48L2ZhbWlseU5hbWU+CiAgICA8L2NyZWF0b3I+CiAgICA8Y3JlYXRvcj4KICAgICAgPGNyZWF0b3JOYW1lIG5hbWVUeXBlPSJQZXJzb25hbCI+S2FuZywgSnVuamllPC9jcmVhdG9yTmFtZT4KICAgICAgPGdpdmVuTmFtZT5KdW5qaWU8L2dpdmVuTmFtZT4KICAgICAgPGZhbWlseU5hbWU+S2FuZzwvZmFtaWx5TmFtZT4KICAgIDwvY3JlYXRvcj4KICAgIDxjcmVhdG9yPgogICAgICA8Y3JlYXRvck5hbWUgbmFtZVR5cGU9IlBlcnNvbmFsIj5TaGFuLCBKaW5qdW48L2NyZWF0b3JOYW1lPgogICAgICA8Z2l2ZW5OYW1lPkppbmp1bjwvZ2l2ZW5OYW1lPgogICAgICA8ZmFtaWx5TmFtZT5TaGFuPC9mYW1pbHlOYW1lPgogICAgPC9jcmVhdG9yPgogIDwvY3JlYXRvcnM+CiAgPHRpdGxlcz4KICAgIDx0aXRsZT5HekRSTDogUmVwcm9kdWNpYmxlIGFuZCBTY2FsYWJsZSBEZWVwIFJlaW5mb3JjZW1lbnQgTGVhcm5pbmcgd2l0aCBHYXplYm88L3RpdGxlPgogIDwvdGl0bGVzPgogIDxwdWJsaXNoZXI+YXJYaXY8L3B1Ymxpc2hlcj4KICA8cHVibGljYXRpb25ZZWFyPjIwMjY8L3B1YmxpY2F0aW9uWWVhcj4KICA8c3ViamVjdHM+CiAgICA8c3ViamVjdCB4bWw6bGFuZz0iZW4iIHN1YmplY3RTY2hlbWU9ImFyWGl2Ij5Sb2JvdGljcyAoY3MuUk8pPC9zdWJqZWN0PgogICAgPHN1YmplY3QgeG1sOmxhbmc9ImVuIiBzdWJqZWN0U2NoZW1lPSJhclhpdiI+TWFjaGluZSBMZWFybmluZyAoY3MuTEcpPC9zdWJqZWN0PgogICAgPHN1YmplY3Qgc3ViamVjdFNjaGVtZT0iRmllbGRzIG9mIFNjaWVuY2UgYW5kIFRlY2hub2xvZ3kgKEZPUykiPkZPUzogQ29tcHV0ZXIgYW5kIGluZm9ybWF0aW9uIHNjaWVuY2VzPC9zdWJqZWN0PgogIDwvc3ViamVjdHM+CiAgPGRhdGVzPgogICAgPGRhdGUgZGF0ZVR5cGU9IlN1Ym1pdHRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyNi0wOS0wM1QxNzozNToxM1o8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iVXBkYXRlZCIgZGF0ZUluZm9ybWF0aW9uPSJ2MSI+MjAyNi0wOS0xNVQwMDowMjoxNVo8L2RhdGU+CiAgICA8ZGF0ZSBkYXRlVHlwZT0iQXZhaWxhYmxlIiBkYXRlSW5mb3JtYXRpb249InYxIj4yMDI2LTA5PC9kYXRlPgogIDwvZGF0ZXM+CiAgPHJlc291cmNlVHlwZSByZXNvdXJjZVR5cGVHZW5lcmFsPSJQcmVwcmludCI+QXJ0aWNsZTwvcmVzb3VyY2VUeXBlPgogIDx2ZXJzaW9uPjE8L3ZlcnNpb24+CiAgPHJpZ2h0c0xpc3Q+CiAgICA8cmlnaHRzIHJpZ2h0c1VSST0iaHR0cDovL2FyeGl2Lm9yZy9saWNlbnNlcy9ub25leGNsdXNpdmUtZGlzdHJpYi8xLjAvIj5hclhpdi5vcmcgcGVycGV0dWFsLCBub24tZXhjbHVzaXZlIGxpY2Vuc2U8L3JpZ2h0cz4KICA8L3JpZ2h0c0xpc3Q+CiAgPGRlc2NyaXB0aW9ucz4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9IkFic3RyYWN0Ij5XZSBwcmVzZW50IEd6RFJMLCBhIG5vdmVsIHNpbmdsZS1wcm9jZXNzIHJlaW5mb3JjZW1lbnQgbGVhcm5pbmcgKFJMKSBmcmFtZXdvcmsgZm9yIEdhemVibyB0aGF0IG92ZXJjb21lcyBsb25nc3RhbmRpbmcgYm90dGxlbmVja3MgaW4gc2NhbGFibGUsIHJlcHJvZHVjaWJsZSByb2JvdGljcyBleHBlcmltZW50YXRpb24uIFVubGlrZSBjb252ZW50aW9uYWwgbWlkZGxld2FyZS1iYXNlZCBSTC1HYXplYm8gaW50ZWdyYXRpb25zIHRoYXQgc3VmZmVyIGZyb20gbm9uZGV0ZXJtaW5pc20gYW5kIGlycmVwcm9kdWNpYmlsaXR5LCBHekRSTCBpbnRyb2R1Y2VzIGEgc3lzdGVtYXRpYywgbWlkZGxld2FyZS1mcmVlIGVudmlyb25tZW50LXN0ZXBwaW5nIG1lY2hhbmlzbSB0aGF0IGRpcmVjdGx5IHN5bmNocm9uaXplcyBhZ2VudCBhY3Rpb25zIGFuZCBwaHlzaWNzIHVwZGF0ZXMuIFRoaXMgZGVzaWduIGVuYWJsZXMgZGV0ZXJtaW5pc3RpYywgaGlnaC10aHJvdWdocHV0IGRhdGEgY29sbGVjdGlvbiwgZWZmaWNpZW50IHZlY3Rvcml6YXRpb24sIGFuZCByZXByb2R1Y2libGUgUkwgdHJhaW5pbmcgYW5kIGV2YWx1YXRpb24uIENvbXByZWhlbnNpdmUgYmVuY2htYXJrcyBkZW1vbnN0cmF0ZSB0aGF0IEd6RFJMIGFjaGlldmVzIHRoZSBoaWdoZXN0IHdvcmtzdGF0aW9uIHRocm91Z2hwdXQgYW1vbmcgdGhlIGV2YWx1YXRlZCBmcmFtZXdvcmtzIHdoaWxlIHJlbWFpbmluZyBjb21wZXRpdGl2ZSB3aXRoIEdQVS1hY2NlbGVyYXRlZCBzaW11bGF0b3JzIG9uIGxhcHRvcCBoYXJkd2FyZSwgYW5kIG1haW50YWlucyBwcmVjaXNlIGFnZW50LWVudmlyb25tZW50IHN5bmNocm9uaXphdGlvbiwgbXVsdGktYWdlbnQgc2NhbGFiaWxpdHksIGFuZCBleHBlcmltZW50LWxldmVsIHJlcHJvZHVjaWJpbGl0eS4gV2UgZnVydGhlciB2YWxpZGF0ZSBzaW0tdG8tcmVhbCB0cmFuc2ZlciBieSBkZXBsb3lpbmcgbGVhcm5lZCBwb2xpY2llcyBkaXJlY3RseSBvbnRvIGEgcGh5c2ljYWwgcXVhZHJvdG9yLCB3aXRob3V0IGZpbmUtdHVuaW5nLiBPdXIgcmVzdWx0cyBlc3RhYmxpc2ggR3pEUkwgYXMgYW4gYWNjZXNzaWJsZSBhbmQgcmVwcm9kdWNpYmxlIHBsYXRmb3JtIGZvciBhZHZhbmNpbmcgUkwgaW4gcm9ib3RpY3MgYW5kIGF1dG9tYXRpb24uPC9kZXNjcmlwdGlvbj4KICAgIDxkZXNjcmlwdGlvbiBkZXNjcmlwdGlvblR5cGU9Ik90aGVyIj5UaGlzIHdvcmsgaGFzIGJlZW4gc3VibWl0dGVkIHRvIHRoZSBJRUVFIGZvciBwb3NzaWJsZSBwdWJsaWNhdGlvbi4gQ29weXJpZ2h0IG1heSBiZSB0cmFuc2ZlcnJlZCB3aXRob3V0IG5vdGljZSwgYWZ0ZXIgd2hpY2ggdGhpcyB2ZXJzaW9uIG1heSBubyBsb25nZXIgYmUgYWNjZXNzaWJsZTwvZGVzY3JpcHRpb24+CiAgPC9kZXNjcmlwdGlvbnM+CjwvcmVzb3VyY2U+","url":"https://arxiv.org/abs/2609.13243","contentUrl":null,"metadataVersion":0,"schemaVersion":"http://datacite.org/schema/kernel-4","source":"mds","isActive":true,"state":"findable","reason":null,"viewCount":0,"viewsOverTime":[],"downloadCount":0,"downloadsOverTime":[],"referenceCount":0,"citationCount":0,"citationsOverTime":[],"partCount":0,"partOfCount":0,"versionCount":0,"versionOfCount":0,"created":"2026-09-15T03:33:18.000Z","registered":"2026-09-15T03:33:19.000Z","published":"2026","updated":"2026-09-15T03:33:19.000Z"},"relationships":{"client":{"data":{"id":"arxiv.content","type":"clients"}},"provider":{"data":{"id":"arxiv","type":"providers"}},"media":{"data":{"id":"10.48550/arxiv.2609.13243","type":"media"}},"references":{"data":[]},"citations":{"data":[]},"parts":{"data":[]},"partOf":{"data":[]},"versions":{"data":[]},"versionOf":{"data":[]}}}}