[1707.06347] Proximal Policy Optimization Algorithms

[1707.06347] Proximal Policy Optimization Algorithms 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 

 
 
 
 
 
--> 

 
 
 Computer Science > Machine Learning

 

 
 arXiv:1707.06347 (cs)
 
 
 
 
 
 [Submitted on 20 Jul 2017 ( v1 ), last revised 28 Aug 2017 (this version, v2)] 
 Title: Proximal Policy Optimization Algorithms

 Authors: John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , Oleg Klimov View a PDF of the paper titled Proximal Policy Optimization Algorithms, by John Schulman and 4 other authors 
 View PDF 

 
 Abstract: We propose a new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent. Whereas standard policy gradient methods perform one gradient update per data sample, we propose a novel objective function that enables multiple epochs of minibatch updates. The new methods, which we call proximal policy optimization (PPO), have some of the benefits of trust region policy optimization (TRPO), but they are much simpler to implement, more general, and have better sample complexity (empirically). Our experiments test PPO on a collection of benchmark tasks, including simulated robotic locomotion and Atari game playing, and we show that PPO outperforms other online policy gradient methods, and overall strikes a favorable balance between sample complexity, simplicity, and wall-time.
 

 
 
 
 Subjects: 
 
 Machine Learning (cs.LG) 
 
 Cite as: 
 arXiv:1707.06347 [cs.LG] 
 
 
 
 (or 
 arXiv:1707.06347v2 [cs.LG] for this version)
 
 
 
 
 https://doi.org/10.48550/arXiv.1707.06347 
 
 
 Focus to learn more 
 
 
 
 arXiv-issued DOI via DataCite 
 
 
 
 
 
 
 
 Submission history

 From: John Schulman [ view email ] 
 [v1] 
 Thu, 20 Jul 2017 02:32:33 UTC (2,178 KB)

 [v2] 
 Mon, 28 Aug 2017 09:20:06 UTC (2,537 KB)

 
 
 
 
 
 Full-text links: 
 Access Paper:

 
 
View a PDF of the paper titled Proximal Policy Optimization Algorithms, by John Schulman and 4 other authors View PDF 
 TeX Source
 
 
 view license 
 
 
 Current browse context: cs.LG 

 
 
 < prev 
 
 | 
 next > 
 

 
 new 
 | 
 recent 
 | 2017-07 
 
 Change to browse by:
 
 cs 
 
 

 
 
 References & Citations

 
 NASA ADS 
 Google Scholar 

 Semantic Scholar 

 
 
 

 
 
 18 blog links 

 ( what is this? )
 
 
 
 DBLP - CS Bibliography

 
 listing | bibtex 
 
 John Schulman 
 Filip Wolski 
 Prafulla Dhariwal 
 Alec Radford 
 Oleg Klimov 
 
 export BibTeX citation 
 Loading... 
 

 
 
 
 BibTeX formatted citation

 &times; 
 
 
 loading... 
 
 
 Data provided by: 
 
 
 
 
 Bookmark

 
 
 
 
 
 
 
 
 
 
 
 Bibliographic Tools 
 
 Bibliographic and Citation Tools

 
 
 
 
 
 
 Bibliographic Explorer Toggle 
 
 
 
 Bibliographic Explorer ( What is the Explorer? ) 
 
 
 
 
 
 
 
 Connected Papers Toggle 
 
 
 
 Connected Papers ( What is Connected Papers? ) 
 

... (truncated, 5 KB total)