Draft:General Optimal control Problem Solver

From Wikipedia, the free encyclopedia


General Optimal control Problems Solver (GOPS) is an open-source reinforcement learning (RL) package that aims to address optimal control problems in industrial fields.[1] GOPS is developed by iDLab (Intelligent Driving Laboratory)[2] at Tsinghua University. It is built with a modular structure, enabling the creation of controllers for diverse industrial tasks.

Overview[edit]

Addressing optimal control problems is essential for meeting the basic requirements of industrial control tasks. Traditional approaches such as model predictive control often encounter significant computational burdens during real-time execution. GOPS is developed for building real-time controllers in industrial applications using RL techniques. GOPS has a modular architecture, which provides flexibility for further development, catering to the diverse needs of industrial control tasks. GOPS includes a conversion tool that enables integration with Matlab/Simulink, facilitating environment construction, controller design, and performance validation. GOPS also incorporate both serial and parallel trainers with embedded buffers and samplers to tackle large-scale control problems. Moreover, GOPS offers a range of common approximate functions for policy and value functions, including polynomial, multilayer perceptron, and convolutional neural network models.

Features[edit]

GOPS presents a set of features specifically designed for industrial control applications:

  • Modular Configuration: GOPS is built with a modular structure, allowing for customization and development of environments and algorithms.
  • Diverse Training Modes: GOPS supports different training modes, including serial and parallel setups, on-policy and off-policy approaches, as well as model-free and model-based algorithms.
  • Compatibility with Matlab/Simulink: GOPS provides a conversion tool for Matlab/Simulink, which converts Simulink models into GOPS-compatible environments and sends learned policies back to Simulink for further integration and evaluation.

Applications[edit]

Applications of GOPS in industrial control scenarios include: coupled velocity and energy management optimization,[3] travel pattern analysis and demand prediction,[4] design of reward functions in vehicle control,[5] improving freeway merging efficiency,[6] vehicle speed control strategies,[7] multi-agent RL for platoon following,[8] origin-destination ride-hailing demand prediction,[9] accelerating model predictive path integral,[10] drill boom hole-seeking control,[11] etc.

Documentation and Usage[edit]

The GOPS package is available on GitHub at Intelligent-Driving-Laboratory/GOPS,[12] where users can access the source code and contribute to its development. Further details, including installation instructions, usage guidelines, and examples, are provided in the GOPS documentation.[13]

References[edit]

  1. ^ Wang, Wenxuan; Zhang, Yuhang; Gao, Jiaxin; Jiang, Yuxuan; Yang, Yujie; Zheng, Zhilong; Zou, Wenjun; Li, Jie; Zhang, Congsheng; Cao, Wenhan; Xie, Genjin; Duan, Jingliang; Li, Shengbo Eben (2023). "GOPS: A general optimal control problem solver for autonomous driving and industrial control applications". Communications in Transportation Research. 3: 100096. doi:10.1016/j.commtr.2023.100096. ISSN 2772-4247.
  2. ^ "iDLab, Tsinghua (清华大学智能驾驶实验室)". www.idlab-tsinghua.com.
  3. ^ Zhang, Hao; Chen, Boli; Lei, Nuo; Li, Bingbing; Chen, Chaoyi; Wang, Zhi (2024). "Coupled velocity and energy management optimization of connected hybrid electric vehicles for maximum collective efficiency". Applied Energy. 360: 122792. doi:10.1016/j.apenergy.2024.122792. ISSN 0306-2619.
  4. ^ Lin, Hongyi; He, Yixu; Li, Shen; Liu, Yang (2024). "Insights into Travel Pattern Analysis and Demand Prediction: A Data-Driven Approach in Bike-Sharing Systems". Journal of Transportation Engineering, Part A: Systems. 150 (2). doi:10.1061/JTEPBS.TEENG-8137. ISSN 2473-2907. S2CID 265334489.
  5. ^ He, Yixu; Liu, Yang; Yang, Lan; Qu, Xiaobo (2024-01-17). "Exploring the design of reward functions in deep reinforcement learning-based vehicle velocity control algorithms". Transportation Letters: 1–15. doi:10.1080/19427867.2024.2305018. ISSN 1942-7867. S2CID 267038400.
  6. ^ Zhu, Jie; Wang, Liang; Tasic, Ivana; Qu, Xiaobo (2024). "Improving Freeway Merging Efficiency via Flow-Level Coordination of Connected and Autonomous Vehicles". IEEE Transactions on Intelligent Transportation Systems: 1–13. arXiv:2108.01875. doi:10.1109/TITS.2023.3346832. ISSN 1524-9050. S2CID 267181762.
  7. ^ Ma, Changxi; Li, Yuanping; Meng, Wei (2023). "A Review of Vehicle Speed Control Strategies". Journal of Intelligent and Connected Vehicles. 6 (4): 190–201. doi:10.26599/JICV.2023.9210010. ISSN 2399-9802.
  8. ^ Lin, Hongyi; Lyu, Cheng; He, Yixu; Liu, Yang; Gao, Kun; Qu, Xiaobo (2024). "Enhancing State Representation in Multi-Agent Reinforcement Learning for Platoon-Following Models". IEEE Transactions on Vehicular Technology: 1–5. doi:10.1109/TVT.2024.3373533. ISSN 0018-9545.
  9. ^ Lin, Hongyi; He, Yixu; Liu, Yang; Gao, Kun; Qu, Xiaobo (2024). "Deep Demand Prediction: An Enhanced Conformer Model With Cold-Start Adaptation for Origin–Destination Ride-Hailing Demand Prediction". IEEE Intelligent Transportation Systems Magazine: 2–15. doi:10.1109/MITS.2023.3309653. ISSN 1939-1390. S2CID 261800438.
  10. ^ Qu, Yue; Chu, Hongqing; Gao, Shuhua; Guan, Jun; Yan, Haoqi; Xiao, Liming; Li, Shengbo Eben; Duan, Jingliang (2023). "RL-Driven MPPI: Accelerating Online Control Laws Calculation With Offline Policy". IEEE Transactions on Intelligent Vehicles: 1–12. doi:10.1109/TIV.2023.3348134. ISSN 2379-8904. S2CID 266669474.
  11. ^ Yan, Haoqi; Xu, Haoyuan; Gao, Hongbo; Ma, Fei; Li, Shengbo Eben; Duan, Jingliang (2023-10-13). "Integrated Drill Boom Hole-Seeking Control via Reinforcement Learning". 2023 IEEE International Conference on Unmanned Systems (ICUS). IEEE. pp. 1247–1254. arXiv:2312.01836. doi:10.1109/ICUS58632.2023.10318393. ISBN 979-8-3503-1630-8. S2CID 265355416.
  12. ^ "Intelligent-Driving-Laboratory/GOPS". March 7, 2024 – via GitHub.
  13. ^ "Welcome to GOPS's documentation! — GOPS 1.1.0 documentation". gops.readthedocs.io.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Draft:General_Optimal_control_Problem_Solver&oldid=1214479717"