Reinforcement Learning for Engineering

The book is written from the perspective of the optimal feedback control of dynamic systems that evolve in either continuous- or discrete-time domains with emphasis on deterministic problem formulations over the corresponding stochastic problem formulations. Bellman's dynamic programmingoptimal feedback controlin continuous- and discrete-time domains forms the mathematical foundations of this book. In a simple and clear manner, this book relates the relation of one of the main techniques of the reinforcement learning approach in computer science, so-called Q-learning, to the Bellman dynamic programming functional difference equation.

Reinforcement Learning for Engineering contains several exercises, homework problems, and design projects (most using MATLAB® and Simulink® and some using Python) for real physical engineering systems. The book is a valuable reference for all researchers and practitioners interested in an engineering approach to reinforcement learning because it covers many essential results in a systematic manner. The book also presents and defines several future interesting and challenging research problems by providing a deeper physical and mathematical understanding of the optimal control Hamiltonians from the reinforcement learning point of view.

A structured treatment of significant results in the uses of reinforcement learning in control and engineering Exercises, homework problems and design projects implemented in MATLAB® and Python Presents several challenging open questions for future research

Autorentext

Professor Zoran Gaji received the Dipl.-Ing. (five year program) and Mgr.Sci. (two year program) degrees in electrical engineering from the University of Belgrade, the M.S. degree in applied mathematics, and the Ph.D. degree in systems science engineering under direction of Prof. Hassan Khalil from Michigan State University Department of Electrical Engineering and System Science in 1984. He was a Visiting Professor with Princeton University in 2003, and the American University of Sharjah in 2011, and Ajman University during 2022-23 school year. He is a Professor of Electrical and Computer Engineering with Rutgers University, where he has been involved in teaching linear systems and signals, controls, communication networks, optical networks, reinforcement learning, and electrical circuit courses since 1984. He has authored/co-authored over 100 journal papers, primarily published in IEEE Transactions on Automatic Control and Automatica , and eight books on linear systems and linear and bilinear control systems published by Academic Press, Prentice Hall, Marcel Dekker, Taylor and Francis, and Springer. The Chinese Simplified translation of his book Linear Dynamic Systems and Signal s was published by Jiaotong University Press in 2004. His 1995 book Lyapunov Matrix Equation in Systems Stability and Control was republished in 2008 by Dover Publications.

Dr. Gaji has supervised 19 doctoral dissertations and 26 master theses. Fourteen of his former doctoral students hold faculty positions with respected universities across the world. He has delivered four plenary lectures at international conferences and presented close to 150 conference papers. Dr. Gaji has served on editorial boards for nine journals and as a guest editor for six journal special issues. From 2003 to 2020 he was the Electrical and Computer Engineering Graduate Program Director at Rutgers. He served on the American Association of University Professors National Council for several years.

Dr. Lingyi Xu received her Ph.D. in Electrical and Computer Engineering from Rutgers University in 2024 with Prof. Zoran Gaji as her advisor. Prior to her doctoral studies, she received a Master's in control theory and control engineering from the Institute of Automation, Chinese Academy of Sciences in 2017 and a Master's in electrical and computer engineering from Rutgers University in 2018.

Dr. Xu has served as a reviewer for several journals such as IEEE Transactions on Automatic Control , Optics and Lasers in Engineering , and Engineering Optimization and conferences. Her research lies at the intersection and boundary of data-driven machine learning, reinforcement learning, computer vision, game theory, and optimal control, especially on machine learning and optimal control in trustworthy and responsible AI. Her work aims to train AI models to understand images and so to make reliable and trustworthy decisions. Dr. Xu's work on security-oriented cyber-physical systems improves the generalizability and reliability of data-driven machine learning on low-quality object detection and recognition in non-cooperative circumstances. Her work on multi-agent systems intends to enhance the efficacy and robustness of autonomous driving on tracked vehicles. Focussing on simultaneous on-line tracking and planning in leader-follower formation navigation for searching and rescue tasks. Her long-term research goal is on trustworthy and reliable general AI, teaching AI models to understand visual data in order to make interpretable, reliable, and trustworthy decisions in adversarial environments.

Inhalt

Introduction.- Dynamic Programming in Continuous Time.- Dynamic Programming (DP) in Discrete Time.- Hamiltonians and Their Physical and Mathematical Meanings.- Approximate Dynamic Programming (ADP) in Continuous Time.- Approximate Dynamic Programming (ADP) in Discrete Time.- Dynamic Programming for Zero-Sum Differential Games.- Dynamic Programming for Nash Differential Games.- ADP for Zero-Sum Dynamic Games.- ADP for Non-Zero Sum Nash Dynamic Games.- RL for Other Types of Differential Games.- Markov Decision Processes and Stochastic Dynamic Programming.- Reinforcement Learning Design Projects.- Conclusions and Future Work.- Appendix.