This book approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). It shows readers how to derive necessary stability and convergence criteria for their own systems.

There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods:
• infinite-horizon control for which the difficulty of solving partial differential HamiltonJacobiBellman equations directly is overcome, and proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences;
• finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control;
• nonlinear games for which a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point.
Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium.
In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time:
• establishes the fundamental theory involved clearly with each chapter devoted to aclearly identifiable control paradigm;
• demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and
• shows how ADP methods can be put to use both in simulation and in real applications.
This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study.

Convergence proofs of the algorithms presented teach readers how to derive necessary stability and convergence criteria for their own systems Establishes the fundamentals of ADP theory so that student readers can extrapolate their learning into control, operations research and related fields Applications examples show how the theory can be made to work in real example systems Includes supplementary material: sn.pub/extras

Autorentext

Hongjing Liang received his B.S. degree in mathematics from Bohai University, Jinzhou, China, in 2009, his M.S. degree in fundamental mathematics from Northeastern University, Shenyang, China, in 2011, and a Ph.D. degree in control theory and control engineering from Northeastern University, Shenyang, China, in 2016. He is currently working at Bohai University. His research interests include multi-agent systems, complex systems, and output regulation. Huaguang Zhang received his B.S. and M.S. degrees in control engineering from Northeastern Electric Power University, Jilin, China, 1982 and 1985, respectively, and a Ph.D. degree in thermal power engineering and automation from Southeast University, Nanjing, China, in 1991. Dr. Zhang joined the Department of Automatic Control, Northeastern University, Shenyang, China, in 1992, as a Postdoctoral Fellow. Since 1994, he has been a professor and the head of the Electric Automation Institute, Northeastern University. He has authored three English monographs and holds 30 patents. His main research interests are neural network-based control, fuzzy control, chaos control, nonlinear control, signal processing, adaptive dynamic programming (ADP) and their industrial applications. Dr. Zhang was a recipient of the Nationwide Excellent Post-Doctor, the Outstanding Youth Science Foundation Award from the National Natural Science Foundation Committee of China in 2003, the Cheung Kong Scholar Award from the Education Ministry of China in 2005, and the IEEE Transactions on Neural Networks Outstanding Paper Award in 2012. He was an Associate Editor of Automatica, IEEE Transactions on Cybernetics and Neurocomputing. He is the Deputy Director of the Intelligent System Engineering Committee of Chinese Association of Artificial Intelligence.

Inhalt
Optimal Stabilization Control for Discrete-time Systems.- Optimal Tracking Control for Discrete-time Systems.- Optimal Stabilization Control for Nonlinear Systems with Time Delays.- Optimal Tracking Control for Nonlinear Systems with Time-delays.- Optimal Feedback Control for Continuous-time Systems via ADP.- Several Special Optimal Feedback Control Designs Based on ADP.- Zero-sum Games for Discrete-time Systems Based on Model-free ADP.- Nonlinear Games for a Class of Continuous-time Systems Based on ADP.- Other Applications of ADP.