Autorentext
Elad Hazan is Professor of Computer Science at Princeton University. His research focuses on the design and analysis of algorithms for basic problems in machine learning and optimization. He is a pioneer of online nonstochastic control theory. Karan Singh is Assistant Professor of Operations Research at Carnegie Mellon University, and has previously worked at Google Brain and Microsoft Research. He works on the foundations of machine learning, control, and reinforcement learning.

Klappentext
This tutorial guide introduces online nonstochastic control, an emerging paradigm in control of dynamical systems and differentiable reinforcement learning that applies techniques from online convex optimization and convex relaxations to obtain new methods with provable guarantees for classical settings in optimal and robust control. In optimal control, robust control, and other control methodologies that assume stochastic noise, the goal is to perform comparably to an offline optimal strategy. In online control, both cost functions and perturbations from the assumed dynamical model are chosen by an adversary. Thus, the optimal policy is not defined a priori and the goal is to attain low regret against the best policy in hindsight from a benchmark class of policies. The resulting methods are based on iterative mathematical optimization algorithms and are accompanied by finite-time regret and computational complexity guarantees. This book is ideal for graduate students and researchers interested in bridging classical control theory and modern machine learning.

Inhalt

Symbols; Part I. Background in Control and RL: 1. Introduction; 2. Dynamical systems; 3. Markov decision processes; 4. Linear dynamical systems; 5. Optimal control of linear dynamical systems; Part II. Basics of Online Control: 6. Regret in control; 7. Online nonstochastic control; 8. Online nonstochastic system identification; Part III. Learning and Filtering: 9. Learning in unknown linear dynamical systems; 10. Kalman filtering; 11. Spectral filtering; Part IV. Online Control with Partial Observation: 12. Policy classes for partially observed systems; 13. Online nonstochastic control with partial observation; References; Index.