Stochastic Averaging and Stochastic Extremum Seeking

This book develops stochastic averaging theorems and stochastic extremum-seeking algrithms, illustrating their use in a variety of models. Includes simulation examples based in bacterial locomotion, multi-agent robotic systems, and economic market models.

Stochastic Averaging and Extremum Seeking treats methods inspired by attempts to understand the seemingly non-mathematical question of bacterial chemotaxis and their application in other environments. The text presents significant generalizations on existing stochastic averaging theory developed from scratch and necessitated by the need to avoid violation of previous theoretical assumptions by algorithms which are otherwise effective in treating these systems. Coverage is given to four main topics.
Stochastic averaging theorems are developed for the analysis of continuous-time nonlinear systems with random forcing, removing prior restrictions on nonlinearity growth and on the finiteness of the time interval. The new stochastic averaging theorems are usable not only as approximation tools but also for providing stability guarantees.
Stochastic extremum-seeking algorithms are introduced for optimization of systems without available models. Both gradient- and Newton-based algorithms are presented, offering the user the choice between the simplicity of implementation (gradient) and the ability to achieve a known, arbitrary convergence rate (Newton).
The design of algorithms for non-cooperative/adversarial games is described. The analysis of their convergence to Nash equilibria is provided. The algorithms are illustrated on models of economic competition and on problems of the deployment of teams of robotic vehicles.
Bacterial locomotion, such as chemotaxis in E. coli, is explored with the aim of identifying two simple feedback laws for climbing nutrient gradients. Stochastic extremum seeking is shown to be a biologically-plausible interpretation for chemotaxis. For the same chemotaxis-inspired stochastic feedback laws, the book also provides a detailed analysis of convergence for models of nonholonomic robotic vehicles operating in GPS-denied environments.
The book contains block diagrams and several simulation examples, including examples arising from bacterial locomotion, multi-agent robotic systems, and economic market models.
Stochastic Averaging and Extremum Seeking will be informative for control engineers from backgrounds in electrical, mechanical, chemical and aerospace engineering and to applied mathematicians. Economics researchers, biologists, biophysicists and roboticists will find the applications examples instructive.

For engineers the text develops a stochastic version of increasingly popular deterministic extremum-seeking algorithms Demonstrates to the mathematician how stochastic averaging theory can be used as a tool for studying stability rather than just approximation Stochastic algorithms are intuitive and connect with the huge field of stochastic optimization Shows how control ideas derived from study of a biological system can be generalized into other widely-different fields of application Includes supplementary material: sn.pub/extras

Autorentext
Miroslav Krstic is an author of several books on adaptive control, stochastic nonlinear control, extremum seeking, and control of PDEs. Several of these books have had a high impact in the control field and inspired many researchers to work on the topics that the books have covered and apply the tools from the books in their research and in practice.
Shujun Liu is a young researcher in mathematics and control theory in China with strong connections with the leading research groups in control theory at the Chinese Academy of Sciences. Her doctoral work on stochastic stability and stabilization has had considerable influence on a number of research groups in China who have taken on this topic after her initial work with her doctoral advisor Professor Jifeng Zhang.
Much of the material of this book was developed while the first author was a postdoctoral scholar with the second author at University of California, San Diego.

Klappentext

Stochastic Averaging and Stochastic Extremum Seeking develops methods of mathematical analysis inspired by the interest in reverse engineering and analysis of bacterial convergence by chemotaxis and to apply similar stochastic optimization techniques in other environments.

The first half of the text presents significant advances in stochastic averaging theory, necessitated by the fact that existing theorems are restricted to systems with linear growth, globally exponentially stable average models, vanishing stochastic perturbations, and prevent analysis over infinite time horizon.

The second half of the text introduces stochastic extremum seeking algorithms for model-free optimization of systems in real time using stochastic perturbations for estimation of their gradients. Both gradient- and Newton-based algorithms are presented, offering the user the choice between the simplicity of implementation (gradient) and the ability to achieve a known, arbitrary convergence rate (Newton).

The design of algorithms for non-cooperative/adversarial games is described. The analysis of their convergence to Nash equilibria is provided. The algorithms are illustrated on models of economic competition and on problems of the deployment of teams of robotic vehicles.
Bacterial locomotion, such as chemotaxis in E. coli, is explored with the aim of identifying two simple feedback laws for climbing nutrient gradients. Stochastic extremum seeking is shown to be a biologically plausible interpretation for chemotaxis. For the same chemotaxis-inspired stochastic feedback laws, the book also provides a detailed analysis of convergence for models of nonholonomic robotic vehicles operating in GPS-denied environments.

The book contains block diagrams and several simulation examples, including examples arising from bacterial locomotion, multi-agent robotic systems, and economic market models.
Stochastic Averaging and ExtremumSeeking will be informative for control engineers from backgrounds in electrical, mechanical, chemical and aerospace engineering and to applied mathematicians. Economics researchers, biologists, biophysicists and roboticists will find the applications examples instructive.

The Communications and Control Engineering series reports major technological advances which have potential for great impact in the fields of communication and control. It reflects research in industrial and academic institutions around the world so that the readership can exploit new possibilities as they become available.

Inhalt
Stochastic Averaging for Aymptotic Stability.- Stochastic Averaging for Practical Stability.- Single-parameter Stochastic Extremum Seeking.- Stochastic Source Seeking for Nonholonomic Vehicles.- Stochastic Source Seeking with Tuning of Forward Velocity.- Multi-parameter Stochastic Extremum Seeking and Slope Seeking.- Stochastic Nash Equilibrium Seeking for Games with General Nonlinear Payoffs.- Nash Equilibrium Seeking for Quadratic Games and Application to Oligopoly Markets and Vehicle Deployment.- Newton-based Stochastic Extremum Seeking.