This textbook provides a structured introduction to stochastic controlthe mathematical framework for decision-making in systems influenced by randomness. It explains how stochastic processes evolve, how decisions affect their trajectories, and which mathematical tools are required to model and analyze these interactions, finding rich applications in areas such as financial engineering, artificial intelligence, and industrial automation, to name a few.

The content progresses from probability preliminaries and Markov chains to dynamic programming, controlled Markov processes, asymptotic control problems, and ergodic and stability considerations. Additional appendices supply supporting material on matrices, convergence, analysis results, convexity, and Python-based simulations, enabling readers to review prerequisite concepts as needed.

Intended for upper-undergraduate and early graduate students, the book will benefit those in mathematics, engineering, actuarial science, and related disciplines. It is also a useful resource for instructors designing courses on probability, stochastic processes, or stochastic control. Readers with a background in calculus, linear algebra, and probability can use the text to develop the theoretical and computational skills required for the study and application of the methods contained within.

Introduces the core principles of stochastic control for systems driven by randomness Covers key topics from probability and Markov chains to dynamic programming and ergodic analysis Serves as a resource for instructors and students in mathematics, engineering, actuarial science, and related fields

Autorentext

Daniel Hernández-Hernández is a Full Professor at the Centro de Investigación en Matemáticas (CIMAT) in Guanajuato, Mexico. He holds a PhD in Mathematics from CINVESTAV-IPN (1993), where he completed a dissertation on optimal control and infinite-dimensional linear programming. His research focuses on stochastic control, stochastic processes, and mathematical finance.

Nuria Sydykova Méndez studied Mathematics at the University of Guanajuato and works in the financial sector. She has represented Mexico in several international mathematical competitions, earning multiple medals at the European Girls' Mathematical Olympiad.

Inhalt

Probability preliminaries.- Markov chains.- Dynamic programming.- Controlled Markov processes.- Asymptotic problems.- Exponential ergodicity.- Matrices and convergence -.Analysis results.- Convexity concepts.- Python programs.