Optimization in Engineering Sciences

The purpose of this book is to present the main metaheuristics
and approximate and stochastic methods for optimization of complex
systems in Engineering Sciences. It has been written within the
framework of the European Union project ERRIC (Empowering Romanian
Research on Intelligent Information Technologies), which is funded
by the EU's FP7 Research Potential program and has been
developed in co-operation between French and Romanian teaching
researchers. Through the principles of various proposed algorithms
(with additional references) this book allows the reader to explore
various methods of implementation such as metaheuristics, local
search and populationbased methods. It examines multi-objective and
stochastic optimization, as well as methods and tools for
computer-aided decision-making and simulation for
decision-making.

Autorentext

Dan Stefanoiu is Professor in the fields of Signal Processing and System Identification at Politehnica University of Bucharest. In 2002 he was elected as a member of the American Romanian Academy of Arts and Sciences (ARA).

Pierre Borne is Professor at École Centrale de Lille, France. He has received honorary degrees from the University of Moscow, Russia, the Politehnica University of Bucharest, Romania, and the University of Waterloo, Canada. He is a Fellow of the IEEE.

Dumitru Popescu is Professor at Politehnica University of Bucharest, Romania, in the fields of Advanced Control and Systems Optimization and also Associate Professor at major universities in France and Italy. He is Director of the Research Center in Automatics, Process Control and Computers (APCC) affiliated to Politehnica University of Bucharest, a member of the IFAC Technical Committees for the Control of Bioprocesses and Chemical Processes and a corresponding member of the Romanian Academy of Technical Sciences.

Florin Gh. Filip is a researcher and Professor in the fields of Optimization and Control of Large-scale Systems, applied IT including decision support systems. He was elected as a member of the Romanian Academy (National Academy of Sciences of Romania) in 1991 and President of the "Information Science and Technology" section of the Academy in 2011. He was the Vice President of the Romanian Academy from 2001 to 2010 and the chair of the IFAC TC 5.4 "Large-scale complex systems" from 2002 to 2008.

Abdelkader El Kamel is Professor in the fields of Advanced Control, System Optimization and Decision-Making at École Centrale de Lille in France. He is also a regular Visiting Professor, mainly in China, Chile and at major engineering and business schools in Tunisia.

Zusammenfassung

The purpose of this book is to present the main metaheuristics and approximate and stochastic methods for optimization of complex systems in Engineering Sciences. It has been written within the framework of the European Union project ERRIC (Empowering Romanian Research on Intelligent Information Technologies), which is funded by the EU's FP7 Research Potential program and has been developed in co-operation between French and Romanian teaching researchers. Through the principles of various proposed algorithms (with additional references) this book allows the reader to explore various methods of implementation such as metaheuristics, local search and populationbased methods. It examines multi-objective and stochastic optimization, as well as methods and tools for computer-aided decision-making and simulation for decision-making.

Inhalt

LIST OF FIGURES ix

LIST OF TABLES xiii

LIST OF ALGORITHMS xv

LIST OF ACRONYMS xvii

PREFACE xix

ACKNOWLEDGEMENTS xxi

CHAPTER 1. METAHEURISTICS LOCAL METHODS 1

1.1. Overview 1

1.2. Monte Carlo principle 6

1.3. Hill climbing 12

1.4. Taboo search 20

1.4.1. Principle 20

1.4.2. Greedy descent algorithm 20

1.4.3. Taboo search method 23

1.4.4. Taboo list 25

1.4.5. Taboo search algorithm 26

1.4.6. Intensification and diversification 30

1.4.7. Application examples 31

1.5. Simulated annealing 39

1.5.1. Principle of thermal annealing 39

1.5.2. Kirkpatrick's model of thermal annealing 41

1.5.3. Simulated annealing algorithm 43

1.6. Tunneling 46

1.6.1. Tunneling principle 46

1.6.2. Types of tunneling 48

1.6.3. Tunneling algorithm 49

1.7. GRASP methods 51

CHAPTER 2. METAHEURISTICS GLOBAL METHODS 53

2.1. Principle of evolutionary metaheuristics 53

2.2. Genetic algorithms 55

2.2.1. Biology breviary 55

2.2.2. Features of genetic algorithms 57

2.2.3. General structure of a GA 73

2.2.4. On the convergence of GA 77

2.2.5. How to implement a genetic algorithm 84

2.3. Hill climbing by evolutionary strategies 100

2.3.1. Climbing by the steepest ascent 101

2.3.2. Climbing by the next ascent 104

2.3.3. Hill climbing by group of alpinists 106

2.4. Optimization by ant colonies 107

2.4.1. Ant colonies 107

2.4.2. Basic optimization algorithm by ant colonies 110

2.4.3. Pheromone trail update 118

2.4.4. Systemic ant colony algorithm 122

2.4.5. Traveling salesman example 128

2.5. Particle swarm optimization 132

2.5.1. Basic metaheuristic 132

2.5.2. Standard PSO algorithm 141

2.5.3. Adaptive PSO algorithm with evolutionary strategy 146

2.5.4. Fireflies algorithm 163

2.5.5. Bats algorithm 173

2.5.6. Bees algorithm 182

2.5.7. Multivariable prediction by PSO 194

2.6. Optimization by harmony search 207

2.6.1. Musical composition and optimization 207

2.6.2. Harmony search model 208

2.6.3. Standard harmony search algorithm 212

2.6.4. Application example 215

CHAPTER 3. STOCHASTIC OPTIMIZATION 219

3.1. Introduction 219

3.2. Stochastic optimization problem 221

3.3. Computing the repartition function of a random variable 222

3.4. Statistical criteria for optimality 230

3.4.1. Case of totally admissible solutions 231

3.4.2. Case of partially admissible solutions 234

3.5. Examples 240

3.6. Stochastic optimization through games theory 245

3.6.1. Principle 245

3.6.2. Wald strategy (maximin) 247

3.6.3. Hurwicz strategy 248

3.6.4. Laplace strategy 249

3.6.5. BayesLaplace strategy 249

3.6.6. Savage strategy 250

3.6.7. Example 251

CHAPTER 4. MULTI-CRITERIA OPTIMIZATION 253

4.1. Introduction 253

4.2. Introductory examples 255

4.2.1. Choosing the first job 255

4.2.2. Selecting an IT tool 256

4.2.3. Setting the production rate of a continuous process plant 256

4.3. Multi-criteria optimization problems 257

4.3.1. Two subclasses of problems 257

4.3.2. Dominance and Pareto optimality 262

4.4. Model solving methods 265

4.4.1. Classifications 265

4.4.2. Substitution-based methods 266

4.4.3. Aggregation-based methods 270

4.4.4. Other methods 282

4.5. Two objective functions optimization for advanced control systems 292

4.5.1. Aggregating identification with the design of a dynamical control system 292 <p&...