Problems and Solutions in Optimization

Problems and Solutions in Optimization

by Willi-Hans Steeb International School for Scientific Computing at University of Johannesburg, South Africa

Yorick Hardy Department of Mathematical Sciences at University of South Africa

George Dori Anescu email: george.anescu@

Preface

v

Preface

The purpose of this book is to supply a collection of problems in optimization theory.

Prescribed book for problems.

The Nonlinear Workbook: 5th edition by Willi-Hans Steeb World Scientific Publishing, Singapore 2011 ISBN 978-981-4335-77-5

The International School for Scientific Computing (ISSC) provides certificate courses for this subject. Please contact the author if you want to do this course or other courses of the ISSC.

e-mail addresses of the author:

steebwilli@ steeb_wh@

Home page of the author:



Contents

Notation

ix

1 General

1

1.1 One-Dimensional Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Solved Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.2 Supplementary Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2 Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.3 Two-Dimensional Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4 Problems with Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.5 Problems with Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1.6 Problems with Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2 Lagrange Multiplier Method

27

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Solved Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.3 Supplementary Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3 Differential Forms and Lagrange Multiplier

43

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.2 Solved Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.3 Supplementary Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4 Penalty Method

49

5 Simplex Method

50

6 Karush-Kuhn-Tucker Conditions

56

6.1 Linear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.2 Nonlinear Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6.3 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

7 Fixed Points

66

vi

Contents

vii

8 Neural Networks

69

8.1 Hopfield Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8.2 Kohonen Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

8.3 Hyperplanes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

8.4 Perceptron Learning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

8.5 Back-Propagation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

8.6 Hausdorff Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

8.7 Miscellany . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

9 Genetic Algorithms

85

10 Euler-Lagrange Equations

91

11 Numerical Implementations

95

Bibliography

98

Index

100

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download