viz-optimization-algorithms

Jupyter notebooks, scripts, and results associated with the paper Visualization of Optimization Algorithms by Marco Morais (Morais, 2020).

Paper is available as tex and pdf.

I completed this project as the end of semester assignment for CS 519 Scientific Visualization at UIUC.

If you find this repo helpful, please cite this paper and star this repository. Thank you!

@article{morais2020,
  title={Visualization of Optimization Algorithms},
  author={Morais, Marco},
  year={2020}
}

Abstract

Optimization algorithms seek to find the best solution $x^{*}$ from a set S such that $f(x^{*}) \leq f(x)$ for all $x$ in S. For this project we describe and implement a handful of optimization algorithms, evaluate their performance on some well known test functions, and create visualizations to build some intuition and verify their function. Finally, we perform a comparative analysis between algorithms using repeated trials on these test functions in order to draw broader conclusions.

Notebooks

Self contained notebook that contains all of the code and results that appear in the paper.

Optimization-Visualization-mmorais2

Test Functions

Rosenbrock

Test-Function-Rosenbrock

Goldstein-Price

Test-Function-Goldstein-Price

Bartels-Conn

Test-Function-Bartels-Conn

Egg Crate

Test-Function-Egg-Crate

Optimization Algorithms

Gradient Descent

Gradient-Descent

BFGS

BFGS

Simulated Annealing

Simulated-Annealing

Particle Swarm

Particle-Swarm

Misc

Line-Search

Visualizations

Test Functions

Test-Function-Visualization-2D

Solutions, 2D

Solution-Visualization-2D

Solutions, 3D

Solution-Visualization-3D

Animations, 2D

Solution-Animation-2D

Results

Simulation-Results

Videos

The table belows lists links to selected 2d animations referred to in the project and uploaded to youtube.

Algorithm	Surface	Trial	Youtube Link
BFGS	Rosenbrock	1	youtube
BFGS	Rosenbrock	11	youtube
BFGS	Goldstein-Price	4	youtube
BFGS	Goldstein-Price	5	youtube
Simulated Annealing	Rosenbrock	10	youtube
Simulated Annealing	Goldstein-Price	6	youtube
Simulated Annealing	Bartels-Conn	1	youtube
Simulated Annealing	Bartels-Conn	12	youtube
Simulated Annealing	Egg Crate	1	youtube
Particle Swarm	Rosenbrock	5	youtube
Particle Swarm	Goldstein-Price	4	youtube
Particle Swarm	Goldstein-Price	7	youtube
Particle Swarm	Goldstein-Price	8	youtube
Particle Swarm	Bartels-Conn	2	youtube
Particle Swarm	Egg Crate	10	youtube
Particle Swarm	Egg Crate	11	youtube