Lecture 5A: Introduction to Simulated Annealing (2020-02-24)

In this lecture, we dive into the history behind simulated annealing and how it intersects with other tools such as MaxEnt (maximum entropy) methods used in NLP and Markov Chain Monte Carlo (MCMC) sampling. We go back to the original article by Metropolis et al. on using random sampling methods to estimate the mean properties of materials at equilibrium. This allows us to introduce the Boltzmann distribution and discuss its relationship to the exponential distribution and maximum entropy justifications for using Poisson process models. We then hint at the likelihood ratio generalization by Hastings and the ultimate co-opting of these methods for optimization later by Kirkpatrick et al.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/gqij9e21xf3cke2/IEE598-Lecture5A-Notes.pdf?dl=0