Lecture 5D (2026-03-31): Metropolis–Hastings Markov Chain Monte Carlo and Simulated Annealing/Parallel Tempering

In this lecture, we start with a reminder that the Boltzmann–Gibbs distribution is the maximal entropy (MaxEnt) distribution of physical microstates when the average energy is fixed at a temperature at thermal equilibrium. We then move toward motivations where it would be useful to sample microstates from such a distribution. First, we introduce Monte Carlo methods for parameter estimation, and we pivot toward applications of Monte Carlo sampling for numerical integration. This leads us back to physics applications where integration using the Boltzmann–Gibbs is much more practical. This gives the opportunity to introduce Metropolis–Hastings Markov Chain Monte Carlo (MCMC) sampling, which allows for sampling from the Boltzmann–Gibbs and more. After discussing connections to importance sampling (from stochastic simulation) and Bayesian/MCMC statistics, we introduce Simulated Annealing, which combines Metropolis–Hastings sampling with an annealing schedule for temperature. We close with a very brief introduction to Parallel Tempering, which swaps out the annealing schedule for parallel MCMC samplers that periodically swap states based on their relative energies. We will pick up with Parallel Tempering in the next lecture.

On-line simulations referenced in this lecture can be found at:

• Boltzmann–Gibbs distribution: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/boltzmann_maxent/boltzmann_maxent_random_exchange.html
• SoftMax Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/softmax/softmax_temperature_explorer.html
• Monte Carlo Estimation/Integration Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/monte_carlo/mc_explorer.html
• Simulated Annealing Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/simulated_annealing/simulated_annealing_demo.html
• Parallel Tempering Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/parallel_tempering/parallel_tempering.html

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/s5dcgqrvm4qzz4y0fs64a/IEE598-Lecture5D-2026-03-31-Markov_Chain_Monte_Carlo_Metropolis_and_Simulated_Annealing_Parallel_Tempering-Notes.pdf?rlkey=v2m33lhh7sjhwogffotbyq3k7&dl=0