Lecture 4D/5A (2026-03-19): Distributed and Parallel GA's and Introduction to Simulated Annealing (SA)

In this lecture, we wrap up our units on evolutionary algorithms, closing on Distributed (Island Model) and Parallel Genetic Algorithms. We describe the basic population structure and migration approaches in Distributed GA's and explore whether Sewall Wright's shifting-balance theory (SBT) can explain DGA's success on certain landscapes. We then pivot to a new unit on physics-inspired ML and optimization approaches, where Simulated Annealing (SA) is one of the key topics. We introduce Simulated Annealing and discuss how hardware annealers can solve a broad set of combinatorial problems that can be QUBO (Quadratic Unconstrained Binary Optimization) encoded. We setup the basic content grammar for the unit by introducing macrostate, microstate, temperature, and energy, and then we give an animated outline of how the basic SA algorithm works. We will use this SA to motivate our explorations into entropy, MaxEnt, Boltzmann sampling, and more in future lectures in this unit.

Shifting-Balance Theory visualizer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/shifting_balance_theory/sbt_four_peaks.html

Simulated Annealing explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/simulated_annealing/simulated_annealing_demo.html

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/b8v78jmem4j9spju7sa8k/IEE598-Lecture4D_5A-2026-03-19-Distributed_and_Parallel_GAs_and_Introduction_to_Simulated_Annealing_SA-Notes.pdf?rlkey=qfh29uk7ckfb8aphn1k645r9e&dl=0

Lecture 4C (2026-03-17): From Niches to Meta-Populations: Toward Distributed and Parallel Genetic Algorithms (DGA/PGA)

In this lecture, we close out our discussion of "niching" diversity-preservation approaches for multi-modal and multi-objective evolutionary algorithms. We had covered clearing/clustering algorithms in the past lecture (Lecture 4B), and so we start on crowding algorithms, including Restricted Tournament Selection (RTS), briefly the introduce Species Conserving Genetic Algorithm (SCGA), and then close with a discussion of islanding approaches. This sets up an introduction to distributed (and parallel) genetic algorithms, which we will start out with in the next lecture.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/ngcurzxer85i4oft1qn68/IEE598-Lecture4C-2026-03-17-From_Niches_to_Meta_Populations-Distributed_and_Parallel_GA-Notes.pdf?rlkey=x8mb0bn5d56lhwtjftjx323u6&dl=0

Lecture 4B (2026-03-05): Niching Methods for Diversity Preservation in Multi-Objective and Multi-Modal Evolutionary Algorithms

In this lecture, we cover several of the different "niching methods" used for diversity preservation in both multi-objective and multi-modal evolutionary algorithms. We start with an overall goal to create "negative frequency-dependent selection" (or density dependence) that has the potential to be able to stabilize different subpopulations coexisting with each other. We start by discussing how evolutionary models like Hawk–Dove ("Chicken") have mixed Nash equilibria that can represent stable co-existence of discrete phenotypes (due to negative frequency dependence). But then we pivot to habitat selection models, with particular focus on the Ideal Free Distribution (IFD), as a better match for the diversity-preservation problem in MOEA's and MMEA's. That allows us to introduce "fitness sharing" (which matches very closely to the IFD) and various other fitness-modification methods that each have different computational costs and diversity benefits. We close by introduction selection-based approaches, such as breaking tournament-selection ties by crowding distance.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/d2ucw5j4lqtlj6hzue2mk/IEE598-Lecture4B-2026-03-05-Niching_Methods_for_Diversity_Preservation_in_MOEA_and_MMO-Notes.pdf?rlkey=rvvs5xy2qmbva7xl1glsmhnja&dl=0

Lecture 3D/4A (2026-03-03): From Multi-Objective to Multi-Modal Optimization

In this lecture, we wrap up our discussion of Pareto ranking for Multi-Objective Evolutionary Algorithms (MOEA's) and then introduce the topic of diversity-preservation methods ("niching" methods) that maintain diversity across the Pareto frontier. We then pivot to introducing Multi-Modal Optimization (MMO), which also requires "niching" methods to populate the different peaks of the optimization objective. We close by starting to set up background that motivates the particular designs of niche-preserving methods.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/blmmcyw7e0uf2ivjh1lkk/IEE598-Lecture3D_4A-2026-03-03-From_Multi_Objective_to_Multi_Modal_Optimization-Notes.pdf?rlkey=q95a982to30ovnv6izej1cd5y&dl=0