Lecture 2A (2022-01-27): GA Approaches Beyond Optimization - Artificial Immune Systems

In this lecture, we finish our discussion of (a simple version of) the Genetic Algorithm (GA), focusing on different genetic operators for recombination and mutation. We then close our discussion of the GA (for now) emphasizing that selection of these operators and other hyperparameters reflects the delicate balance between natural selection and drift -- with too much selective pressure leading to premature convergence on local suboptima and too little selective pressure leading to drift taking over and a random strategy becoming fixed in the population. At the end of this lecture, we pivot to introducing immunocomputing and artificial immune systems. We will pick up with the AIS discussion next lecture.

Whiteboard notes from this lecture can be found at: https://www.dropbox.com/s/gp0jd6acmtfj8ty/IEE598-Lecture2A-2022-01-27-Beyond_Genetic_Optimization-AIS_and_GenProg.pdf?dl=0

Lecture 1E (2022-01-25): The Genetic Algorithm and its Implementation, Part 2

In this lecture, we reveal the basic architecture of the simple GA. We then introduce three common selection operators (fitness-proportionate selection, ranking selection, and tournament selection). We then start to discuss crossover operators; we will finish with mutation operators next lecture. Throughout this lecture, the basic tension between natural/artificial selection and drift is emphasized -- there is a downside to having selection operators that have high selection pressure (because they get stuck in suboptimal traps), but very low selection pressure can cause small populations to stagnate due to drift.

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/s/vobploq2iclraln/IEE598-Lecture1E-2022-01-25-The_Genetic_Algorithm_and_its_Implementation-Part_2.pdf?dl=0

Lecture 1D (2022-01-20): The Genetic Algorithm and its Implementation, Part 1

In this lecture, we review the two major evolutionary forces of natural selection and genetic drift and how they tend to reduce variance in a population over time. We then introduce mutation and recombination, which add diversity to populations. So a more complete picture of evolution is a tense combination of these forces, each of them leading to different kinds of effects on the distribution of alleles (strategies) in a population. This allows us to introduce a concrete example of an evolutionary algorithm -- the genetic algorithm (GA). We start to define its key components and sequence of operation. In Part 2 (the next lecture), we will go into detail about the selection, mutation, and recombination "genetic operators."

Whiteboard notes for this lecture are archived at: https://www.dropbox.com/s/arz17hcpiw4wacy/IEE598-Lecture1D-2022-01-20-The_Genetic_Algorithm_and_its_Implementation-Part_1.pdf?dl=0

Lecture 1C (2022-01-18): Population Genetics of Evolutionary Algorithms

In this lecture, we review the fundamentals of evolutionary (population-based) metaheuristic optimization algorithms (for engineering design optimization, EDO). This allows us to introduce the evolutionary notions of selective pressure and drift, two forces that independently drive changes in the frequency of strategies in populations over generations. Before getting into the concrete details of the basic Genetic Algorithm (GA) next time, we used the rest of this lecture to introduce terminology from evolutionary biology and population genetics in life sciences. These terms (chromosomes, genes, alleles, phenotype, characters, traits, fitness) will often be used in discussions of evolutionary algorithms (EA) as well, and so it is useful to have some background in them.

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/s/9z2lc4y0om6kwxy/IEE598-Lecture1C-2022-01-18-Population_Genetics_of_Evolutionary_Algorithms.pdf?dl=0

Lecture 1B (2022-01-13): The Evolutionary Approach to Engineering Design Optimization

In this lecture, we formally introduce the Engineering Design Optimization (EDO) problem and several application spaces where it may apply. We then discuss classical approaches for using computational methods to solve this difficult optimization problem -- including both gradient-based and direct search methods. This allows us to introduce trajectory and local search methods (like tabu search and simulated annealing) and population-based methods (like the genetic algorithm, ant colony optimization, and particle swarm optimization). We then start down the path of exploring evolutionary algorithms, a special (but very large) set of population-based methods. In the next lecture, we will connect this discussion to population genetics and a basic Genetic Algorithm (GA).

The whiteboard notes taken during this lecture can be found at: https://www.dropbox.com/s/st88vx87y6tv0xo/IEE598-Lecture1B-2022-01-13-The_Evolutionary_Approach_to_Engineering_Design_Optimization.pdf?dl=0

Lecture 1A (2022-01-11): Introduction to Course, Metaheuristics, and Evolutionary Algorithms

In this first lecture of the semester, I introduce the course and its expectations and policies. After highlighting several sections of the syllabus, we pivot to discussing heuristics, metaheuristics, and hyperheuristics more broadly. This sets up for the next lecture which will introduce Evolutionary Algorithms, for use as metaheuristics.

The whiteboard notes for this lecture can be found at: https://www.dropbox.com/s/n1nrlk9k38tlgcx/IEE598-Lecture1A-2022-01-11-Introduction_to_Course_Metaheuristics_EA-Notes.pdf?dl=0