Lecture 1G (2026-02-03): GA Wrap Up – Crossover, Mutation, & Tuning GA Operator Choices

In this lecture, we almost finish our discussion of the canonical Genetic Algorithm (GA) by covering different crossover and mutation operator choices. We discuss how mutation and crossover rates might change over time. We then end by returning to the selection operator to introduce Stochastic Uniform Sampling, a stratified sampling approach that reduce the variance in the number of offspring selected per high-fitness individual without affecting the mean. Next time, we will discuss how the five major hyperparameters and selection pressure work together to determine the effectiveness of the GA for a particular objective. We will also transition to Unit 2, where we will start by introducing ES and CMA-ES.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/rvv4bkrsiz4ixrhgitt39/IEE598-Lecture1G-2026-02-03-GA_Wrap_Up-Crossover_Mutation_and_Tuning_GA_Operator_Choices-Notes.pdf?rlkey=vcgleumzuv85moizkqibdnjhw&dl=0

Lecture 1F (2026-01-29): Operators of the Genetic Algorithm

In this lecture, we dive deeper into the basic Genetic Algorithm by describing the three major operators in any GA iteration – the selection operator, the crossover operator, and the mutation operator. We describe different forms of selection (fitness proportionate, ranking, and tournament) and how they vary in their ability to control selection pressure. We also discuss several forms of crossover (from single point to multi-point to uniform to taking random linear combinations) and their function as they move individuals around fitness landscapes. We will finish with the mutation operator next time, but that content is also covered in the pre-written slide notes linked below. After discussing the mutation operator and some optimizations of the GA itself, we will transition next to to evolutionary computing/programming.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/sa96xjlv5d8q3mc8l0bde/IEE598-Lecture1F-2026-01-29-Operators_of_the_Genetic_Algorithm-Notes.pdf?rlkey=54eow2a79g1437r7g19be7gjy&dl=0

Lecture 1E (2026-01-27): Structure of the Basic Genetic Algorithm

In this lecture, we reveal the basic architecture of the simple GA. We start with defining how to concretely implement chromosomes/genomes, genes, alleles, characters, and traits numerically within an Engineering Design Optimization context. We then move on to a general definition of multi-objective fitness (which we will return to in Unit 3 when we study multi-objective evolutionary algorithms) and show how fitness functions can be scaled not only to meet the assumptions on fitness functions but also to adjust selective pressure as desired. We close with a flowchart of the steps of a basic genetic algorithm, highlighting operators (selection, crossover, and mutation) that we will discuss in detail in the next lecture.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/zdvfjtp88fl8omly7sd6u/IEE598-Lecture1E-2026-01-27-Structure_of_the_Basic_Genetic_Algorithm-Notes.pdf?rlkey=f0t4apfkyj0v9ketqxoy1xew1&dl=0

Lecture 1D (2025-01-22): The Four Forces of Evolution and The Drift Barrier

In this lecture, we review the four forces of evolution -- mutation, migration/gene flow, genetic drift, and natural selection -- and the contribution that each one makes to either increasing or decreasing variance in a population over time. So, a more complete picture of evolution is a tense combination of these forces, each of them leading to different kinds of effects on the distribution of alleles (strategies) in a population. We discuss the so-called "drift barrier" -- how the tendency for natural selection to produce higher quality solutions is ultimately limited by genetic drift that dominates when populations have low fitness diversity (low selective pressure) -- and we discuss how this sets up a speed–accuracy tradeoff between mutation (which counteracts drift in a way  that does not require more time for convergence but makes it impossible to fine tune solutions) and population size (which can fine tune solutions but requires a longer time to converge to a good solution). Selection operators and evolutionary hyper-parameters should be chosen with these pressures and tradeoffs in mind.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/g4sqsadzh22p2lay05n18/IEE598-Lecture1D-2025-01-22-The_Four_Forces_of_Evolution_and_The_Drift_Barrier-Notes.pdf?rlkey=ky9ol5itw1ipuehdmuhmylqd1&dl=0

Lecture 1C (2026-01-20): Population Genetics of Evolutionary Algorithms

In this lecture, we start by reviewing the basics our motivation to solve Engineering Design Optimization problems with evolutionary metaheuristics (a form of population-baed direct search approach). To prepare to introduce the Genetic Algorithm (GA), one of the most well-known Evolutionary Algorithms, we spend most of this lecture covering foundational topics from population and quantitative genetics that will give us the necessary vocabulary for discussing the GA. In particular, we introduce concepts of qualitative and quantitative traits, characters, phenotypes, genes, chromosomes, genomes, and genotypes. We also discuss the "GxE to P" relationship between genotype and phenotype and the connection between phenotype and fitness. We close with a discussion of the four forces of evolution (mutation, gene flow/migration, natural selection, and genetic drift). Next time, we will discuss the constant tension between natural selection and genetic drift (and mutation) and how to manage (and sometimes harness) this tension in an evolutionary metaheuristic.

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/0llubbbjkxxconlia235z/IEE598-Lecture1C-2026-01-20-Population_Genetics_of_Evolutionary_Algorithms-Notes.pdf?rlkey=pyc5jxvcmbjfietop5jchgiyp&dl=0

Lecture 1B (2026-01-15): Evolutionary Approach to Engineering Design Optimization

In this lecture, we formally introduce the Engineering Design Optimization (EDO) problem and several application spaces where it may apply. We then discuss classical approaches for using computational methods to solve this difficult optimization problem -- including both gradient-based and direct search methods. This allows us to introduce the categories of trajectory and local search methods (like tabu search and simulated annealing) and population-based methods (like the genetic algorithm, ant colony optimization, and particle swarm optimization). We then start down the path of exploring evolutionary algorithms, a special (but very large) set of population-based methods. In the next lecture, we will connect this discussion to population genetics and a basic Genetic Algorithm (GA).

The whiteboard notes taken during this lecture can be found at:
https://www.dropbox.com/scl/fi/kslpzf961mp4viwj557ed/IEE598-Lecture1B-2026-01-15-Evolutionary_Approach_to_Engineering_Design_Optimization-Notes.pdf?rlkey=xb0zoc1h74kbl5m1je7jb1p1c&dl=0

Lecture 1A (2026-01-13): Introduction to Course Policies and Motivations

This lecture introduces the main policies of the course and an outline of its content. We close with an introduction to the concepts of heuristics, metaheurisitcs, and hyperheuristics in the context of Engineering Design Optimization specifically and optimization more generally. We hint at the idea that nature provides templates for heuristics at all three levels, and this class aims to understand how these natural systems work and what can be taken from them in the design of heuristics for engineered systems.

Whiteboard note for this lecture can be found at: https://www.dropbox.com/scl/fi/gza807hargomj414wo7fz/IEE598-Lecture1A-2026-01-13-Introduction_to_Course_Policies_and_Motivations-Notes.pdf?rlkey=bl3tx0oa1vbaz79vxahrxipmi&dl=0