IEE/CSE 598: Bio-Inspired AI and Optimization

Thursday, April 23, 2026

Lecture 7E (2026-04-23): Natural Learning Experiences – Reinforcement and Unsupervised Learning

In this lecture, we introduce Temporal Difference (TD) Q-learning and Deep Q Networks, starting with an analogy to how ants encode estimates of reward for state–action pairs in pheromone trails in the environment (another way to store a "Q" table in a network). We then pivot to discussing unsupervised learning – including both clustering and multi-dimensional scaling. After discussing PCA and t-SNE (briefly), we pivot to describing the deep autoencoder and show an example of its use in an MNIST-like clustering task. Interactive demonstrations mentioned in this lecture include: * Marginal Value Theorem Explorer (to better understand discount rate): https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/optimal_foraging_theory/mvt_explorer.html * Autoencoder Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/unsupervised_learning/autoencoder_explorer.html Whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/assv5cheln8xqp2tvzj1k/IEE598-Lecture7E-2026-04-23-Natural_Learning_Experiences-Reinforcement_and_Unsupervised_Learning-Notes.pdf?rlkey=a3iyshlufzgkyfxl7gby85pe2&dl=0 An unabridged version of the whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/lgibsff4lhh0ezb1lnm41/IEE598-Lecture7E-2026-04-23-Natural_Learning_Experiences-Reinforcement_and_Unsupervised_Learning-Notes-Full.pdf?rlkey=jayejujed8ervq8zsrddi1vcr&dl=0

Tuesday, April 21, 2026

Lecture 7D (2026-04-21): RNN's – Backpropagation Through Time (BPTT), Long Short Term Memory (LSTM), and Reservoir Computing/Echo State Networks (ESNs)

In this lecture, we continue our discussion of Recurrent Neural Networks (RNN's) as generalized forms of Time Delay Neural Network (TDNN) that can do time-series classification (and prediction) using an inductive bias that can pull in information from a wide range of times (well beyond the simple size of the neural network, due to the use of output feedback to maintain state). We discuss how these networks can be trained with Backpropagation Through Time (BPTT) and some limitations of this approach. This motivates the more constrained Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures, which mitigate some issues with training general RNN's. We then pivot to a different approach entirely -- using recurrent neural networks as untrained "reservoirs" whose outputs are dynamical encoders that spread out temporal patterns into spatial ones that can be learned with a single-layer perceptron. We demonstrate this using an Echo State Network (ESN) and walk through how even small networks can provide significant separability for time series. We also have a discussion of how these approaches can be used for predicting chaotic time series, with applications in finance as well as digital twins (e.g., for manufacturing systems).

Interactive demonstrations connected to this lecture can be found at:

Multi-Layer Perceptron and Backpropagation Explainer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/multi_layer_perceptron/mlp_explorer.html
Recurrent Neural Networks (and BPTT and LSTM) Explainer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/recurrent_neural_networks/rnn_explorer.html
Reservoir Computing/Echo State Networks Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/reservoir_computing/esn_explorer.html

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/jdwe24zsmevhmaxl7x954/IEE598-Lecture7D-2026-04-21-RNNs-BPTT_LSTM_and_Reservoir_Computing-Notes.pdf?rlkey=22dv6950zcsjl98e0o96de11q&dl=0

Thursday, April 16, 2026

Lecture 7C (2026-04-16): Recurrent Networks and Temporal Supervision

In this lecture, we finish up our coverage of supervised learning of feedforward multi-layer perceptrons with a discussion of how the Convolutional Neural Network imposes an inductive bias that simplifies training and pays off for images but may not work so well for text strings. We then shift our focus to recurrent networks with temporal supervision, which may help to provide a solution when highly local inductive biases aren't effective (as in for text and time-series analysis). We discuss several coincidence detectors from neuroscience in the context of hearing and vision, and we use them to motivate Time Delay Neural Networks (TDNNs) as our bridge to Recurrent Neural Networks (RNNs). This allows for analogies to be made to Finite Impulse Response (FIR) and Infinite Impulse Response (IIR) filters. We close by transitioning from a basic output-feedback configuration to a generic RNN with hidden states but effectively no "layers." We will pick up next time with backpropagation-through-time (BPTT), Long Short Term Memory (LSTM), reservoir computing (Echo State Networks, ESN's), and an introduction to reinforcement learning. Interactive demonstration widgets related to this lecture can be found at:

Toward Multimodal AI: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/transformers/toward_multimodal_AI.html
RNN Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/recurrent_neural_networks/rnn_explorer.html

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/pi8vxjrn6gbftpdab977w/IEE598-Lecture7C-2026-04-16-Recurrent_Networks_and_Temporal_Supervision-Notes.pdf?rlkey=1xltyg1ttcpyqhvtdquczxjr5&dl=0

Tuesday, April 14, 2026

Lecture 7B (2026-04-14): Feeding Forward from Neurons to Networks (SLP, RBFNN, MLP, and CNN)

In this lecture, we move from the basics of learning foundations from the last lecture into models of neurons that can be combined to form machine learning tools. We start with the single-layer perceptron (SLP), explain where the term "weights" comes, and describe how it can linearly separate a space. We then introduce a hidden layer of receptive field units (RFU's) and discuss how Radial Basis Function Neural Networks use Gaussian or Logistic RBF's as nonlinear projections into high-dimensional space that Cover's theorem suggests should be more likely to e linearly separable. After demonstrating how RBFNN's work, we then introduce Cybenko's Universal Approximation Theorem (UAT) and use it to motivate looking for other (and deeper) latent structures. That leads us to the Multi-Layer Perceptron (MLP), backpropagation, and the Convolutional Neural Network.

Interactive widgets referenced in this lecture include:

Single Layer Perceptron: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/single_layer_perceptron/slp_explainer.html
Radial Basis Function Neural Network: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/radial_basis_function_nn/rbfnn_explorer.html
Toward Multimodal AI (for visualizing CNN receptive fields): https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/transformers/toward_multimodal_AI.html
Transformer Architecture Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/transformers/transformer_explorer.html

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/t2aoepucn0swlkvisococ/IEE598-Lecture7B-2026-04-14-Feeding_Forward_from_Neurons_to_Networks-Notes.pdf?rlkey=s5pr1zdrnup2ca1nthf7zxp3n&dl=0

Lecture 7A (2026-04-09): Neural Foundations of Learning

In this lecture, we prepare to discuss artificial and spiking neural networks -- bio-inspired information processing mechanisms inspired by the central nervous system and models of learning in psychology. We open with a discussion of the relationship between learning, memory, and neuroplasticity and then introduce a canonical model of a neuron that is the basis of the mechanisms thought to underly neuroplasticity. We discuss the different ways in which neuroplasticity supports working, short-term, and long-term memory. We introduce Hebbian learning (and briefly mention spike-timing-dependent plasticity, STDP) as a foundational learning paradigm that, when combined with neuromodluation and specialized circuits, can implement all forms of learning described in the lecture. Those forms of learning include non-associative learning (habituation and sensitization), associative learning (classical and operant conditioning), and latent learning. We map each of those to machine learing paradigms including unsupervised learning, self-supervised learning/pre-training, reinforcement learning, and supervised learning. In the next lecture, we will directly model the canonical neuron with a signle-layer perceptron and start to build statistical models based on this artificial neuron model. Interactive demonstrations mentioned in this video:

SLP: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/single_layer_perceptron/slp_explainer.html
Hebbian Learning: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/hebbian_learning/hebbian_competitive_clustering.html
Memristor-based STDP Learning: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/memristors/memristor_stdp_array.html

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/x4t0y6q9rblrn78o8ns2r/IEE598-Lecture7A-2026-04-09-Neural_Foundations_of_Learning-Notes.pdf?rlkey=im6unlrptbfppqeds2y9gpga7&dl=0

Sunday, April 5, 2026

Lecture 6B (2026-04-07): Bacterial Foraging Optimization and Ant Colony Optimization

Closing out the Swarm Intelligence unit, this lecture pivots from Particle Swarm Optimization (PSO) to two examples of stigmergic swarm optimization – Bacterial Foraging Optimization (BFO) and Ant Colony Optimization (ACO). Stigmergy is the act of indirection through modifications of the environment, as in leaving chemical trails or depositing chemical gradients, as opposed to direct communication between one individual and another. BFO solves continuous optimization problems similar to PSO but uses attractants and repellants to modify the environment as opposed to directly informing others of information about discovered solutions. The repellants in BFO along with its reproduction and elimination–dispersal phases help to ensure it searches globally over a space as opposed to the more concentrated search of PSO. ACO also uses chemical coordination, but it is developed for combinatorial optimization problems. Although ACO was originally developed for the Traveling Salesman Problem (TSP), we discuss ACO first in a simpler layered model that better matches the foraging paths of real ants before briefly discussing the application to the TSP. We close with a brief mention of more complex recruitment dynamics in real ants, where trail laying plus noise can provide the ability to track changing feeder distributions and how one-on-one recruitment by some ants and bees can lead to different distributions of recruits across options (similar to changing the temperature in a softmax).

Interactive demonstrations referenced in this lecture can be found at:

Particle Swarm Optimization: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/particle_swarm_optimization/pso_explorer.html
Bacterial Foraging Optimization: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/bacterial_foraging_optimization/bfo_explorer.html
Ant Colony Optimization: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/ant_colony_optimization/aco_explorer.html
Case Study for More Realistic Ant Recruitment Dynamics: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/collective_behavior/ant_foraging_explorer.html
Softmax Exploration: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/softmax/softmax_temperature_explorer.html

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/scl/fi/fqm4jcfr1mkxsnz8ng61r/IEE598-Lecture6B-2026-04-07-Bacterial_Foraging_Optimization_and_Ant_Colony_Optimization-Notes.pdf?rlkey=q4omc6oyot9vrq8nnq3etx6k4&dl=0

Thursday, April 2, 2026

Lecture 5E/6A (2026-04-04): Parallel Tempering and Swarm Intelligence through Social Cohesion (Particle Swarm Optimization)

In this lecture, we finish our unit on physics-inspired ML and optimization by covering Parallel Tempering (PT), which combines multiple, parallel Metropolis–Hastings MCMC samplers each with different temperatures (rather than using an annealing schedule, as in Simulated Annealing (SA)). We then pivot toward motivating why certain problem sets, like optimizing high-dimensional weights of neural networks, may not be well suited by the optimization metaheuristics discussed so far in the course. We use this as an opportunity to introduce Swarm Intelligence and the Particle Swarm Optimization (PSO) algorithm, which is particularly good at finding and exploring local optima in spaces with many similarly performing local optima. We explore how PSO was inspired by the Boids Model from Craig Reynolds (in computer graphics) and how it overlaps with the Vicsek model (from statistical physics). We also show how PSO really depends on is social information but, under the influence of social information, tends to very quickly purge the diversity in its solution candidates. Online interactive demonstration modules associated with this lecture can be found at: