Lecture 7B (2026-04-14): Feeding Forward from Neurons to Networks (SLP, RBFNN, MLP, and CNN)

In this lecture, we move from the basics of learning foundations from the last lecture into models of neurons that can be combined to form machine learning tools. We start with the single-layer perceptron (SLP), explain where the term "weights" comes, and describe how it can linearly separate a space. We then introduce a hidden layer of receptive field units (RFU's) and discuss how Radial Basis Function Neural Networks use Gaussian or Logistic RBF's as nonlinear projections into high-dimensional space that Cover's theorem suggests should be more likely to e linearly separable. After demonstrating how RBFNN's work, we then introduce Cybenko's Universal Approximation Theorem (UAT) and use it to motivate looking for other (and deeper) latent structures. That leads us to the Multi-Layer Perceptron (MLP), backpropagation, and the Convolutional Neural Network.

Interactive widgets referenced in this lecture include:

• Single Layer Perceptron: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/single_layer_perceptron/slp_explainer.html
• Radial Basis Function Neural Network: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/radial_basis_function_nn/rbfnn_explorer.html
• Toward Multimodal AI (for visualizing CNN receptive fields): https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/transformers/toward_multimodal_AI.html
• Transformer Architecture Explorer: https://tpavlic.github.io/asu-bioinspired-ai-and-optimization/transformers/transformer_explorer.html

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/scl/fi/t2aoepucn0swlkvisococ/IEE598-Lecture7B-2026-04-14-Feeding_Forward_from_Neurons_to_Networks-Notes.pdf?rlkey=s5pr1zdrnup2ca1nthf7zxp3n&dl=0