Lecture 8C: Elementary Cellular Automata, Stochastic Cellular Automata, Ising models, and Cellular Evolutionary Algorithms (2020-04-29)

In this lecture, we close out the semester's technical content with some final words about cellular automata (CA) and interacting particle systems (IPS). We keep the focus from the previous lecture on Elementary CA's (ECA's), reminding of the naming convention and a mention of some popular rules (such as rule 30, rule 110, rule 184, rule 232). We then pivot to stochastic/probabilistic CA's, with a focus on 1-D stochastic ECA and show how probabilistic rules do not only generate "fuzzy patterns." This allowed us to talk about Ising models and physical/hardware implementations of annealing machines for high-speed combinatorial optimization by metaheuristics. That closes some of the loop between the end of this class and earlier in the class. To fully close that loop, we close with an introduction to Cellular Evolutionary Algorithms (cEA's), which define genetic operators on spatially defined neighborhoods in a way clearly inspired by CA's (and these evolutionary algorithms can certainly be viewed as a kind of CA).

Whiteboard notes for this lecture can be found at:
https://www.dropbox.com/s/iwrwe59u5xt2y0v/IEE598-Lecture8C-Notes.pdf?dl=0

Lecture 8B: Elementary Cellular Automata, Computation, and Experimentation (2020-04-27)

In this lecture, we continue discussing Interacting Particle Systems (IPS) and complex systems approaches for understanding adaptive properties of distributed systems, such as neural networks. We specifically focus on cellular automata (CA), starting with 2D cases (such as models of fire spread, the Game of Life, and "Brian's Brain" from NetLogo) and then moving to Elementary Cellular Automata (ECA). We discuss some popular ECA rules, such as the traffic rule (184) and the majority rule (rule 232) and how they can be combined to do density classification. We also discuss how rule 110 is Turing complete and thus can do much more.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/sexl774qrv0zgd1/IEE598-Lecture8B-Notes.pdf?dl=0

Lecture 8A: Complex Systems Approaches to Computation - Interacting Particle Systems and the Voter Model (2020-04-22)

In this lecture, we introduce thinking about computation from a complex systems perspective. This transition is motivated by thinking about physiological details of neuron-to-neuron communication (e.g., different synapse types, a variety of neurotransmitters, neuromodulatory feedback from other parts of the brain, neuropharmacology) are much more well understood at the local level than they are at the level of emergent cognitive phenomena, and none of these details are currently being incorporated into neuromorphic approaches to artificial cognition. Complex systems methods (which are methods applied to systems where local-level interactions are easy to describe but large-scale, system-level phenomena emerge non-trivially from those interactions) provide one perspective to better understanding such emergence.

We then introduce Interacting Particle Systems (IPS), for which cellular automata (and both artificial neural networks, ANNs, and spiking neural networks, SNNs) can be viewed as a special case. Before jumping into cellular automata in the next lecture, we close this lecture discussing the "voter model", which is a model of neutral evolution (i.e., evolution by drift alone with no selective/fitness effects). Asking whether the voter model reaches fixation is equivalent to asking whether the computational system of interacting particles reach consensus. The answer to this requires using a dual, time-reversed system of coalescing Markov chains akin to doing contact tracing in an epidemiological system. The result (making use of Polya's recurrence theorem) is that consensus (fixation) will occur for 1 or 2 dimensions but will not always occur (i.e., will fail with non-zero probability) in 3 dimensions or higher.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/x2a7xmlsprwn9eq/IEE598-Lecture8A-Notes.pdf?dl=0

Lecture 7H: Associative Learning, Spiking Neural Networks, and Alternatives to Backpropagation (2020-04-20)

In this lecture, we start with a review of this unit on Artificial Neural Networks, starting with the original inspiration from Otto Schmitt's work on the squid neuron (inspiring switches with hysteresis, like the Schmitt trigger). We revisit the basic model of an artificial neural as a generalized linear model (GLM) for regression and classification and discuss how the receptive fields and input layers of radial basis function neural networks (RBFNN)'s transform data so that a neuron built for linear separability is able to perform nonlinear classification as well. That lets us pivot into revisiting deep neural networks (such as the multi-layer perceptron, MLP) and forms like the convolutional neural network (CNN) that also make use of receptive fields to simplify the downstream learning. We then remind ourselves of the recurrent neural network (RNN) and more exotic versions like reservoir machines (echo state networks, ESNs) that use randomly connected recurrent neural networks as filters that allow downstream neurons to learn to classify patterns in time (not just space). After being reminded of backpropagation for the training of all of these networks, we discuss how associative learning (specifically Hebbian learning and spike-timing-dependent plasticity, STDP) can be viewed as an alternative to backpropagation for training. This requires us to discuss how psychologists view associative learning as able to generate both classical and operant conditioning, which can then be used to implement supervised, unsupervised, and reinforcement learning. This lets us discuss a new result from Google, AutoML-Zero, that evolves (using genetic programming, GP, genprog) novel ML algorithms that themselves instantiate new neural networks. AutoML-Zero has "discovered" backpropagation on its own, but it has also found other ways to train weights. We end with a discussion of the hardware implementation from Boyn et al. (2017, Nature Communications) of a crossbar array of memristors that can do unsupervised pattern recognition through STDP (no backpropagation required).

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/q61u6sa2e5oxwc3/IEE598-Lecture7H-Notes.pdf?dl=0

Lecture 7G: Intro. to Spiking Neural Networks and Neuromorphic Computing (2020-04-15)

In this lecture, we continue discussing associative/Hebbian learning in neural networks – starting with the inspiration from real neurons and leading to the approximation of this learning mechanism in Artificial Neural Networks (ANN). We discuss a simple pattern-recognition example that is well suited for spike-timing-dependent plasticity (STDP). We then setup the foundations for discussing true spiking neural networks (SNN), introducing some of the existing platforms (IBM TrueNorth, Intel Loihi, and SpiNNaker). We will start the next lecture with a discussion of memristor-based crossbar arrays that implement STDP, as first described in Boyn et al. (2017) in Nature Communications.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/g3e73roz5v5h31p/IEE598-Lecture7G-Notes.pdf?dl=0

Lecture 7F: Unsupervised Learning, Multidimensional Scaling, & Hebbian/Associative Learning with Artificial Neural Networks (2020-04-13)

In this lecture, we continue our discussion of unsupervised learning methods with artificial neural networks (ANN's), with reminders of clustering, anomaly detection, and generative adversarial networks (GANs) – all of which represent a form of computational creativity. We then pivot to review the basics of principal component analysis (PCA) as an example method for multi-dimensional scaling (MDS). That allows us to introduce the autoencoder ("self encoder") feed-forward network, which is an example of using methods from supervised learning to generate an unsupervised learning process that implements nonlinear multidimensional scaling. We close the lecture with an introduction to Hebbian/associative learning (and motivations from spike-timing-dependent plasticity, STDP) and how Hebbian update rules can be applied to conventional artificial neural networks for pattern learning.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/oy4k95g67kv6ju6/IEE598-Lecture7F-Notes.pdf?dl=0

Lecture 7E: Reinforcement and Unsupervised Learning (2020-04-08)

In this lecture, we continue our investigation of Artificial Neural Networks (ANN), but we move from supervised learning through reinforcement learning (RL) to an introduction to unsupervised learning. We pick up where we left off with reservoir machines (e.g., Echo-State Networks, ESN's), which reduce the amount of training necessary for a complex supervised learning task. We then pivot to talk about reinforcement learning and introduce Q-learning (specifically deep Q-learning). We then close with an introduction to unsupervised learning, with a brief discussion of clustering, anomaly detection, and computational creativity provided by generative adversarial networks (GAN). This will set us up for multidimensional scaling (MDS) in the next lecture and a march toward Hebbian learning, spike-time-dependent plasticity, and spiking neural networks (SNN's).

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/aizw77vqsra6kep/IEE598-Lecture7E-Notes.pdf?dl=0

Lecture 7D: More Complex Neural Networks and their Training (2020-04-06)

In this lecture, we continue our review of artificial neural networks (ANN's) and some of the nature-inspired directions that inspired them. We move from feed-forward multi-layer perceptrons to more general topologies that skip layers or add recurrent connections (as in Recurrent Neural Networks, RNN's). This provides a brief introduction to Time Delayed Neural Networks (TDNN) and an opportunity to make connections between Finite Impulse Response (FIR) and Infinite Impulse Response (IIR) filtering from linear systems theory and signal processing. We then discuss how a generalized version of backpropagation, known as backpropagation thru time (BPTT) can be used to train neural networks in much the same was as multi-layered perceptrons. We then discuss a specialized RNN known as Long Short-Term Memory (LSTM) and its motivations, and we close with a brief introduction to reservoir computing ("reservoir machines", "Echo-State Networks", ESN's), which only require a small subset of the total network to be trained and yet can do a wide variety of the tasks that other "fully trained" RNN's can do.

Whiteboard notes for this lecture can be found at: https://www.dropbox.com/s/pa5ju9pxja38zxv/IEE598-Lecture7D-Notes.pdf?dl=0

Lecture 7C: Deep Neural Networks (backpropagation, CNNs, & insect brains) (2020-04-01)

In this lecture, we continue our introduction of supervised learning for feed-forward neural networks. We start with a re-introduction of convolutional neural networks (CNN's) and their loose inspiration from the vertebrate brain's visual cortex and cast them as a special case of multi-layer perceptrons (MLP's). We then continue our explanation of backpropagation as a method of using (stochastic) gradient descent (SGD) to train neural networks in supervised learning. We then pivot to discuss alternative deep neural network topologies potentially inspired by the architecture of other natural systems, such as the insect brain. We introduce the major neuropils of the insect brain (optic lobe, antennal lobe (and glomeruli), lateral horn, mushroom bodies (and Kenyon cells), and central complex) and the various interneurons that connect locally within and project among them. This allows us to portray the apparently random projections between the antennal lobe's glomeruli to the Kenyon cells as a kind of hashing function. The lecture ends with a discussion of the utility of thinking of conventional ANN's as being nature inspired.

Whiteboard notes for this lecture can be found at:

https://www.dropbox.com/s/t9lzfpf7d59j3yx/IEE598-Lecture7C-Notes.pdf?dl=0