Deep Learning Discussion: Backpropagaton

For Next Week:

• Next week will be a “catch-up” session since the backpropagation material is critical for understanding how networks are optimized and takes a fair bit of time to work through.
• Time permitting we may introduce the bigram model or dig deeper into dataloaders depending on the length of the discussion next week.

Materials

• Here is the notebook I wrote during our meeting today (Demo Notebook) showing how how gradients are tracked in PyTorch and how to create a small multilayer perceptron (MLP).