Welcome to Neural Networks and Deep Learning 2025 (NNDL-2025)

(last update 4/24/2025)

This is the landing page for the course on Neural Networks and Deep Learning held at William & Mary during Spring 2025

Important

The lectures will take place at the Integrated Science Center, room 1291

Synopsis: Neural networks and deep learning have revolutionized the field of artificial intelligence, enabling breakthroughs in image recognition, natural language processing, and more. This course will explore the foundational concepts and practical applications of neural networks, covering how to build them from scratch as well as utilizing popular deep learning libraries. Students will delve into various architectures, including Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), Graph Neural Networks (GNN), and more recent advancements. By the end of the course, participants will have hands-on experience and a deep understanding of both the theory and practice of neural networks and deep learning.

This course is based on the following references: [BGC17, KK22, RLM22]

Syllabus

• Syllabus (last update 1/23/2025)

Schedule

• Schedule

Assignments

• Assignment 1, Deep Neural Networks from Scratch
• Solution to Assignment 1, by Steven Morse
• Assignment 2, Convolutional Neural Networks
• Solution to Assignment 2, by Kaibu Feng
• Assignment 3, Generative Adversarial Networks
• Solution to Assignment 3, by Tyson Walsh
• Assignment 4, Graph Neural Networks
• Solution to Assignment 4, by Kaibu Feng

Intro

• Welcome and Intro by Prof. C.Fanelli

[Pre-flight] Intro to Modeling

• Linear Regression (lecture slides)
• 1. Linear Regression (Jupyter Notebook)

[Pre-flight] Introduction to git

• 2. Brief course on git
  • 2.1. Version Control System (VCS)
  • 2.2. What is Git?
  • 2.3. Configuring your git
• 3. Exercises on git
  • 3.1. Exercise 1: Basic Git Commands
  • 3.2. Exercise 2: Commits, push and branches
  • 3.3. Advanced exercises
  • 3.4. GitHub Actions and Workflow Tutorial

[Pre-flight] Using VS-Code

• Installing Visual Studio Code
• Installing Extensions
• Signing up and Installing GitHub Copilot

[Pre-flight] Introduction to High Performance Computing (HPC)

• 4. Introduction to High Performance Computing
  • 4.1. High Performance Computing
  • 4.2. SSH into a Front-End Node Using Proxy Jump
  • 4.3. William & Mary Research Computing: File Systems Tutorial
  • 4.4. Introduction to SLURM
• Other Resources: Distributed ML @ W&M
• Other Resources: Student Manual for using the HPC @ W&M for Data Science

[Pre-flight] Linux Commands

• Linux Commands

[1/28/2025] Linear Classifiers

• Perceptrons (lecture slides)
• Perceptron - Proof of Convergence (PDF)
• 5. Training a Perceptron (Jupyter notebook)
• Adaline (lecture slides)
• 6. Training Adaline (Jupyter notebook)

[1/30/2025] Gradient Descent, stochastic GD, Logistic Regression and Other classifiers

• Stochastic Gradient Descent (lecture slides)
• 7. Gradient Descent, Stochastic Gradient Descent, Mini-batch GD (Jupyter notebook)
• An Intro to Logistic Regression, Regularization, linear (and non-linear) boundaries (lecture slides)
• 8. Logistic Regression, Regularization, linear (and non-linear) boundaries (Preview)

[2/4/2025] Building Training Datasets

• Building Training Datasets (lecture slides)
• 9. Data Preprocessing (Jupyter notebook)

[2/6/2025] Regularization (Continued, see 1/30/2025)

• 10. Logistic Regression, Regularization, linear (and non-linear) boundaries (Preview)

[2/11/2025] Model Evaluation, Hyperparameter Tuning: Examples and Discussion

• 11. Model Evaluation, Hyperparameter Tuning (Jupyter notebook)

[2/13/2025] Building Multi-Layer Neural Networks from scratch + discussion on assignment

• Adding hidden layers + Math of Backprop (lecture slides)
• 12. Multi-Layer Perceptron (Jupyter notebook)

[2/18/2024] Building Multi-Layer Neural Networks with PyTorch

• 13. Using PyTorch preflight (Jupyter notebook)
• 14. Using torch.nn (Jupyter notebook)

[2/25 - 2/27 2025] PyTorch and Autograd

• 15. Using PyTorch to build Neural Networks (continue)
• 16. Autograd in PyTorch
• 17. Autograd with PyTorch (continue)

[3/4 - 3/6 2025] CNN with PyTorch

• 18. Using PyTorch with CNN
• 19. Using CNN and Celeb Dataset
• (A really nice and well-known resource:) Interactive Node-Link Visualization of Convolutional Neural Networks by Adam Harley

[3/18/2025] Explainable AI using Grad-CAM for visual explanations

• 20. Explainable AI (XAI) and Grad-CAM

[3/20/2025] CNN: from Classification to Regression Tasks

• 21. age prediciton based on face images

[3/25 - 3/26 - 4/1 - 2025] Recurrent Neural Network

• Introduction to Sequential Data and Time Series Data; Intro to RNN
• 22. LSTM Recurrent Neural Networks (updated)

[4/3 - 4/8 - 4/10 - 4/15 - 2025] Generative Adversarial Networks

• Introduction to GAN
• 23. Vanilla GAN Notebook
• 24. Deep Convolutional Generative Adversarial Network (DCGAN)
• 25. Wasserstein GAN (WGAN)
• 26. Conditional DCGAN (cDCGAN)

[4/15 - 4/17 - 4/22 - 4/24 2025] Graph Neural Networks

• 27. An Intorduction to GNN (part1)
• 28. GNN Notebook (part2)

Additional resources

• References

Credits: Lectures on how to use git, VS-Code, and HPC prepared by K. Suresh