Welcome to DATA621 - Neural Networks and Deep Learning

(last update 12/3/2024)

This is the landing page for the course on Neural Networks and Deep Learning held at William & Mary during Fall 2024

Important

The lectures will take place at the Integrated Science Center, room 2291

Synopsis: Neural networks and deep learning have revolutionized the field of artificial intelligence, enabling breakthroughs in image recognition, natural language processing, and more. This course will explore the foundational concepts and practical applications of neural networks, covering how to build them from scratch as well as utilizing popular deep learning libraries. Students will delve into various architectures, including Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), Graph Neural Networks (GNN), and more recent advancements. By the end of the course, participants will have hands-on experience and a deep understanding of both the theory and practice of neural networks and deep learning.

This course is based on the following references: [BGC17, KK22, RLM22]

Syllabus

• Syllabus (last update 10/6/2024)

Schedule

• Schedule

Assignments

• Assignment 1, Deep Neural Networks from Scratch
• Assignment 2, Convolutional Neural Networks
• Assignment 3, Generative Adversarial Networks
• Assignment 4, Graph Neural Networks

Intro

• Welcome and Intro by Prof. C.Fanelli

[Pre-flight] Intro to Modeling

• Linear Regression (lecture slides)
• 1. Linear Regression (Jupyter Notebook)

[Pre-flight] Introduction to git

• 2. Brief course on git
  • 2.1. Version Control System (VCS)
  • 2.2. What is Git?
  • 2.3. Configuring your git
• 3. Exercises on git
  • 3.1. Exercise 1: Basic Git Commands
  • 3.2. Exercise 2: Commits, push and branches
  • 3.3. Advanced exercises
  • 3.4. GitHub Actions and Workflow Tutorial

[Pre-flight] Using VS-Code

• Installing Visual Studio Code
• Installing Extensions
• Signing up and Installing GitHub Copilot

[8/29/2024] Introduction to High Performance Computing (HPC)

• 4. Introduction to High Performance Computing
  • 4.1. High Performance Computing
  • 4.2. SSH into a Front-End Node Using Proxy Jump
  • 4.3. William & Mary Research Computing: File Systems Tutorial
  • 4.4. Introduction to SLURM

[9/3/2024] Linear Classifiers

• Perceptrons (lecture slides)
• Perceptron - Proof of Convergence (PDF)
• 5. Training a Perceptron (Jupyter notebook)
• Adaline (lecture slides)
• 6. Training Adaline (Jupyter notebook)

[9/5/2024] Gradient Descent, stochastic GD, Logistic Regression and Other classifiers

• Stochastic Gradient Descent, Logistic Regression (lecture slides)
• 7. Gradient Descent, Stochastic Gradient Descent, Mini-batch GD (Jupyter notebook)
• 8. Logistic Regression, Regularization, linear (and non-linear) boundaries

[9/10/2024] Building Training Datasets

• Building Training Datasets (lecture slides)
• 9. Data Preprocessing (Jupyter notebook)

[9/12/2024] Model Evaluation, Hyperparameter Tuning: Examples and Discussion

• 10. Model Evaluation, Hyperparameter Tuning (Jupyter notebook)

[9/17/2024, 9/19/2024] Building Multi-Layer Neural Networks from scratch + discussion on assignment

• Adding hidden layers + Math of Backprop (lecture slides)
• 11. Multi-Layer Perceptron (Jupyter notebook)

[9/24/2024] Building Multi-Layer Neural Networks with PyTorch

• 12. Using PyTorch preflight (Jupyter notebook)
• 13. Using torch.nn (Jupyter notebook)

[9/26/2024] PyTorch and Autograd

• 14. Using PyTorch Autograd (Jupyter notebook)

[10/1/2024, 10/3/2024, 10/8/2024] CNN with PyTorch

• 15. Using PyTorch with CNN
• 16. Using CNN and Celeb Dataset
• 17. Using CNN for Regression Tasks (age prediction based on images)
• 18. Using U-Net for Image Segmentation

[10/15/2024, 10/17/2024] Explainable AI using Grad-CAM for visual explanations

• 19. Introduction to Grad-CAM
• 20. Grad-CAM (part 2)

[10/22/2024, 10/24/2024] Recurrent Neural Network

• 21. 2024 Nobel Prize in Physics: Hopfield NN and Boltzmann Machines
• Introduction to Sequential Data and Time Series Data; Intro to RNN
• 22. LSTM Recurrent Neural Networks

[10/29/2024, 10/31/2024, 1/7/2024] Generative Adversarial Networks

• Introduction to GAN
• 23. GAN Notebook (part1)
• 24. DCGAN and Wasserstein GAN
• 25. Conditional GAN

[11/24/2024] Graph Neural Networks

• 26. GNN Notebook (part1)
• 27. GNN Notebook (part2)

[12/03/2024, 12/05/2024] Transformers

• 28. Introduction to Transformers

Additional resources

• References

Credits: Material on git, VS-Code, and HPC prepared by K. Suresh