MAT2253

Applied Linear Algebra

Prerequisite: MAT1214/MAT1213 Calculus I

This comprehensive course in linear algebra provides an in-depth exploration of core concepts and their applications to optimization, data analysis, and neural networks. Students will gain a strong foundation in the fundamental notions of linear systems of equations, vectors, and matrices, as well as advanced topics such as eigenvalues, eigenvectors, and canonical solutions to linear systems of differential equations. The course also explores he critical techniques of calculus operations in vectors and matrices, optimization, and Taylor series in one and multiple variables. By the end of the course, students will have a thorough understanding of the mathematical framework underlying principal component analysis, gradient descent, and the implementation of simple neural networks.

The primary textbook is "Mathematics for Machine Learning" by Deisenroth, Faisal, and Ong, 2020, Cambridge University Press. The book is available for free for personal use at https://mml-book.github.io/book/mml-book.pdf

The secondary textbook is "Pattern Recognition and Machine Learning" by Bishop, 2006, Springer Information Science and Statistics. The book is available for free for personal use at https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

Session	Section	Topic	Prerequisites	SLOs
1	2.1	Systems of Linear Equations
2	2.2	Matrices
3	2.3	Solving systems of linear equations
4	2.4	Vector spaces
5	2.5	Linear Independence
6	2.6	Basis & Rank
7	Exam 1
8	2.7	Linear Mappings
9	2.7	Linear Mappings (examples)
10	3.1, 3.2, 3.3	Norms, Inner Products, Lengths & Distances
11	3.4	Angles & orthogonality
12	3.5	Orthonormal Basis
13	3.7	Inner Product of Functions
14	Project 1
15	4.1	Determinant and Traces
16	4.2	Eigenvalues & Eigenvectors
17	4.3, 4.4	Matrix Factorization
18	5.1	Vector Calculus Intro and Taylor Series
19	5.1, 5.2	Differentiation Rules Review and Partial Derivatives
20	5.2	Gradients- Examples, visualizations, computation
21	5.3	Gradients of Vector-Valued Functions
22	5.4, Dhrymes 78	Gradients of Matrices
23	Exam 2
24	5.5, Dhrymes 78	Useful Identities for Computing Gradients
25	5.3	Gradients of Vector-Valued Functions
26	5.4, Dhrymes 78	Gradients of Matrices
27	5.5, Dhrymes 78	Useful Identities for Computing Gradients
28	5.7	Higher-Order Derivatives
29	Notes	Minimization via Newton's Method & Backpropagation
30	Project 2
31	5.8	Multivariate Taylor Series
32	Notes	Linear optimization: Simplex method
33	7.1	Optimization Using Gradient Descent
34	7.2 and Notes	Constrained Optimization and Lagrange Multipliers: PCA
35	7.3	Convex Optimization (time permitting)
36	Bishop, Duda et al.	Feed-forward Artificial Neural Networks
37	Bishop, Duda et al.	Backpropagation in ANNs
38	Bishop, Duda et al.	Activation Functions: Linear & Nonlinear
39	Bishop, Duda et al.	Step-by-step simple ANN
40	Bishop, Duda et al.	Measures of performance
41	Bishop, Duda et al.	More complex architectures of ANNs
42	Final Project Introduction
43	Final project
44	Review