Advanced Topics in Digital Signal Processing

Topics

Signal representations in vector spaces

1. • Introduction to discretizing signals using a basis: The Shannon-Nyquist sampling theorem
   • Linear vector spaces, linear independence, and basis expansions
   • Norms and inner products
   • Orthobases and the reproducing formula
   • Parseval’s theorem and the general discretization principle
   • Important bases: Fourier, discrete cosine, lapped orthogonal, splines, wavelets
   • Signal approximation in an inner product space
   • Gram-Schmidt and the QR decomposition

Linear inverse problems

1. Introduction to linear inverse problems, examples
2. The singular value decomposition (SVD)
3. Least-squares solutions to inverse problems and the pseudo-inverse
4. Stable inversion and regularization
5. Weighted least-squares and linear estimation
6. Least-squares with linear constraints

Matrix approximation using least-squares

1. Low-rank approximation of matrices using the SVD
2. Total least-squares
3. Principal components analysis
4. Signal and noise subspaces in array processing

Computing the solutions to least-sqaures problems

1. Cholesky and LU decomposition
2. Structured matrices: Toeplitz, diagonal+low rank, banded systems
3. Large-scale systems: Steepest descent
4. Large-scale systems: The conjugate gradient method

Low-rank updates for streaming solutions to least-squares problems

1. Recursive least-squares
2. The Kalman filter
3. Adaptive filtering using LMS