Matrix inversion: orthogonal projections; left and right inverses; minimum-norm least squares solutions; Moore-Penrose pseudoinverse; reularization; singular value decomposition; Eckart and Young theorem; total least squares; principal components analysis
Projections in Hilbert space: Hilbert space; projection theorem; normal equations, approximation and Fourier series; pseudoinverse operators, application to extrapolation of bandlimited sequences
Hilbert space of random variables: spectral representation of discrete-time stochastic processes; spectral factorization; linear minimum-variance estimation; discrete-time Wiener filter; innovations representation; Wold decomposition; Gauss Markov theorem; sequential least squares; discrete-time Kalman filter
Power spectrum estimation: system identification; Prony's linear prediction method; Fourier and other nonparametric methods of spectrum estimation; resolution limits and model based methods; autoregressive models and the maximum entropy method; Levinson's algorithm; lattice filters; harmonic retrieval by Pisarenko's method; direction finding with passive multi-sensor arrays
Reading
Class notes by Bresler, Basu and Couvreur (BBC) - Available on the lectures page