ECE 498MR: Introduction to Stochastic Systems(Spring 2017)

Schedule

The schedule will be updated and revised as the course progresses. To get a rough idea of what to expect, consult the course syllabus. Required reading in the course notes is indicated on the left; key topics will be highlighted.
Wed, Jan 18
[notes]

• What are stochastic signals and systems?
• Noise, uncertainty, randomness
• Descriptions of stochastic systems: declarative vs. imperative
Mon, Jan 23
[.ipynb]

Probability review

• Axioms of probability theory: sample space, events, probability measures
• Conditioning and independence
• Random variables: discrete and continuous
• Cumulative distribution functions, probability mass functions, probability density functions
• Examples of discrete and continuous r.v.'s
Wed, Jan 25
Mon, Jan 30
Wed, Feb 1
[notes]

First look: Markov chains as stochastic systems

• Systems with state: deterministic and stochastic
• Markov chains as noise-driven systems with state
• Motivating examples: simple random walk on the integers, two-state Markov chain
• Descriptions of Markov chains: imperative and declarative
• Markov chains as linear systems in the space of probabilities: discrete-time Fourier transforms and matrix multiplication
• Equilibrium distributions and the PageRank algorithm
Mon, Feb 6
Wed, Feb 8
Mon, Feb 13
[notes]

Random signals and probabilistic systems

• Random processes as signals (in discrete and continuous time)
• Random walks: independent increments, Markov property
• Wiener and Poisson processes as continuous-time limits of random walks
• Stationarity: strong and weak
Wed, Feb 15

Recitation: Homeworks 1 and 2

Mon, Feb 20
Wed, Feb 22
Mon, Feb 27
Wed, Mar 1
Mon, Mar 6
Wed, Mar 8
[notes]

Stochastic signal processing

• Stationarity: weak and strong
• Moments, auto- and cross-correlation in time and frequency domain; power spectral density; white and colored noise
• Gaussian random vectors and Gaussian stochastic signals; Bussgang's theorem
• Poisson point processes and Campbell's theorem
• Basic analysis of convergence and stability via Lyapunov (or potential) functions
• Case studies: average consensus and PageRank revisited
Mon, Mar 13
Review
Wed, Mar 15
In-Class Midterm
Mon, Mar 20
Wed, Mar 22
No class: Spring Break

Mon, Mar 27
Wed, Mar 29
Mon, Apr 3
Wed, Apr 5
[notes]
Noise
• Noise mechanisms in physical systems: shot noise, Johnson-Nyquist noise, van der Ziel (1/f) noise, amplifier noise.

Mon, Apr 10
Wed, Apr 12
Mon, Apr 17
[notes]

Uncertainty

• Dynamical view: evolution of uncertainty and information in time.
• Bayesian filtering and Hidden Markov Models.
• Kalman filter (details in the homework).
Wed, Apr 19
Mon, Apr 24
Wed, Apr 26
[notes]
Randomness and determinism
• Law of Large Numbers and the Central Limit Theorem through the lens of linear systems.
• Variance reduction by averaging (examples: invention of least squares; diversification in financial portfolios following Harry Markowitz; Monte Carlo simulation).
• Large-deviation bounds via the Chernoff technique and Taylor series (example: probabilistic interpretation of multiplexing gain in telephony).
Mon, May 1
Wed, May 3
[notes]
Feedback and control
• Controlled Markov chains: imperative and declarative descriptions
• Finite-horizon optimal control problem
• Blackwell's principle of irrelevant information and dynamic programming