ECE 498MR: Introduction to Stochastic Systems
(Spring 2017)

About  |  Schedule  |  Coursework


The schedule will be updated and revised as the course progresses. To get a rough idea of what to expect, consult the course syllabus. Required reading in the course notes is indicated on the left; key topics will be highlighted.
Wed, Jan 18

Introduction and administrivia

  • What are stochastic signals and systems?
  • Noise, uncertainty, randomness
  • Descriptions of stochastic systems: declarative vs. imperative
Mon, Jan 23

Probability review

  • Axioms of probability theory: sample space, events, probability measures
  • Conditioning and independence
  • Random variables: discrete and continuous
  • Cumulative distribution functions, probability mass functions, probability density functions
  • Examples of discrete and continuous r.v.'s
Wed, Jan 25
Mon, Jan 30
Wed, Feb 1

First look: Markov chains as stochastic systems

  • Systems with state: deterministic and stochastic
  • Markov chains as noise-driven systems with state
  • Motivating examples: simple random walk on the integers, two-state Markov chain
  • Descriptions of Markov chains: imperative and declarative
  • Markov chains as linear systems in the space of probabilities: discrete-time Fourier transforms and matrix multiplication
  • Equilibrium distributions and the PageRank algorithm
Mon, Feb 6
Wed, Feb 8
Mon, Feb 13

Random signals and probabilistic systems

  • Random processes as signals (in discrete and continuous time)
  • Random walks: independent increments, Markov property
  • Wiener and Poisson processes as continuous-time limits of random walks
  • Stationarity: strong and weak
Wed, Feb 15

Recitation: Homeworks 1 and 2

Mon, Feb 20
Wed, Feb 22
Mon, Feb 27
Wed, Mar 1
Mon, Mar 6
Wed, Mar 8

Stochastic signal processing

  • Stationarity: weak and strong
  • Moments, auto- and cross-correlation in time and frequency domain; power spectral density; white and colored noise
  • Gaussian random vectors and Gaussian stochastic signals; Bussgang's theorem
  • Poisson point processes and Campbell's theorem
  • Basic analysis of convergence and stability via Lyapunov (or potential) functions
  • Case studies: average consensus and PageRank revisited
Mon, Mar 13
Wed, Mar 15
In-Class Midterm
Mon, Mar 20
Wed, Mar 22
No class: Spring Break

Mon, Mar 27
Wed, Mar 29
Mon, Apr 3
Wed, Apr 5
  • Noise mechanisms in physical systems: shot noise, Johnson-Nyquist noise, van der Ziel (1/f) noise, amplifier noise.

Mon, Apr 10
Wed, Apr 12
Mon, Apr 17


  • Dynamical view: evolution of uncertainty and information in time.
  • Bayesian filtering and Hidden Markov Models.
  • Kalman filter (details in the homework).
Wed, Apr 19
Mon, Apr 24
Wed, Apr 26
Randomness and determinism
  • Law of Large Numbers and the Central Limit Theorem through the lens of linear systems.
  • Variance reduction by averaging (examples: invention of least squares; diversification in financial portfolios following Harry Markowitz; Monte Carlo simulation).
  • Large-deviation bounds via the Chernoff technique and Taylor series (example: probabilistic interpretation of multiplexing gain in telephony).
Mon, May 1
Wed, May 3
Feedback and control
  • Controlled Markov chains: imperative and declarative descriptions
  • Finite-horizon optimal control problem
  • Blackwell's principle of irrelevant information and dynamic programming