First look: Markov chains as stochastic systems
- Systems with state: deterministic and stochastic
- Markov chains as noise-driven systems with state
- Motivating examples: simple random walk on the integers, two-state Markov chain
- Descriptions of Markov chains: imperative and declarative
- Markov chains as linear systems in the space of probabilities: discrete-time Fourier transforms and matrix multiplication
- Equilibrium distributions and the PageRank algorithm