ECE 563 - Information Theory (Fall 2021)

Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
Lectures: Tuesday and Thursday, 12:30PM - 1:50PM, ECEB 3081 (lecture videos available on echo360)
Instructor: Ilan Shomorony, ilans@illinois.edu. Office hours (via Zoom): Monday 10:30am-12:00pm (info)

TA: Seiyun Shin, seiyuns2@illinois.edu. Office hours (via Zoom): Wednesday 4:00pm-5:00pm (info)
Problem solving sessions: Biweekly on Friday 4:00pm-5:00pm via Zoom (info)
Recorded videos of the previous problem-solving sessions can be found here


Prerequisites: Solid background in probability (ECE 534, MATH 464, or MATH 564).
Textbook: T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006.

Announcements

  • No Office Hours on Monday, October 25.
  • Solutions to Midterm 1 can be checked here
  • There will be no lecture on Tuesday, October 5.
  • Midterm 1 information:
    • Time: Tuesday, October 5, 7pm-8:50pm (via Zoom)
    • You will be allowed one 8.5 x 11” sheets of handwritten notes. No calculators or other electronic devices will be allowed.
    • The exam will cover everything until Gaussian channels.
  • First problem solving session will be on Friday, 09/17.
  • You can access the class notes here.
  • Lecture videos will be available at echo360.

Main topics

  • Information measures and fundamental inequalities
  • Typicality
  • Channel Capacity and the Channel Coding Theorem
  • Gaussian channels
  • Lossless compression
  • Huffman coding
  • Universal compression of i.i.d. and Markov sources
  • Lempel-Ziv coding
  • Rate-distortion theory
  • Large Deviation Theory and Hypothesis Testing
  • Topics in multi-user information theory

Grading

  • Homework 15%
  • Exam 1 - 20% (10/05 7pm)
  • Exam 2 - 20% (tentative time: 11/09 7pm)
  • Project - 10%
  • Final Exam - 35%