Course Websites

ECE 563 - Information Theory

Last offered Fall 2021

Official Description

Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, and rate-distortion theory. Course Information: Prerequisite: One of ECE 534, MATH 464, MATH 564.

Subject Area

• Communications

Description

Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, rate-distortion theory.

Notes

Same as: CS 578 and STAT 563

Topics

• Entropy, relative entropy, mutual information
• Asymptotic equipartition property
• Entropy rates of a stochastic process
• Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
• Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
• Differential entropy
• Gaussian channels
• Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion

Detailed Description and Outline

Topics:

• Entropy, relative entropy, mutual information
• Asymptotic equipartition property
• Entropy rates of a stochastic process
• Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
• Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
• Differential entropy
• Gaussian channels
• Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion

Same as: CS 578 and STAT 563

Texts

T. Cover and J. Thomas, Elements of Information Theory, Wiley, 1991.
TitleSectionCRNTypeHoursTimesDaysLocationInstructor
Information TheoryA37140DIS41230 - 1350 T R  3081 Electrical & Computer Eng Bldg Ilan Shomorony
Information TheoryON174009OD41230 - 1350 T R    Ilan Shomorony
Seiyun Shin