ECE 563  Information Theory (Fall 2018)
Lecturer: Lav Varshney (office hours, Friday 9:3011:00am, 314 CSL and by appointment)
Teaching Assistants: Ravi Kiran Raman (office hours, Tuesday 4:005:00pm, 3036 ECE) and Sam Spencer (office hours, Thursday 2:303:30pm, 114 CSL)
Lectures: Tuesday and Thursday, 12:30pm, 2015 Electrical and Computer Engineering Building
Problem Solving Sessions: Friday, 2:00pm, 141 Coordinated Science Laboratory [optional]
Course Goals
Catalog Description
Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, and ratedistortion theory.
Prerequisites: Solid background in probability (ECE 534, MATH 464, or MATH 564).
Textbook: T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006.
Grading: Homework [including programming assignments] (25%), Midterm exam [in class] (25%), Final exam [as designated by university] (25%), Group juxtaposition paper [in groups of three, in roughly Allerton format] (25%)
Homework
Problem Solving Sessions
Exams
Juxtaposition Paper
Course Schedule
Date  Topic  Reading Assignment  Learning Objectives  Multimedia Supplements 
8/28 
1. The problem of communication, information theory beyond communication [slides] 


8/30 
2. The idea of errorcontrol coding and linear codes [slides] [handwritten] 


9/4  3. Information measures and their axiomatic derivation 



4. Basic inequalities with information measures 


9/11  5. Asymptotic Equipartition Property 



9/13  6. Source Coding Theorem 



9/18  7. Variablelength Codes 


9/20  8. Entropy Rate of Stochastic Processes 


9/25  9. Distributed Source Coding 


9/27  10. Universal Source Coding 



10/2  11. Method of Types 


10/4  12. Allerton Conference [no lecture]  
10/9  13. Hypothesis Testing 


10/11  14. Channel Coding Theorem: Converse and Joint AEP 



10/16  15. Channel Coding Theorem: Achievability and Examples 


10/18  16. Midterm [no lecture]  
10/23  17. SourceChannel Separation 


10/25  18. Differential Entropy, Maximum Entropy, and Capacity of RealValued Channels 


10/30  19. RateDistortion Theorem: Converse and Examples 



11/1  20. RateDistortion Theorem: Achievability and More Examples 


11/6  21. Quantization Theory 


11/8  22. BlahutArimoto 


11/13  23. Strong Data Processing Inequalities 


11/15  24. Large Deviations 


11/27  25. Error Exponents for Channel Coding 

Topics: