Disclaimer: This is still mostly the syllabus from previous years. While we will follow the general sequence of topics, you can expect the reading materials and specific lectures to change somewhat this year.

Required readings refer to chapters in Jurafsky and Martin (2008), Speech and Language Processing, 2nd edition, unless stated otherwise. Note that the 3rd edition is still in preparation, although the website has (currently) a number of new and/or rewritten chapters available as PDFs.

Optional readings are often more advanced. "MS" refers to chapters in Manning and Schütze (1999), Foundations of Statistical Natural Language Processing (you may need to use a campus machine to access the links to the chapters below) or to original research papers (you can find many more on the ACL anthology). I also recommend the Handbook of Computational Linguistics and Natural Language Processing (you also need to be on the campus network to access this site).

Lecture slides and Assignments are also linked to on this page. We typically release assignments in the evening of the day they come out.

Schedule

Wed, 08/29 01 Introduction 1up 4up
What is NLP? What will you learn in this class?
Required reading: Ch.1
Optional reading: Python tutorial (sec. 1-5), Jelinek (2009), Ferrucci et al. (2010)
 
Fri, 08/31 02 Finite-state methods for morphology 1up 4up MP0 out
What is the structure of words, and how can we model it? Review of finite-state automata. Finite-state transducers
Required reading: Ch.3.1-7 (2nd Ed.);
Optional reading: Karttunen and Beesley '05, Mohri (1997), the Porter stemmer, Sproat et al. (1996)
Other links:
 
Wed, 09/05 03 N-gram language models 1up 4up
The most basic probabilistic models of language. N-gram models. Evaluation. Also: review of basic probability
Required reading: Ch. 3.1-3 (3rd Ed.)
Optional reading: MS, Ch. 2
 
Fri, 09/07 04 Smoothing 1up 4up MP1 out
How can we predict what we haven't seen before?
Required reading: Ch. 3.4-8 (3rd Ed.)
Optional reading: MS, Ch.6, Chen and Goodman (1998)
 
Wed, 09/12 05 Part-of-speech tagging (I) 1up 4up
What are parts of speech? How many are there? Basic intro to HMMs.
Required reading: Ch. 8.1-4(3rd Edition)
Optional reading: Merialdo (1994), Christodoulopoulos et al. (2010), Roche & Schabes (1995)
 
Fri, 09/14 06 POS tagging (II) 1up 4up
Required reading: Ch. 8.1-4(3rd Edition)
Optional reading: MS, Ch. 9
 
Wed, 09/19 07 Sequence Labeling 1up 4up
Chunking, shallow parsing, named entity recognition
Required reading: Ch. 6.6-8 (2nd Ed)
Optional reading: Sutton & McCallum (2008) (Introduction to Conditional Random Fields), Berger et al. (1996), Etzioni et al. (2008) (web-scale information extraction)
 
Fri, 09/21 08 Grammars 1up 4up
What is the structure of sentences, and how can we model it? Phrase-structure grammar and dependency grammar. Review of basic English grammar and context-free grammars
Required reading: Ch. 10.1-3 (3rd Ed) or Ch. 12.1-3 and Ch. 12.7 (2nd Edition)
Optional reading: MS, Ch. 3, Woods (2010)
 
Wed, 09/26 09 Syntactic Parsing 1up 4up
PCFGs and Viterbi
Required reading: Ch. 11 (3rd Ed.) or Ch. 13.1-4, Ch. 14.1
Optional reading: Chi (1999)
 
Fri, 09/28 10 Syntactic Parsing 1up 4up MP1 due. MP2 out
Going beyond simple PCFGs; Penn Treebank parsing
Required reading: Ch 12 (3rd Edition) or Ch. 14.1-7, Ch. 12.4
Optional reading: Collins' notes, Chi & Geman (1998), Schabes et al. (1993), Schabes & Pereira (1992), Stolcke (1995), Marcus et al. (1993), Collins (1997), Johnson (1998), Klein & Manning (2003), Petrov & Klein (2007), Hindle & Rooth
 
Wed, 10/03 11 Dependency Grammars 1up 4up
Dependency treebanks and parsing
Required reading: Ch. 13 (3rd ed), McDonald & Nivre (2007)
Optional reading: Nivre & Scholz (2004), Kubler et al. (2009), Nivre (2010), McDonald & Nivre (2011)
 
Fri, 10/05 12 Expressive Grammars 1up 4up [4Cr] Proposal due
Going beyond CFGs
Required reading: Ch. 15.1-4, Ch. 16.1, Ch.16.3
Optional reading: Abney (1997), Miyao & Tsujii (2008), Joshi and Schabes (1997), Steedman & Baldridge (2011), Schabes & Shieber, Schabes & Waters (1995), Bangalore & Joshi (1999), Hockenmaier & Steedman (2007), Clark & Curran (2007)
 
Wed, 10/10 13 Review for midterm 1up 4up
NB: Please go over the material before class by yourself
 
Fri, 10/12 14 Midterm Exam
In-class midterm
 
Wed, 10/17 15 Compositional Semantics 1up 4up
What is the meaning of a sentence, and how can we represent it? Basic predicate logic and lambda calculus
Required reading: 17.2-3
Optional reading: Blackburn & Bos (2003), Penn Lambda Calculator
 
Fri, 10/19 16 Semantic Role Labeling 1up 4up MP2 due. MP3 out
How do we represent who does what to whom in a sentence?
Required reading: Ch 18, 3rd Edition
Optional reading: Palmer et al. (2005), Gildea & Jurafsky (2002), Punyakanok et al. (2008)
 
Wed, 10/24 17 Vector-based Lexical Semantics 1up 4up
What is the meaning of a word, and how can we represent it?
Required reading: Chapter 6, 3rd Edition
Optional reading:
 
Fri, 10/26 18 WordNet, Word Relations and Word Senses 1up 4up
Another approach to lexical semantics
Required reading: Appendix C, 3rd Edition
Optional reading:
 
Wed, 10/31 19 No lecture
(Julia out of town -- TAs are available to discuss 4th credit hour projects)
Required reading:
Optional reading:
 
Fri, 11/02 20 Introduction to neural networks for NLP 1up 4up
(Julia out of town)
Required reading: Chapter 7, 3rd Edition
Optional reading:
 
Wed, 11/07 21 Machine Translation 1up 4up
Why is MT difficult? IBM models
Required reading: Ch. 25 (
Optional reading: Brown et al. (1990), Lopez (2008), Lopez (2008)
 
Fri, 11/09 22 Machine Translation 1up 4up MP3 due. MP4 out
More on statistical MT
Required reading: Ch. 25
Optional reading: Koehn et al., Och& Ney (2004), Wu (1997), Chiang (2007) www.statmt.org
 
Wed, 11/14 23 Discourse 1up 4up
What does it take for a text to "make sense"?
Required reading: Grosz et al. (1995), Poesio et al. (2004), Barzilay and Lapata (2008)
Optional reading:
 
Fri, 11/16 24 Dialog 1up 4up
BLURB
Required reading:
Optional reading:
 
Wed, 11/28 25 Neural Language Models and Word Embeddings 1up 4up
Word2Vec, Glove, etc.
Required reading: TBA
Optional reading: TBA
 
Fri, 11/30 26 Neural sequence models 1up 4up MP4 due.
RNNs, LSTMs, GRUs, etc.
Required reading: TBA
Optional reading: TBA
 
Wed, 12/05 27 Seq2Seq models 1up 4up
Neural machine translation
Required reading: TBA
Optional reading: TBA
 
Fri, 12/07 28 Review for final exam 1up 4up M4 due.
We'll go over the material after the midterm
 
Wed, 12/12 29 Final exam (in-class)