Disclaimer: This is still mostly the syllabus from previous years. While we will follow the general sequence of topics, you can expect the reading materials and specific lectures to change somewhat this year.
Required readings refer to chapters in Jurafsky and Martin (2008), Speech and Language Processing, 2nd edition, unless stated otherwise. Note that the 3rd edition is still in preparation, although the website has (currently) a number of new and/or rewritten chapters available as PDFs.
Optional readings are often more advanced. "MS" refers to chapters in Manning and Schütze (1999), Foundations of Statistical Natural Language Processing (you may need to use a campus machine to access the links to the chapters below) or to original research papers
(you can find many more on the ACL anthology). I also recommend the Handbook of Computational Linguistics and Natural Language Processing (you also need to be on the campus network to access this site).
Lecture slides and Assignments are also linked to on this page. We typically release assignments in the evening of the day they come out.
Wed, 08/29 | 01 | Introduction | 1up 4up | |
What is NLP? What will you learn in this class? | ||||
Required reading: Ch.1 | ||||
Optional reading: Python tutorial (sec. 1-5), Jelinek (2009), Ferrucci et al. (2010) | ||||
Fri, 08/31 | 02 | Finite-state methods for morphology | 1up 4up | MP0 out |
What is the structure of words, and how can we model it? Review of finite-state automata. Finite-state transducers | ||||
Required reading: Ch.3.1-7 (2nd Ed.); | ||||
Optional reading: Karttunen and Beesley '05, Mohri (1997), the Porter stemmer, Sproat et al. (1996) | ||||
Other links: | ||||
Wed, 09/05 | 03 | N-gram language models | 1up 4up | |
The most basic probabilistic models of language. N-gram models. Evaluation. Also: review of basic probability | ||||
Required reading: Ch. 3.1-3 (3rd Ed.) | ||||
Optional reading: MS, Ch. 2 | ||||
Fri, 09/07 | 04 | Smoothing | 1up 4up | MP1 out |
How can we predict what we haven't seen before? | ||||
Required reading: Ch. 3.4-8 (3rd Ed.) | ||||
Optional reading: MS, Ch.6, Chen and Goodman (1998) | ||||
Wed, 09/12 | 05 | Part-of-speech tagging (I) | 1up 4up | |
What are parts of speech? How many are there? Basic intro to HMMs. | ||||
Required reading: Ch. 8.1-4(3rd Edition) | ||||
Optional reading: Merialdo (1994), Christodoulopoulos et al. (2010), Roche & Schabes (1995) | ||||
Fri, 09/14 | 06 | POS tagging (II) | 1up 4up | |
Required reading: Ch. 8.1-4(3rd Edition) | ||||
Optional reading: MS, Ch. 9 | ||||
Wed, 09/19 | 07 | Sequence Labeling | 1up 4up | |
Chunking, shallow parsing, named entity recognition | ||||
Required reading: Ch. 6.6-8 (2nd Ed) | ||||
Optional reading: Sutton & McCallum (2008) (Introduction to Conditional Random Fields), Berger et al. (1996), Etzioni et al. (2008) (web-scale information extraction) | ||||
Fri, 09/21 | 08 | Grammars | 1up 4up | |
What is the structure of sentences, and how can we model it? Phrase-structure grammar and dependency grammar. Review of basic English grammar and context-free grammars | ||||
Required reading: Ch. 10.1-3 (3rd Ed) or Ch. 12.1-3 and Ch. 12.7 (2nd Edition) | ||||
Optional reading: MS, Ch. 3, Woods (2010) | ||||
Wed, 09/26 | 09 | Syntactic Parsing | 1up 4up | |
PCFGs and Viterbi | ||||
Required reading: Ch. 11 (3rd Ed.) or Ch. 13.1-4, Ch. 14.1 | ||||
Optional reading: Chi (1999) | ||||
Fri, 09/28 | 10 | Syntactic Parsing | 1up 4up | MP1 due. MP2 out |
Going beyond simple PCFGs; Penn Treebank parsing | ||||
Required reading: Ch 12 (3rd Edition) or Ch. 14.1-7, Ch. 12.4 | ||||
Optional reading: Collins' notes, Chi & Geman (1998), Schabes et al. (1993), Schabes & Pereira (1992), Stolcke (1995), Marcus et al. (1993), Collins (1997), Johnson (1998), Klein & Manning (2003), Petrov & Klein (2007), Hindle & Rooth | ||||
Wed, 10/03 | 11 | Dependency Grammars | 1up 4up | |
Dependency treebanks and parsing | ||||
Required reading: Ch. 13 (3rd ed), McDonald & Nivre (2007) | ||||
Optional reading: Nivre & Scholz (2004), Kubler et al. (2009), Nivre (2010), McDonald & Nivre (2011) | ||||
Fri, 10/05 | 12 | Expressive Grammars | 1up 4up | [4Cr] Proposal due |
Going beyond CFGs | ||||
Required reading: Ch. 15.1-4, Ch. 16.1, Ch.16.3 | ||||
Optional reading: Abney (1997), Miyao & Tsujii (2008), Joshi and Schabes (1997), Steedman & Baldridge (2011), Schabes & Shieber, Schabes & Waters (1995), Bangalore & Joshi (1999), Hockenmaier & Steedman (2007), Clark & Curran (2007) | ||||
Wed, 10/10 | 13 | Review for midterm | 1up 4up | |
NB: Please go over the material before class by yourself | ||||
Fri, 10/12 | 14 | Midterm Exam | ||
In-class midterm | ||||
Wed, 10/17 | 15 | Compositional Semantics | 1up 4up | |
What is the meaning of a sentence, and how can we represent it? Basic predicate logic and lambda calculus | ||||
Required reading: 17.2-3 | ||||
Optional reading: Blackburn & Bos (2003), Penn Lambda Calculator | ||||
Fri, 10/19 | 16 | Semantic Role Labeling | 1up 4up | MP2 due. MP3 out |
How do we represent who does what to whom in a sentence? | ||||
Required reading: Ch 18, 3rd Edition | ||||
Optional reading: Palmer et al. (2005), Gildea & Jurafsky (2002), Punyakanok et al. (2008) | ||||
Wed, 10/24 | 17 | Vector-based Lexical Semantics | 1up 4up | |
What is the meaning of a word, and how can we represent it? | ||||
Required reading: Chapter 6, 3rd Edition | ||||
Optional reading: | ||||
Fri, 10/26 | 18 | WordNet, Word Relations and Word Senses | 1up 4up | |
Another approach to lexical semantics | ||||
Required reading: Appendix C, 3rd Edition | ||||
Optional reading: | ||||
Wed, 10/31 | 19 | No lecture | ||
(Julia out of town -- TAs are available to discuss 4th credit hour projects) | ||||
Required reading: | ||||
Optional reading: | ||||
Fri, 11/02 | 20 | Introduction to neural networks for NLP | 1up 4up | |
(Julia out of town) | ||||
Required reading: Chapter 7, 3rd Edition | ||||
Optional reading: | ||||
Wed, 11/07 | 21 | Machine Translation | 1up 4up | |
Why is MT difficult? IBM models | ||||
Required reading: Ch. 25 ( | ||||
Optional reading: Brown et al. (1990), Lopez (2008), Lopez (2008) | ||||
Fri, 11/09 | 22 | Machine Translation | 1up 4up | MP3 due. MP4 out |
More on statistical MT | ||||
Required reading: Ch. 25 | ||||
Optional reading: Koehn et al., Och& Ney (2004), Wu (1997), Chiang (2007) www.statmt.org | ||||
Wed, 11/14 | 23 | Discourse | 1up 4up | |
What does it take for a text to "make sense"? | ||||
Required reading: Grosz et al. (1995), Poesio et al. (2004), Barzilay and Lapata (2008) | ||||
Optional reading: | ||||
Fri, 11/16 | 24 | Dialog | 1up 4up | |
BLURB | ||||
Required reading: | ||||
Optional reading: | ||||
Wed, 11/28 | 25 | Neural Language Models and Word Embeddings | 1up 4up | |
Word2Vec, Glove, etc. | ||||
Required reading: TBA | ||||
Optional reading: TBA | ||||
Fri, 11/30 | 26 | Neural sequence models | 1up 4up | MP4 due. |
RNNs, LSTMs, GRUs, etc. | ||||
Required reading: TBA | ||||
Optional reading: TBA | ||||
Wed, 12/05 | 27 | Seq2Seq models | 1up 4up | |
Neural machine translation | ||||
Required reading: TBA | ||||
Optional reading: TBA | ||||
Fri, 12/07 | 28 | Review for final exam | 1up 4up | M4 due. |
We'll go over the material after the midterm | ||||
Wed, 12/12 | 29 | Final exam (in-class) |