Advanced statistical methods used in Natural Language Processing, with a focus on Bayesian methods.


Please check our Piazza group for announcements.

Paper presentations

Signup will soon be available


(subject to ongoing revisions)

01/15 Introduction 1up
01/17 Conjugate priors 1up T. Griffiths and A. Yuille A primer on probabilistic inference; Chapters 8 and 9 of D. Barber Bayesian Reasoning and Machine Learning. See also this diagram of conjugate prior relationships
01/22 Text classification: frequentist vs Bayesian approaches 1up Resnik/Hardisty 2010
01/24 The EM algorithm 1up
01/29 Sampling 1up Ch. 29, McKay (2003), Besag (2001), Casella and George (1992)
01/31 Probabilistic Latent Semantic Analysis 1up Hofmann (2001)
02/05 Latent Dirichlet Allocation 1up Blei et al. 2003, Blei/Lafferty 2009, Blei 2011, Griffiths/Steyvers 2004
02/07 Variational Inference for LDA 1up Blei et al. 2003, Ch. 9.4 Bishop (2006), Jordan et al. (1999), Jaakkola (2000)
02/12 Papers: Correlated topic models 1up Blei and Lafferty (2006a)
02/14 Papers: Evaluating topic models 1up Chang et al. (2009)
02/19 Papers: Supervised LDA 1up Blei and McAuliffe(2007) Supervised Topic Models (pdf)
02/21 Papers: Correspondence LDA 1up Blei and Jordan (2003) (pdf)
02/26 Dirichlet Processes 1up Teh (2010) Dirchlet processes (pdf); Teh (2007)'s tutorial slides (pdf) Navarro et al. Modeling individual differences using Dirichlet processes (pdf); Frigyik et al (2010) Introduction to the Dirichlet Distribution and related processes (pdf)
02/28 Papers: Inference for Dirichlet Processes 1up Blei and Jordan (2006) Variational inference for Dirichlet process mixtures (pdf)
03/05 Hierarchical Dirichlet Processes 1up Teh et al. Hierarchical Dirichlet processes ((pdf))
03/07 Project proposals
03/14 Guest lecture: Yonatan Bisk 1up
03/26 Papers: Unsupervised coreference resolution with HDPs 1up Haghighi and Klein (2007) Unsupervised Coreference resolution in a nonparametric Bayesian model (pdf)
03/28 Papers: Nonparametric language modeling 1up Teh (2006) A hierarchical Bayesian language model based on Pitman-Yor processes (pdf)
04/02 Papers: Comparing estimation methods 1up Asuncion et al (2009) On Smoothing and Inference for Topic Models (pdf) and Gao and Johnson (2008) A comparison of Bayesian estimators for unsupervised Hidden Markov Model POS taggers (pdf)
04/04 Papers: The infinite HMM 1up Beal et al. (2002) The infinite Hidden Markov Model (pdf)
04/09 Papers: Nonparametric PCFGs 1up Liang et al. (2009) (pdf)
04/11 Project updates
04/16 Papers: Grammar induction 1up Cohen and Smith (2010) Covariance in unsupervised learning of probabilistic grammars (pdf);
04/18 Papers: Morphology 1up Dreyer and Eisner (2011) Discovering Morphological Paradigms from Plain Text Using a Dirichlet process mixture model (pdf)
04/23 Papers: Multilingual POS tagging 1up Naseem et al. (2009) Multilingual Part-of-Speech Tagging: Two unsupervised approaches (pdf)
04/25 Papers: Indian Buffet Processes 1up Griffiths and Ghahramani (2011) The Indian Buffet Process: An Introduction and Review (pdf)
04/30 Papers: The nested Chinese Restaurant Process 1up Blei, Griffiths, Jordan (2010) The Nested Chinese Restaurant Process and Bayesian Nonparametric Inference of Topic Hierarchies (pdf)
05/09 Final project presentations (12:30-2:00pm, Room 3405)


50% Research project
30% Paper presentations
20% Attendance/in-class participation

Useful online resources

Tutorials and textbooks
Noah Smith (2011) Linguistic structure prediction
David Barber (2012) Bayesian Reasoning and Machine Learning
Chapter 2 of Erik Sudderth (2006) Graphical Models for Visual Object Recognition and Tracking, PhD Thesis, MIT.

NLP research papers
ACL anthology

Topic models
David Blei's topic modeling site