CS 769: Advanced Natural Language Processing 
Fall 2011 - Benjamin Snyder 

This is the new version of the Advanced Natural Language Processing course.
The goal of this class will be to introduce you to all the techniques and
results of modern statistical natural language processing (NLP). By the end of
the semester you will be able to read and understand NLP research papers and
have the tools you need to conduct original research.

Prerequisites: Familiarity with basic discrete math (e.g. probability),
a tiny bit of calculus and linear algebra, some basic knowledge of algorithms
and basic programming ability. Prior knowledge of linguistics and/or machine
learning would be fantastic, but is not necessary. Knowledge of two or more
languages a plus.

Tentative topics: 

Background: Brief overviews of linguistics (i.e. how do we study
language scientifically?) and probability theory (i.e. how do we model
uncertainty?). Basic statistical estimation. Basic information theory.

Traditional NLP: Supervised and unsupervised learning of
language structures, including phonology, morphology, parts-of-speech,
syntax, and semantics.

NLP Applications: Machine translation, information extraction,
computer assisted language learning.

Computational linguistics and cognitive science: Computational
modeling of language history and phylogenies, reconstruction of ancient
scripts and languages, modeling psycholinguistics and human language
acquisition.

Possible technical topics: Maximum likelihood estimators, Bayesian
vs non-Bayesian models, PCFG's, HMM's, MRF's, CRF's, directed graphical
models, classifiers, Gibb's sampling, variational inference, information
distance. 

Schedule: 11:00--12:15. 1213 Engineering Hall

Instructor: Benjamin Snyder 
   Office: 6395 CS
   E-mail: 
   Office Hours: 1:00--2:00 Tuesdays, or by appointment

Textbook: In lieu of a single textbook, we will focus on reading
research papers and select chapters from various reference books (to be made
available to the students as PDF's). Our main reference texts will be:

        [1] Noah Smith, Linguistic Structure Prediction. Morgan & Clay Publishers, 2011.
        [2] Jurafsky and Margin, Speech and Language Processing. Pearson, Prentice-Hall, 2009.
	[3] Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference.  Springer, 2003. 
	[4] Christopher M. Bishop, Pattern Recognition and Machine Learning. Springer Verlag, 2006.
	[5] Chris Manning and Hinrich Schutze, Foundations of Statistical Natural Language Processing. MIT Press. Cambridge, MA: May 1999.
	[6] David MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2002.

[3] and [4] are worth buying if you're serious about any sort of
applied machine learning research (including NLP). [1] is a new 
book, focusing on structured prediction and machine learning, applied 
to NLP (freely available online from a UW networked computer). [2] is a 
great introduction to speech processing and NLP, [5] is a bit older, but
still provides excellent background material. [6] is available online for
free at the URL above.

Grading: Homeworks (35%), midterm exam (20%), a project (35%), and class participation (10%).

Other: Course URL: http://pages.cs.wisc.edu/~bsnyder/cs769.html