AI 539-003, Natural Language Processing, Fall 2022

Coordinates T/Th, 2-3:20pm, STAG 212 [Registrar] [Canvas] [Teams]
Instructor Liang Huang (huanlian@)
TAs Ning Dai (dain@)
Office hours Liang: T/Th: 3:25-3:50pm, KEC 2069.
Ning: even weeks: F 4-5pm; odd weeks: M 4-5pm (i.e., the Friday/Monday before HWs are due); KEC Atrium.
Prerequisites
  • required: algorithms: CS 325/519/515.
    a solid understanding of dynamic programming is extremely important.
  • required: proficiency in at least one mainstream programming languages (Python, C/C++, Java).
    HWs will be done in Python 3 only. You can learn Python from these slides in 2 hours.
  • recommended: automata and formal language theory: CS 321/516.
  • recommended: machine learning: CS 534.
Textbooks
(optional)
  • Jurafsky and Martin, 2009 (2nd edi.), Speech and Language Processing. (default)
  • Manning and Schutze. 1999. Foundations of Statistical Natural Language Processing.
Grading
(tentative)
  • HWs 1-5: programming homework (in groups of 3): 12+12+15+15+12 =66% (HW3 and HW4 are 15% each).
    HWi is due on the Monday of week 2*i+1 (e.g., HW1 on 10/3, HW5 on 11/28).
  • EXs: simple exercises (individually): 5% x 2=10%.
    EXi is due on the Friday of week 2*i, to prepare you for HWi (e.g., EX1 on 9/30).
  • HW6 (individually): 12%. Due on the Saturday right after the finals week (12/10).
  • midterm: 12% (week 7, covering HWs 1-3 and EXs 1-2).
  • no final, no project.
Other Policies
  • this course can be used to fulfill the AI area requirement.
  • no late submission will be accepted (since you work in teams).
  • use Teams for discussions, and Canvas for HW submission and checking grades.
  • All course information (slides, HWs, readings, etc) will be available on this page.
MOOCs
(coursera)
  • Jurafsky and Manning (Stanford)
  • Collins (Columbia) -- more mathematical
Objectives This course provides an introduction to natural language processing, the study of human language from a computational perspective. We will cover finite-state machines (weighted FSAs and FSTs), syntactic structures (weighted context-free grammars and parsing algorithms), and machine learning methods (maximum likelihood and expectation-maximization). The focus will be on (a) modern quantitative techniques in NLP that use large corpora and statistical learning, and (b) various dynamic programming algorithms (Viterbi, CKY, Forward-Backward, and Inside-Outside). At the end of this course, students should have a good understanding of the research questions and methods used in different areas of natural language processing. Students should also be able to use this knowledge to implement simple natural language processing algorithms and applications. Students should also be able to understand and evaluate original research papers in natural language processing that build on and go beyond the textbook material covered in class.

Topics/Slides

Exercises

EXs are to be done individually. They usually prepare you for HWs and the midterm.

Programming Assignments

HWs (to be done in groups of 3) are generally due every other Monday at midnight.
They involve Python implementations of various dynamic programming algorithms such as Viterbi, Forward-Backward, and CKY, as well as machine learning algorithms such as MLE and EM.

Background on Japanese

As you can see from the course materials, unlike most NLP courses, this class (following Kevin Knight's tradition) makes very heavy use of the Japanese language as a running example to demonstrate the linguistic diversity, to illustrate transliteration and translation, and to teach the Viterbi and EM algorithms. While we do not require students to have any prior knowledge of Japanese, it is helpful to be familiar with various linguistic aspects of the language, especially in phonology. Here is a great video on the linguistic background of Japanese.