Intro Course Overview (ppt 
, pdf )
Finite State Text Processing, Morphology,
Pynini (ppt , pdf )
Chp, 2
FIRST CLASS on Jan 11
Background Reading on Finite State Automata ONLY-Sec-2.2.1-2.2.2-2.2.3
(if you need it)
Quizz1 (questions 1-6) background/quizzes: FSA Reg Expressions (out 10 - due 17)
Finish FST + Text normalization, Spelling (ppt , pdf )
Appendix B
Chp. 3&7
Background reading On Probability and Information Theory (if
you need it)
Quizz1 (questions 7-12) background/quizzes: Conditional Prob., Bayes rule, ...) (out 10 - due 17)
Hw1 Text Normalization, Pynini (W-FST) and Spelling Checker (out 18 - due 26)
background/quizzes: MLP?
Text Classification (Sentiment) (ppt , pdf )
Hw2 Language Models Traditional vs. neural (out 27 - due Feb 3)
Sequence labeling: Markov Models -POS
tagging and NER (ppt , pdf )
Hw3 text classification newsgroup BOW and fixed embeddings (out Feb 4 - due 13)
Finish Transformers (ppt , pdf )
Pre-trained language models, Transfer Learning
with Contextual Embeddings (ppt , pdf )
BERT, BART, roBERTa.... BERT, The Illustrated BERT, ELMo, and co., Chen2019, BERTScore
March 1st MIDTERM (Practice
Questions )
Intro to syntax, Context Free Grammars and
Parsing (ppt , pdf )
Chunking (Shallow Costituency Parsing by
Fine Tuning), Dependency Parsing, Treebanks (ppt , pdf )
(cont') Dependency/Constituency Parsing
PCFG Traditional CKY / Neural for Both Const. and Dep. (ppt , pdf )
Dependency/Constituency Traditional CKY / Neural
Hw5 newsgroup classification (a) with Roberta document embeddings (b) With syntactic features (POS and dependency relations),
SemEval (Semantic tasks)
Embeddings, Wordnet, Concept Graphs, Lexicons for Sentiment,
Hw6 Syntax & Lexical Semantics
Chp. 18-19
Chp. 6
Hw7 LDA
- Intro Discourse & Discourse Parsing and
Neural topic segmentation/labeling (ppt , pdf )
Coreference, Discourse Parsing (shift-reduce neural)... for e.g. argumentation mining - mention debater IBM system
Hw8 Discourse Parsing (?just to "explore" SOTA system?)
- Intro Summarization (ppt , pdf )
- Apr
7. (read
paper: PEGASUS: Pre-training with Extracted Gap-sentences for
Abstractive Summarization (ICML 2020), paper blog-post) (ppt , pdf ) QUESTIONS (part1)
for a specific task)
Hw9 Summarization (?just to "explore" SOTA system?)