NLP Classic

The NLP Canon: From Statistical Foundations to Large Language Models

Master the science of language. This curated "Awesome" list tracks the core breakthroughs in Natural Language Processing (NLP). We start with the Statistical NLP era (HMMs, CRFs, and PCFGs), move through the Neural Shift of the early 2010s (Word2Vec, LSTMs, and Seq2Seq), and culminate in the Transformer Revolution (BERT, GPT, and beyond). This repository serves as a definitive reading list for understanding the algorithms that taught machines to parse, translate, and generate human text.

Managed by Attendemia

Sort

Unsupervised Machine Translation Using Monolingual Corpora Only
Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato
2018
5,989 checkouts
An efficient framework for learning sentence representations
Lajanugen Logeswaran, Honglak Lee
2018
6,367 checkouts
Domain Adapted Word Embeddings for Improved Sentiment Classification
Prathusha K Sarma, YIngyu Liang, William A Sethares
2018
5,174 checkouts
Generalizing Word Embeddings using Bag of Subwords
Jinman Zhao, Sidharth Mudgal, Yingyu Liang
2018
7,018 checkouts
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
2019
9,680 checkouts
Evaluation of sentence embeddings in downstream and linguistic probing tasks
Christian S. Perone, Roberto Silveira, Thomas S. Paula
2018
5,996 checkouts
Universal Sentence Encoder
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, Ray Kurzweil
2018
9,149 checkouts
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
2018
5,457 checkouts
Learned in Translation: Contextualized Word Vectors
Bryan McCann, James Bradbury, Caiming Xiong, Richard Socher
2018
9,008 checkouts
Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations
Andreas Rücklé, Steffen Eger, Maxime Peyrard, Iryna Gurevych
2018
8,002 checkouts
A Compressed Sensing View of Unsupervised Text Embeddings...
Sanjeev Arora, Mikhail Khodak, Nikunj Saunshi, Kiran Vodrahalli
2018
8,803 checkouts
A Simple but Tough-to-Beat Baseline for Sentence Embeddings
Sanjeev Arora, Yingyu Liang, Tengyu Ma
2017
5,083 checkouts
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
Emma Strubell, Patrick Verga, David Belanger, Andrew McCallum
2017
6,995 checkouts
Advances in Pre-Training Distributed Word Representations
Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin
2017
7,267 checkouts
Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets
Rotem Dror, Gili Baumer, Marina Bogomolov, Roi Reichart
2017
9,398 checkouts
Towards Universal Paraphrastic Sentence Embeddings
John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu
2016
6,376 checkouts
Bag of Tricks for Efficient Text Classification
Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov
2016
8,545 checkouts
Enriching Word Vectors with Subword Information
Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov
2017
8,166 checkouts
Counter-fitting Word Vectors to Linguistic Constraints
Nikola Mrkšić, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, Steve Young
2016
6,975 checkouts
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean
2016
8,881 checkouts