You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -50,6 +48,16 @@ Please feel free to [pull requests](https://github.com/keonkim/awesome-nlp/pulls
50
48
*[Statistical Machine Translation](http://mt-class.org) - a Machine Translation course with great assignments and slides.
51
49
*[Natural Language Processing SFU](http://www.cs.sfu.ca/~anoop/teaching/CMPT-413-Spring-2014/) - course by [Prof Anoop Sarkar](https://www.cs.sfu.ca/~anoop/) on Natural Language Processing. Good notes and some good lectures on youtube about HMM.
52
50
51
+
## Deep Learning for NLP
52
+
[Stanford Natural Language Processing](https://class.coursera.org/nlp/lecture/preview)
53
+
Intro NLP course with videos. This has no deep learning. But it is a good primer for traditional nlp.
54
+
55
+
[Stanford CS 224D: Deep Learning for NLP class](http://cs224d.stanford.edu/syllabus.html)
56
+
[Richard Socher](https://scholar.google.com/citations?user=FaOcyfMAAAAJ&hl=en). (2015) Class with videos, and slides.
57
+
58
+
[A Primer on Neural Network Models for Natural Language Processing](http://u.cs.biu.ac.il/~yogo/nnlp.pdf)
59
+
Yoav Goldberg. October 2015. No new info, 75 page summary of state of the art.
60
+
53
61
54
62
## Codes
55
63
@@ -132,13 +140,88 @@ Please feel free to [pull requests](https://github.com/keonkim/awesome-nlp/pulls
132
140
*[Online named entity recognition method for microtexts in social networking services: A case study of twitter](http://arxiv.org/pdf/1301.2857.pdf)
133
141
134
142
135
-
### Word Vectors
143
+
### Word Vectors (part of it from [DL4NLP](https://github.com/andrewt3000/DL4NLP))
144
+
Resources about word vectors, aka word embeddings, and distributed representations for words.
145
+
Word vectors are numeric representations of words that are often used as input to deep learning systems. This process is sometimes called pretraining.
146
+
147
+
[Efficient Estimation of Word Representations in Vector Space](http://arxiv.org/pdf/1301.3781v3.pdf)
148
+
[Distributed Representations of Words and Phrases and their Compositionality]
[Mikolov](https://scholar.google.com/citations?user=oBu8kMMAAAAJ&hl=en) et al. 2013.
151
+
Generate word and phrase vectors. Performs well on word similarity and analogy task and includes [Word2Vec source code](https://code.google.com/p/word2vec/) Subsamples frequent words. (i.e. frequent words like "the" are skipped periodically to speed things up and improve vector for less frequently used words)
152
+
[Word2Vec tutorial](http://tensorflow.org/tutorials/word2vec/index.html) in [TensorFlow](http://tensorflow.org/)
153
+
154
+
[Deep Learning, NLP, and Representations](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/)
155
+
Chris Olah (2014) Blog post explaining word2vec.
156
+
157
+
[GloVe: Global vectors for word representation](http://nlp.stanford.edu/projects/glove/glove.pdf)
158
+
Pennington, Socher, Manning. 2014. Creates word vectors and relates word2vec to matrix factorizations. [Evalutaion section led to controversy](http://rare-technologies.com/making-sense-of-word2vec/) by [Yoav Goldberg](https://plus.google.com/114479713299850783539/posts/BYvhAbgG8T2)
159
+
[Glove source code and training data](http://nlp.stanford.edu/projects/glove/)
160
+
136
161
*[word2vec](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf) - on creating vectors to represent language, useful for RNN inputs
137
162
*[sense2vec](http://arxiv.org/abs/1511.06388) - on word sense disambiguation
138
163
*[Infinite Dimensional Word Embeddings](http://arxiv.org/abs/1511.05392) - new
139
164
*[Skip Thought Vectors](http://arxiv.org/abs/1506.06726) - word representation method
140
165
*[Adaptive skip-gram](http://arxiv.org/abs/1502.07257) - similar approach, with adaptive properties
141
166
167
+
### Thought Vectors (from [DL4NLP](https://github.com/andrewt3000/DL4NLP))
168
+
Thought vectors are numeric representations for sentences, paragraphs, and documents. The following papers are listed in order of date published, each one replaces the last as the state of the art in sentiment analysis.
169
+
170
+
[Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383.1327&rep=rep1&type=pdf)
171
+
Socher et al. 2013. Introduces Recursive Neural Tensor Network. Uses a parse tree.
172
+
173
+
[Distributed Representations of Sentences and Documents](http://cs.stanford.edu/~quocle/paragraph_vector.pdf)
174
+
[Le](https://scholar.google.com/citations?user=vfT6-XIAAAAJ), Mikolov. 2014. Introduces Paragraph Vector. Concatenates and averages pretrained, fixed word vectors to create vectors for sentences, paragraphs and documents. Also known as paragraph2vec. Doesn't use a parse tree.
175
+
Implemented in [gensim](https://github.com/piskvorky/gensim/). See [doc2vec tutorial](http://rare-technologies.com/doc2vec-tutorial/)
176
+
177
+
[Deep Recursive Neural Networks for Compositionality in Language](http://www.cs.cornell.edu/~oirsoy/files/nips14drsv.pdf)
178
+
Irsoy & Cardie. 2014. Uses Deep Recursive Neural Networks. Uses a parse tree.
179
+
180
+
[Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://aclweb.org/anthology/P/P15/P15-1150.pdf)
181
+
Tai et al. 2015 Introduces Tree LSTM. Uses a parse tree.
Dai, Le 2015 "With pretraining, we are able to train long short term memory recurrent networks up to a few hundred
185
+
timesteps, thereby achieving strong performance in many text classification tasks, such as IMDB, DBpedia and 20 Newsgroups."
186
+
##Machine Translation
187
+
[Neural Machine Translation by jointly learning to align and translate](http://arxiv.org/pdf/1409.0473v6.pdf)
188
+
Bahdanau, Cho 2014. "comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation." Implements attention mechanism.
189
+
[English to French Demo](http://104.131.78.120/)
190
+
191
+
[Sequence to Sequence Learning with Neural Networks](http://arxiv.org/pdf/1409.3215v3.pdf)
192
+
Sutskever, Vinyals, Le 2014. ([nips presentation](http://research.microsoft.com/apps/video/?id=239083)). Uses LSTM RNNs to generate translations. " Our main result is that on an English to French translation task from the WMT’14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8"
193
+
[seq2seq tutorial](http://tensorflow.org/tutorials/seq2seq/index.html) in
194
+
195
+
### Single Exchange Dialogs (from [DL4NLP](https://github.com/andrewt3000/DL4NLP))
196
+
[A Neural Network Approach toContext-Sensitive Generation of Conversational Responses](http://arxiv.org/pdf/1506.06714v1.pdf)
197
+
Sordoni 2015. Generates responses to tweets.
198
+
Uses [Recurrent Neural Network Language Model (RLM) architecture
199
+
of (Mikolov et al., 2010).](http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf) source code: [RNNLM Toolkit](http://www.rnnlm.org/)
200
+
201
+
[Neural Responding Machine for Short-Text Conversation](http://arxiv.org/pdf/1503.02364v2.pdf)
202
+
Shang et al. 2015 Uses Neural Responding Machine. Trained on Weibo dataset. Achieves one round conversations with 75% appropriate responses.
Vinyals, [Le](https://scholar.google.com/citations?user=vfT6-XIAAAAJ) 2015. Uses LSTM RNNs to generate conversational responses. Uses [seq2seq framework](http://tensorflow.org/tutorials/seq2seq/index.html). Seq2Seq was originally designed for machine transation and it "translates" a single sentence, up to around 79 words, to a single sentence response, and has no memory of previous dialog exchanges. Used in Google [Smart Reply feature for Inbox](http://googleresearch.blogspot.co.uk/2015/11/computer-respond-to-this-email.html)
206
+
207
+
### Memory and Attention Models (from [DL4NLP](https://github.com/andrewt3000/DL4NLP))
208
+
[Reasoning, Attention and Memory RAM workshop at NIPS 2015. slides included](http://www.thespermwhale.com/jaseweston/ram/)
209
+
210
+
[Memory Networks](http://arxiv.org/pdf/1410.3916v10.pdf) Weston et. al 2014, and
211
+
[End-To-End Memory Networks](http://arxiv.org/pdf/1503.08895v4.pdf) Sukhbaatar et. al 2015.
212
+
Memory networks are implemented in [MemNN](https://github.com/facebook/MemNN). Attempts to solve task of reason attention and memory.
213
+
[Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](http://arxiv.org/pdf/1502.05698v7.pdf)
214
+
Weston 2015. Classifies QA tasks like single factoid, yes/no etc. Extends memory networks.
215
+
[Evaluating prerequisite qualities for learning end to end dialog systems](http://arxiv.org/pdf/1511.06931.pdf)
216
+
Dodge et. al 2015. Tests Memory Networks on 4 tasks including reddit dialog task.
217
+
See [Jason Weston lecture on MemNN](https://www.youtube.com/watch?v=Xumy3Yjq4zk)
0 commit comments