Natural Language Processing with Deep Learning …
Natural Language Processing with Deep Learning CS224N/Ling284
Christopher Manning Lecture 2: Word Vectors,Word Senses, and
Classifier Review
Lecture Plan
Lecture 2: Word Vectors and Word Senses 1. Finish looking at word vectors and word2vec (10 mins) 2. Optimization basics (8 mins) 3. Can we capture this essence more effectively by counting? (12m) 4. The GloVe model of word vectors (10 min) 5. Evaluating word vectors (12 mins) 6. Word senses (6 mins) 7. Review of classification and how neural nets differ (10 mins) 8. Course advice (2 mins)
G2 oal: be able to read word embeddings papers by the end of class
1. Review: Main idea of word2vec
? Start with random word vectors
? Iterate through each word in the whole corpus
? Try to predict surrounding words using word vectors
5:9 | 5
569 | 5
5:7 | 5
567 | 5
... problems turning
into
banking crises as ...
?
=
&'((*+,-.) 13 &'((*1, -.)
? Update vectors so you can predict better
? This algorithm learns word vectors that capture word similarity and meaningful directions in a wordspace
3
Word2vec parameters and computations
????? ? ? ? ?? ? ? ? ?? ? ? ? ?? ? ? ? ?? ? ? ? ??
U
outside
????? ? ? ? ?? ? ? ? ?? ? ? ? ?? ? ? ? ?? ? ? ? ??
V
center
? ? ? ? ? ? . >?
dot product
? ? ? ? ? ? softmax(. >?)
probabilities
!
4
Same predictions at each position
We want a model that gives a reasonably high probability estimate to all words that occur in the context (fairly often)
Word2vec maximizes objective function by putting similar words nearby in space
5
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- —william ury getting to yes create your life your
- communicator 5 dynavox
- natural language processing with deep learning
- opening a fillable form in windows 10 s microsoft edge
- list of adjectives describing personal qualities
- ab2 gp pe tp cpy 193601
- tote go laptop tm vtech
- microsoft password guidance
- outline it note it draft it don johnston inc
- counting cengage
Related searches
- deep learning conference 2018
- deep learning trend
- deep learning vs machine learning
- deep learning future
- deep learning pdf
- deep learning neural network
- deep learning versus machine learning
- types of deep learning networks
- deep learning neural network tutorial
- deep learning regression
- deep learning types
- deep learning layer types