Lecture 10: Recurrent Neural Networks

Lecture 10: Recurrent Neural Networks

Fei-Fei Li & Justin Johnson & Serena Yeung

Lecture 10 - 1 May 4, 2017

Administrative

A1 grades will go out soon

A2 is due today (11:59pm)

Midterm is in-class on Tuesday! We will send out details on where to go soon

Fei-Fei Li & Justin Johnson & Serena Yeung

Lecture 10 - 2 May 4, 2017

Extra Credit: Train Game

More details on Piazza by early next week

Fei-Fei Li & Justin Johnson & Serena Yeung

Lecture 10 - 3 May 4, 2017

Last Time: CNN Architectures

AlexNet

Fei-Fei Li & Justin Johnson & Serena Yeung

Figure copyright Kaiming He, 2016. Reproduced with permission.

Lecture 10 - 4 May 4, 2017

Last Time: CNN Architectures

Softmax FC 1000 FC 4096 FC 4096

Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512

Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512

Pool 3x3 conv, 256 3x3 conv, 256

Pool 3x3 conv, 128 3x3 conv, 128

Pool 3x3 conv, 64 3x3 conv, 64

Input

VGG16

Softmax FC 1000 FC 4096 FC 4096

Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512

Pool 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512 3x3 conv, 512

Pool 3x3 conv, 256 3x3 conv, 256

Pool 3x3 conv, 128 3x3 conv, 128

Pool 3x3 conv, 64 3x3 conv, 64

Input

VGG19

GoogLeNet

Fei-Fei Li & Justin Johnson & Serena Yeung

Figure copyright Kaiming He, 2016. Reproduced with permission.

Lecture 10 - 5 May 4, 2017

Last Time: CNN Architectures

relu F(x) + x

conv

F(x)

relu

conv

X identity

X Residual block

Softmax FC 1000

Pool

3x3 conv, 64 3x3 conv, 64

3x3 conv, 64 3x3 conv, 64

3x3 conv, 64 3x3 conv, 64

...

3x3 conv, 128 3x3 conv, 128

3x3 conv, 128 3x3 conv, 128

3x3 conv, 128 3x3 conv, 128 / 2

3x3 conv, 64 3x3 conv, 64

3x3 conv, 64 3x3 conv, 64

3x3 conv, 64 3x3 conv, 64

Pool 7x7 conv, 64 / 2

Input

Fei-Fei Li & Justin Johnson & Serena Yeung

Figure copyright Kaiming He, 2016. Reproduced with permission.

Lecture 10 - 6 May 4, 2017

DenseNet

1x1 conv, 64 Concat

1x1 conv, 64 Concat Conv Concat Conv Input

Dense Block

Softmax FC Pool

Dense Block 3 Conv Pool Conv

Dense Block 2 Conv Pool Conv

Dense Block 1 Conv Input

FractalNet

Figures copyright Larsson et al., 2017. Reproduced with permission.

Fei-Fei Li & Justin Johnson & Serena Yeung

Lecture 10 - 7 May 4, 2017

Last Time: CNN Architectures

Fei-Fei Li & Justin Johnson & Serena Yeung

Lecture 10 - 8 May 4, 2017

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download