Tutorial for the Lecture Applied Machine Learning

Names:

Tutorial for the Lecture Applied Machine Learning

Freie Universit?at Berlin, SoS 2015 Annalisa Marsico ? Matt Huska ? Wolfgang Kopp

Sheet 1 ? Assignment on 20.05.2015 Deliver on 03.06.2015 before the lecture

Immatriculation numbers:

Group:

Introduction and goal of the tutorial

This time you will explore neural networks even further. As you have noticed from the previous assignment, R does not have very advanced deep learning packages. In this tutorial you will use a python tutorial on deep learning that takes advantage of a specialized library, called theano, which provides many useful functions for deep learning. Optionally it can greatly improve the runtime performance by taking advantage of the GPU.

In this tutorial we will focus on multilayer perceptrons (also called feed forward neural networks), but the tutorial also contains more advanced methods such as the ones discussed in the class (e.g. restricted Boltzmann machines, deep neural networks).

Neural networks 2

Assignment 1. Download and install the deep learning tutorial

1. Go to

2. Follow the instructions under 'Getting started'

3. Download the tutorial from git by typing 'git clone git://lisa-lab/DeepLearningTutorials.git'

4. You need to install python (>=2.6) and the theano library (see . net/software/theano/install.html#install)

5. Check the installation by running the logistic regression example (python code/logistic sgd.py).

6. Hint: The examples will run much faster on the GPU if you have install everything accordingly.

Assignment 2. Multilayer perceptron (aka feed forward neural network) on MNIST handwritten digits

1. Explore what hidden feature the neural network identifies in order to classify the handwritten digits correctly. For this task reduce the number of hidden neurons to 100. You need to install Python Image Library (PIL) and add the following piece of code in the training loop

image = Image.fromarray(tile_raster_images( X=classifier.hiddenLayer.W.get_value(borrow=True).T, img_shape=(28, 28), tile_shape=(10, 10), tile_spacing=(1, 1))) image.save('/w1_epoch_%i.png' % epoch)

1

2. Hint for the following examples: If you cannot run the examples on a GPU and training takes too long you may reduce the number of hidden neurons.

3. Lower the number of hidden nodes (n hidden) to 20, 50, 100. How fast does it train? What is its final error rate?

4. By default the tutorial uses a constant learning rate. Often it is better to use a high learning rate in the beginning of learning and a smaller one towards the end. This might speed up learning. Change the learning rate to be ?0 , 1+d?t where ?0 is the initial learning rate (find a good value for it), d the decrease of the learning rate (use 10-3) and t the current epoch.

5. The example makes use of early stopping in order to prevent the net from overfitting. Explore what happens when you disable early stopping. Therefore look for the 'patience' variable in the code and prevent it from stopping training. Reduce the number of training instances (e.g. 10000) and hidden neurons (e.g. 100). Output the average validation loss, test loss and training loss at every epoch into a file and plot these in a diagram. For the training loss, introduce a new function just as test model called train model monitor which determines the error on the training data directly. Query the training loss inside the training loop at the same place where validation model was used.

6. Hint: The default maximum number of epochs is 1000. You might want to reduce it if learning takes too long, but not so small that we do not see overfitting.

2

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download