Exercise 1 - Co-occurrence Matrix
Write a co-occurrence matrix for the following sentence:
“A bird in the hand is worth two in the bush.”
Count each word only if it appears directly after the reference word. Is the co-occurrence matrix unique?
Exercise 2 - Convolutional Layers
Consider the following \(4\times 4 \times 1\) input X and a \(2\times 2 \times 1\) convolutional kernel K with no bias term
\[ X = \bpmat 1 & 0 & -2 & 1 \\ 0 & 1 & 1 & 0 \\ 0 & 1 & 0 & 1 \\ -3 & 4 & 0 & 0 \epmat, \qquad % K = \bpmat 2, & 1 \\ 0, & 1 \\ \epmat \]
What is the output of the convolutional layer for the case of stride 1 and no padding?
What if we have stride 2 and no padding?
What if we have stride 2 and zero-padding of size 1?
Exercise 3 - Sizes in MLPs Refresher
You are given an MLP with ReLU activations. It has 3 layers consisting of 5, 10, and 5 neurons respectively. The input is a vector of size 10. How many parameters does this network have?
Exercise 4 - Sizes in CNNs
You are givne a neural network with the following architecture:
Input: 100 x 100 x 3 Image
Layers:
1. Conv(in_channels=3, out_channels=5, kernel_size=3, stride=1, padding=0)
2. MaxPool2d(kernel_size=2, stride=2, padding=0)
3. Conv(in_channels=5, out_channels=10, kernel_size=3, stride=1, padding=0)
4. MaxPool2d(kernel_size=2, stride=2, padding=0)
5. Conv(in_channels=10, out_channels=5, kernel_size=3, stride=1, padding=0)
6. Flatten()
7. MLP(neurons=20)
8. MLP(neurons=10)What is the dimensionality of the activations after each layer.
How many parameters does this network have?