Deep Learning · Quiz 2 · May 2023

Suppose that a neural network has millions of parameters (weights and biases). A team decides to use an optimization algorithm with a learning rate scheme that is local to each individual parameters in the network. Moreover, the learning rate changes in each iteration based on the magnitude of gradients pertaining to a parameter in the past. Which of the following optimization algorithms satisfy the team’s requirements?

Which of the following activation function(s) is (are) prone to vanishing gradient problem?

An input image of dimension 35 × 35 × 3 is to be convolved with 20 filters (kernels) of spatial dimension 3 × 3. In all the sub-questions, assume a bias term for kernels.

Suppose that we pad zeros of size 2 (p = 2) around the sides of the image (in all channels). Assume stride 2 for the convolution operation. All the 20 filters have bias associated with it. What is the dimension of the output volume? Enter the number of elements in the output volume.

An input image of dimension 35 × 35 × 3 is to be convolved with 20 filters (kernels) of spatial dimension 3 × 3. In all the sub-questions, assume a bias term for kernels.

How many numbers of parameters are there in the convolution layer?

An input image of dimension 35 × 35 × 3 is to be convolved with 20 filters (kernels) of spatial dimension 3 × 3. In all the sub-questions, assume a bias term for kernels.

Create two feature maps by convolving the kernels K, F over the input matrix X and stack the outputs to create a tensor as shown below. The dimension of A is m x n x 2. The first feature map of size m x n is the output for K and the second feature map of size m x n is for F. What is the sum of the elements in the tensor A?

Enter the sum of updated bias (exclude weights).