As per gradient descent, we should move towards 180 degrees with respect to gradient direction.
What will happen if we move between 90 and 180 degrees? Consider the loss function to be
convex.
6406533529621.
The loss function will increase.
6406533529622.
The loss function will decrease, although not to the maximum possible extent.
6406533529623.
The loss function will remain the same.
6406533529624.
Can’t say. It depends on other parameters of the convex function.
A published solution is not available for this question yet.
Consider a feed-forward neural network containing three inputs and one output. It consists of 50
hidden layers, each with two neurons. The activation function used in all the hidden layers is ReLU,
the output layer uses the Sigmoid activation function for binary classification. The network is
trained with a binary cross-entropy loss function.
The network architecture is as follows:
- Input layer: 3 nodes
- Hidden layer: 2 nodes each
- Output layer: 1 node
Assume all weights are initialized to 1 and all biases to 0. For an input of [-10, 5, -20], what will be
the output of the neural network?
6406533529637.
0
6406533529638.
1
6406533529639.
0.5
6406533529640.
0.25
6406533529641.
-25
6406533529642.
Insufficient information
A published solution is not available for this question yet.
Suppose you have a vocabulary of 5,000 unique words and you want to train a CBOW model with a
window size of 3 (on each side) and with an embedding dimension of 100. How many parameters
(weights) will the embedding layer have?
A published solution is not available for this question yet.
Assume that your CBOW model outputs a probability distribution over a vocabulary of 20,000
words for a given context. If the correct target word is word number 150, and the model’s
predicted probability for this word is 0.2, calculate the cross-entropy loss for this prediction. (use
natural log)
A published solution is not available for this question yet.
In a Skip-gram model with a window size of 2 (on each side), how many **unique** pairs of target and
context words will be generated for the following sentence:
‘**Predicting the future is not magic, it is artificial intelligence**’
**Note:** Ignore the punctuations in your calculation
A published solution is not available for this question yet.
Based on the above data, answer the given subquestions.
With a total of 7 time steps (T = 7), implying prediction for a word of length 6), what is the total
count of parameters (including bias) within the network?
A published solution is not available for this question yet.
In the context of the Transformer model’s encoder-decoder architecture, which of the following
statements are accurate?
6406533529669.
The encoder in a Transformer model is responsible for converting the input
sequence into a continuous representation that the decoder can then use to generate the output
sequence.
6406533529670.
The decoder in a Transformer model only attends to the encoder’s output
without any form of self-attention on its own inputs.
6406533529671.
Multi-head attention in the encoder allows the model to jointly attend to
information from different representations at different positions.
6406533529672.
The decoder’s self-attention mechanism includes a masking component to
prevent attending to future positions, ensuring the model generates outputs one step at a time.
A published solution is not available for this question yet.
Consider a Transformer model with the following specifications for the decoder part:
• Input dimension (embedding size): 16
• Number of heads in multi-head attention: 2
• Head output dimension: 8
• Dimension of the feed-forward network: 32
• Number of layers in the decoder: 2
Note: The final output from the attention heads is obtained by concatenating the outputs from the
individual heads and hence does not cost any additional parameters.
Assume that each decoder layer contains:
• One multi-head attention mechanism for self-attention.
• One multi-head attention mechanism for encoder-decoder attention.
• One feed-forward network.
• No bias terms are included.
Calculate the total number of parameters in the decoder part.
A published solution is not available for this question yet.
