Question 125 MCQ · 0.0 marks
Useful Data has been mentioned above.
This data attachment is just for a reference & not for an evaluation.
A published solution is not available for this question yet.
Reinforcement Learning · End Term · Sep 2023 · FN
Reinforcement Learning · End Term · Sep 2023 FN
← Course papers · Start practice / exam
This page contains the reliably extracted subset, not the complete original paper.
Questions and published explanations below are available without starting a test. Some questions may not have a published solution yet.
0 / 31 published solutions
Question 126 MCQ · 1.0 marks
In the context of a multi arm bandit (MAB) problem and stationary reward distribution, consider
following:
**Assertion:** UCB minimizes the regret better than ϵ-greedy approach.
**Reason:** ε-greedy approach keeps the probability of choosing a suboptimal arm constant.
Assertion and Reason are both true and Reason is a correct explanation of
Assertion.
Assertion and Reason are both true and Reason is not a correct explanation of
Assertion.
Assertion is true and Reason is false
Both Assertion and Reason are false.
A published solution is not available for this question yet.
Question 127 MCQ · 1.0 marks
In the policy improvement step of policy iteration for a finite MDP, if the tie among actions, which
have the same maximum value, is broken randomly, what would happen to the convergence of
the algorithm?
The algorithm’s convergence is independent of how ties are broken
The algorithm may oscillate between multiple optimal policies and may never
converge or convergence may be delayed
The algorithm will certainly not converge
None of these
A published solution is not available for this question yet.
Question 128 MCQ · 1.0 marks
Real-time dynamic programming, or RTDP, is an on-policy trajectory-sampling version of which of
the following algorithms?
Value iteration.
Q-learning.
SARSA.
TD.
None of these.
A published solution is not available for this question yet.
Question 129 MCQ · 1.0 marks
Which of the following is the correct 5-step truncated corrected return starting from time t ?
A published solution is not available for this question yet.
Question 130 MCQ · 1.0 marks
Which of the following is the correct way to represent policy for policy search methods?
A published solution is not available for this question yet.
Question 131 MSQ · 1.0 marks
Consider a grid world with obstacles and boundary walls. An agent can take only 4 actions, i.e.
north, south, east or west. In a state, an action that might lead to an obstacle or out of boundary,
does not change the state. The agent is trained sufficiently. Which of the following settings will
lead to the agent finding the shortest (i.e. least number of steps) to the goal? Note that a trajectory
ends if the agent reaches the goal.
A published solution is not available for this question yet.
Question 132 MSQ · 1.0 marks
What are the advantages of policy search methods over other approaches?
They can lead to simpler solution description.
They offer better convergence as compared to function approximation based
methods.
In continuous action setting, they work better than value function based
approaches.
They are robust to partial observability.
None of these.
A published solution is not available for this question yet.
Question 133 MSQ · 1.0 marks
A published solution is not available for this question yet.
Question 134 NAT · 1.0 marks
A published solution is not available for this question yet.
Question 135 MCQ · 2.0 marks
In every visit Monte Carlo methods, multiple samples for one state are obtained from a single
trajectory. Which of the following is true?
There is an increase in bias of the estimates.
There is an increase in variance of the estimates.
It does not affect the bias or variance of estimates.
Both bias and variance of the estimates increase.
None of these.
A published solution is not available for this question yet.
Question 136 MCQ · 2.0 marks
A published solution is not available for this question yet.
Question 137 MCQ · 2.0 marks
A published solution is not available for this question yet.
Question 138 MCQ · 2.0 marks
The output layer is a fully connected linear layer in DQNs. Can we use a sigmoid or ReLU activation
at the output layer instead?
We can’t use either ReLU or sigmoid because it becomes difficult to
backpropagate the errors if the last layer has a non-linear activation function.
We can use ReLU but not sigmoid. This is because all the Q values need not lie
between 0 and 1.
We cannot use either ReLU or sigmoid. This is because the Q value could be
any real number.
None of these
A published solution is not available for this question yet.
Question 139 MCQ · 2.0 marks
Which of the following is the correct update rule for policy parameter θ if the policy is represented
in soft-max fashion for a multi arm bandit?
A published solution is not available for this question yet.
Question 140 MCQ · 2.0 marks
Which of the following is the correct update rule for MC with policy gradient?
A published solution is not available for this question yet.
Question 141 MCQ · 2.0 marks
Left
Right
Up
Down
Insufficient information
None of these.
A published solution is not available for this question yet.
Question 142 MCQ · 2.0 marks
Which of the following is correct Q-learning update rule with options ? Make suitable assumptions,
the symbols have usual meaning. s is the state encountered at timestep t.
A published solution is not available for this question yet.
Question 143 MCQ · 2.0 marks
Which of the following is a markov option? Assume the agent is traversing a grid world.
A published solution is not available for this question yet.
Question 144 MCQ · 2.0 marks
yes, yes, no
no, no, no
yes, yes, yes
no, no, yes
A published solution is not available for this question yet.
Question 145 MCQ · 2.0 marks
Consider following steps involved in Monte Carlo Tree Search:
A Simulation
B Backup
C Expansion
D Selection
Organize the above steps in their chronological order:
ABCD
DCAB
BCAD
CBAD
None of these
A published solution is not available for this question yet.
Question 146 MCQ · 2.0 marks
Assertion and Reason are both true and Reason is a correct explanation of
Assertion.
Assertion and Reason are both true and Reason is not a correct explanation of
Assertion.
Assertion is true and Reason is false
Both Assertion and Reason are false.
A published solution is not available for this question yet.
Question 147 MCQ · 2.0 marks
UCT is different from the original version of MCTS in which of the following steps?
Expansion
Selection
Backup
Simulation
A published solution is not available for this question yet.
Question 148 MSQ · 2.0 marks
Which of the following statements are **FALSE** about solving MDPs using dynamic programming?
If the state space is large or computation power is limited, it is preferred to
update only some states through random sampling or selecting states seen in trajectories.
Knowledge of transition probabilities is not necessary for solving MDPs using
dynamic programming.
Methods that update only a subset of states at a time guarantee performance
equal to or better than classic DP.
DP methods bootstrap but do not sample.
A published solution is not available for this question yet.
Question 149 MSQ · 2.0 marks
Which of the following are on policy algorithms?
SARSA
Expeted SARSA
Q-Learning
DQN
DDQN
None of these
A published solution is not available for this question yet.
Question 150 NAT · 1.0 marks
A published solution is not available for this question yet.
Question 151 NAT · 2.0 marks
A published solution is not available for this question yet.
Question 152 NAT · 2.0 marks
A published solution is not available for this question yet.
Question 153 NAT · 2.0 marks
A published solution is not available for this question yet.
Question 154 NAT · 2.0 marks
If you have to train a DQN for an Atari-like game with only 10 actions, how many neurons would
the final layer have?
A published solution is not available for this question yet.
Question 155 NAT · 2.0 marks
A published solution is not available for this question yet.