Statistical Computing · End Term · Jan 2024 · FN

1. Let a random variable X have the following cumulative distribution function: F(x) =1-(1-2°), 2eR, and a,b > 0. Write down the steps of the inverse transform algorithm to sample from this distribution. [3 marks] 2. Let X ~ Normal(0, 1). Suppose we want to estimate @ = E{e~* 4 using the simple importance sampling method. (a) Find the optimal proposal density g’. [3 marks] (b) For the above obtained g’, find the simple importance sampling estimator Ores [3 marks] 3. Suppose we want to generate random samples from a truncated Gamma(2, 1) defined on (5,00) using the accept-reject algorithm with truncated exponential(\) defined on (5,00) as a proposal distribution . Truncated Gamma(2, 1) is given by md f(z) = to where x € (5,00) and truncated exponential(,) is given by Net g(x) = a where x € (5,00) What is the optimal value of \? {7 Marks] 4. Suppose we want to minimize the objective function f(x) = cos(x) using the MM algorithm. (a) Find a majorising function f(z | an). {4 marks] (b) Write the iterates of the MM algorithm. {2 marks] 5. Let (Xi, Yi), (Xo, Ya),..., (Xn, Yn) id F, where F is some unknown bivariate distribution with mean vector (6,6)" and finite marginal variances. In order to estimate 0, we choose the estimator: n 5 od eG AY, a 3 i=l Write down the steps of bootstrapping to obtain a 100(1 — a)% confidence interval around §. Write all the steps clearly. {5 marks] . Let X1,Xo,...,Xn SF, where F isa 2component Bernoulli(p) mixture distribution with pmf F(x | pi, pa, 71,72) = mi fi(e | pr) + mafa(z | pa), where 0 < py,po < 1 and 7,7 € (0,1), 7 + 7 = 1. Suppose we want to obtain the maximum likelihood estimate of of 4 = (71,7 ,p1,p2) using the EM algorithm. Recall that the E-step has the function: Q(8 | 4) = Ellog f(z, 2 | @) | X = 2,0 = 0] (a) Write the E-step Q(p | pg) for the EM algorithm. {5 marks] (b) Write the M-step for the EM algorithm. [6 marks] . Study the below code and answer the following sub-questions. n <- 100 p.<= 50 X <- matrix(rnorm(n*p), nrow = n, ncol = p) # Linear regression: Y = X.beta + epsilon y <- X %*% (rnorm(p, mean = 2)) + rnorm(n, mean = 0, sd = 2) lam.vec <- c(10°(seq(-10, 10, by = .1))) error <- numeric(length = length(lam.vec)) for(1 in 1:length(lam.vec)) Hf track.error <- 0 for(i in i:n) { X.train <- X[-i,] y.train <- y[-i] beta.train <- solve(t(X.train) %*% X.train + lam.vec[1]*diag(1,p)) deh t(X.train) %*% y.train track.error <- track.error + abs(y[i] - X[i,] %*% beta.train) = error(1] <- track.error/n } lam.star <- lam.vec(which.min(error)] beta.ridge <- solve(t(X) %*% X + lam.star*diag(p)) %*% t(X) Wek y beta.ridge (a) Which cross validation technique is being implemented in the above code? —_[2 marks] (b) Which loss function has been used to calculate the cross validation estimate? {2 marks} (c) Describe mathematically the output of the function lam. star. [3 marks]