Machine Learning Foundations · Quiz 2 · May 2026

**1. Apply the Hermitian condition to a diagonal entry:** A Hermitian matrix satisfies \(A=A^*\). On the diagonal this requires \(a_{jj}=\overline{a_{jj}}\). **2. Compare real and imaginary parts:** If \(a_{jj}=u+iv\), then \(u+iv=u-iv\), so \(2iv=0\) and v=0. Every diagonal entry is therefore real; a nonzero imaginary component is impossible. **Answer: B — No.** **Convention note:** Zero is allowed on the diagonal. “Imaginary” here means an entry with a nonzero imaginary part, not the zero number. A: **Incorrect:** A nonzero imaginary component would make a diagonal entry unequal to its conjugate. B: **Correct:** Each diagonal entry must equal its own complex conjugate and hence must be real.

**1. Check the conjugate transpose:** For \(A=\begin{pmatrix}0&i\\-i&0\end{pmatrix}\), transposing and conjugating gives \(A^*=\begin{pmatrix}0&i\\-i&0\end{pmatrix}=A\). Thus A is Hermitian. **2. Check unitarity independently:** Multiplication gives \(A^2=\begin{pmatrix}i(-i)&0\\0&(-i)i\end{pmatrix}=I\). Since \(A^*=A\), this also gives \(A^*A=A^2=I\), making A unitary. **3. Combine the properties:** Hermitian and unitary are compatible properties. Here A is both, so the options containing “only” are incomplete. **Answer: C — A is both Hermitian and unitary.** A: **Incorrect:** Hermitian is true, but “only” is false because A is also unitary. B: **Incorrect:** Unitary is true, but A is also Hermitian. C: **Correct:** Both defining identities, A*=A and A*A=I, hold. D: **Incorrect:** Both properties hold, contrary to this option.

**1. Recall the Taylor coefficient formula:** About zero, \(T_3(x)=f(0)+f^\prime(0)x+\tfrac{f^{\prime\prime}(0)}{2!}x^2+\tfrac{f^{\prime\prime\prime}(0)}{3!}x^3\). The polynomial coefficients are derivatives divided by factorials. **2. Recover the derivatives:** Comparing with \(1+2x+3x^2+4x^3\) gives \(f^\prime(0)=2\), \(f^{\prime\prime}(0)=3\cdot2!=6\) and \(f^{\prime\prime\prime}(0)=4\cdot3!=24\). **3. Evaluate the requested expression:** \(f^\prime(0)+f^{\prime\prime}(0)/2+f^{\prime\prime\prime}(0)=2+6/2+24=29\). **Answer: A — 29.** A: **Correct:** This includes the factorial factors and gives 2+3+24. B: **Incorrect:** This does not equal the recovered derivative combination. C: **Incorrect:** This incorrectly combines the derivatives or omits the division by 2. D: **Incorrect:** This does not match 2+6/2+24.

**1. Express the full PCA transformation:** For centered data with covariance C, choose an orthogonal eigenvector matrix Q satisfying \(C=Q\Lambda Q^T\). Coordinates in the principal-component basis have covariance \(Q^TCQ=\Lambda\), which is diagonal (I). **2. Interpret the new axes:** Eigenvectors of a real symmetric covariance matrix can be chosen orthonormally, so the principal axes form an orthogonal basis (II). Each diagonal covariance entry is the corresponding eigenvalue, hence the variance along component i is \(\lambda_i\) (III). **3. Compare total variance:** Total variance is the covariance trace. Orthogonal changes of basis preserve it: \(\operatorname{tr}(Q^TCQ)=\operatorname{tr}(C)=\sum_i\lambda_i\), proving IV. **4. Select all four statements:** Each property holds for the full PCA rotation. Keeping only some components is a separate dimensionality-reduction step that can discard variance. **Answer: D — statements I, II, III and IV.** **Convention note:** The total-variance claim assumes all principal components are retained. Truncated PCA retains the sum of only the selected eigenvalues and generally reduces total variance. A: **Incorrect:** I and II are true, but this omits the true statements III and IV. B: **Incorrect:** III is also true, but IV remains true for the full transformation. C: **Incorrect:** II is true as well; the PCA axes can be chosen orthogonal. D: **Correct:** I, II, III and IV all hold for full-component PCA.

**1. Apply the matrix identity to an eigenvector:** Let \(Av=\lambda v\) with v nonzero. Then \(A^3v=\lambda^3v\), while \(A^3=A\) gives \(\lambda^3v=\lambda v\). Thus \(\lambda^3-\lambda=0\). **2. Factor the eigenvalue equation:** \(\lambda(\lambda-1)(\lambda+1)=0\), so the only possibilities are -1,0,1. Hermitian matrices also have real eigenvalues, consistent with these values. **3. Verify possibility, not necessity:** A one-by-one matrix containing any of -1,0,1 is Hermitian and satisfies the identity. Therefore each of these can occur, although a particular A need not contain all three eigenvalues. **Answer: A, B, C — −1, 0 and 1.** A: **Correct:** -1 is a root and can occur, for example in the one-by-one matrix [-1]. B: **Correct:** 0 is a root and occurs in the zero matrix. C: **Correct:** 1 is a root and occurs in the identity matrix. D: **Incorrect:** i is nonreal and does not satisfy the polynomial identity. E: **Incorrect:** This nonreal number is neither -1, 0 nor 1.

**1. Determine k from the trace:** For \(C=\begin{pmatrix}k&4\\4&k\end{pmatrix}\), the trace equals the sum of the supplied eigenvalues: \(2k=9+1=10\). Hence k=5. **2. Find the principal direction:** Multiplying \((1,1)^T\) by C gives \((k+4)(1,1)^T=9(1,1)^T\). The perpendicular vector \((1,-1)^T\) has eigenvalue \(k-4=1\). The first principal axis is therefore \((1,1)^T/\sqrt2\), up to sign. **3. Compute explained variance:** The first component captures \(9/(9+1)=0.9=90\%\) of total variance. **Answer: A, C, E — k=5; first direction (1,1)/√2; 90% variance captured.** A: **Correct:** The trace equation gives k=5. B: **Incorrect:** 10 is the trace, not the common diagonal entry k. C: **Correct:** This unit vector corresponds to the largest eigenvalue 9. D: **Incorrect:** This direction corresponds to eigenvalue 1 and is the second component. E: **Correct:** The leading eigenvalue divided by their sum is 9/10.

**1. Derive the scalar iteration:** For \(f(x)=ax^2\), a>0, \(f^\prime(x)=2ax\). Gradient descent gives \(x_{t+1}=(1-2a\eta)x_t\), and consequently \(x_t=(1-2a\eta)^tx_0\). **2. Determine convergence and one-step arrival:** For a nonzero initial value, convergence to zero requires \(|1-2a\eta|<1\), or \(0<\eta<1/a\). At \(\eta=1/(2a)\), the multiplier vanishes, reaching the unique global minimum in one step. **3. Inspect the boundary rate:** At \(\eta=1/a\), the multiplier is -1. Nonzero iterates alternate \(x_0,-x_0,x_0,\ldots\), so they do not reach zero or converge. Larger positive rates can even make the magnitude grow. **Answer: B, D — one-step arrival at η=1/(2a), and alternation at η=1/a.** A: **Incorrect:** Not every positive rate converges; rates at or beyond 1/a fail for nonzero initial points. B: **Correct:** The update multiplier is zero at this rate. C: **Incorrect:** At this rate a nonzero initial point changes sign rather than reaching zero. D: **Correct:** The multiplier -1 produces exactly the stated two-value cycle.

**1. Use the SVD action on each right singular vector:** The supplied 5-by-4 diagonal Sigma has singular values 8,5,2,0. Thus \(Av_1=8u_1,Av_2=5u_2,Av_3=2u_3,Av_4=0\). **2. Apply linearity to the given vector:** With \(x=3v_1-2v_2+4v_3+4v_4\), \[Ax=24u_1-10u_2+8u_3.\] The v4 component lies in the null space and disappears. **3. Compute its squared norm:** The left singular vectors are orthonormal, so all cross terms vanish. Therefore \(\|Ax\|^2=24^2+(-10)^2+8^2=576+100+64=740\). **Answer: A, C — \(Ax=24u_1-10u_2+8u_3\) and \(\|Ax\|^2=740\).** A: **Correct:** Each coefficient is multiplied by its corresponding singular value. B: **Incorrect:** The contribution from v3 lies along u3, not u4. C: **Correct:** Orthonormality gives the sum of the three squared coefficients, equal to 740. D: **Incorrect:** 676 omits the u3 contribution of 8²=64.

**1. Parameterize the line:** Any point on y=x is (t,t). Its squared distance from (2,0) is \(D(t)=(t-2)^2+t^2=2t^2-4t+4\). Minimizing squared distance also minimizes distance. **2. Minimize the one-variable quadratic:** \(D^\prime(t)=4t-4=0\) gives t=1. Since \(D^{\prime\prime}(t)=4>0\), the closest point is \((p_1,p_2)=(1,1)\). **3. Return the requested sum:** \(p_1+p_2=1+1=2\). As a geometric check, (2,0)−(1,1)=(1,-1) is perpendicular to the line’s direction (1,1). **Answer: 2.**

**1. Use norm preservation:** A unitary matrix satisfies \(U^*U=I\), so \(\|Ux\|^2=x^*U^*Ux=x^*x=\|x\|^2\). Therefore no inverse or explicit matrix entries are needed. **2. Use complex magnitudes:** For the supplied output \((2+i,1-2i,3)^T\), the squared coordinate magnitudes are \(|2+i|^2=4+1=5\), \(|1-2i|^2=1+4=5\) and \(|3|^2=9\). **3. Add:** \(\|x\|^2=\|Ux\|^2=5+5+9=19\). Complex norm squares use conjugate products, not ordinary complex squares. **Answer: 19.**

**1. Use the supplied matrix:** \(A=\begin{pmatrix}1&1\\1&1\end{pmatrix}\). Along (1,1), multiplication doubles the vector; along (1,-1), multiplication gives zero. Thus its eigenvalues are 2 and 0. **2. Verify algebraically:** \(\det(A-\lambda I)=(1-\lambda)^2-1=\lambda(\lambda-2)\). Both eigenvalues also sum to trace(A)=2 and multiply to det(A)=0. **Answer: A — 2 and 0.** A: **Correct:** This is the eigenvalue pair obtained from the characteristic polynomial. B: **Incorrect:** The repeated columns imply a zero eigenvalue, so both eigenvalues cannot be 1. C: **Incorrect:** 4 is an eigenvalue of AᵀA, not of A. D: **Incorrect:** This would give trace 4 and a nonzero determinant.

**1. Use the supplied matrix:** \(A=\begin{pmatrix}1&1\\1&1\end{pmatrix}\). Along (1,1), multiplication doubles the vector; along (1,-1), multiplication gives zero. Thus its eigenvalues are 2 and 0. **2. Compute squared singular values:** \(A^TA=\begin{pmatrix}2&2\\2&2\end{pmatrix}\) has eigenvalues 4 and 0. Singular values are their nonnegative square roots, 2 and 0. **3. Add the singular values:** \(\sigma_1+\sigma_2=2+0=2\). Here they also equal the eigenvalues of A because A is symmetric positive semidefinite. **Answer: 2.**

**1. Use the supplied matrix:** \(A=\begin{pmatrix}1&1\\1&1\end{pmatrix}\). Along (1,1), multiplication doubles the vector; along (1,-1), multiplication gives zero. Thus its eigenvalues are 2 and 0. **2. Determine rank and nullity:** Both columns are the same nonzero vector, leaving one independent column, so rank(A)=1. The rank-nullity theorem gives nullity(A)=2−1=1. **3. Check determinant and invertibility:** \(\det A=1\cdot1-1\cdot1=0\). A zero determinant, a nonzero null vector (1,-1), and rank below 2 all show that A is not invertible. **Answer: A, C — rank(A)=1 and det(A)=0.** A: **Correct:** There is exactly one independent nonzero column. B: **Incorrect:** The nullity is 1, not 0. C: **Correct:** The two determinant products cancel to zero. D: **Incorrect:** A is singular because its determinant is zero.

**1. Multiply the matrix:** For the supplied all-ones 2-by-2 matrix, \(A^TA=\begin{pmatrix}2&2\\2&2\end{pmatrix}\). **2. Find eigenvalues and trace:** Its action on (1,1) has eigenvalue 4 and on (1,-1) has eigenvalue 0. Its trace is 2+2=4, matching the sum of eigenvalues. **3. Test determinant and invertibility:** Its determinant is \(2\cdot2-2\cdot2=0\). The zero eigenvalue makes it singular; 4 is the trace and nonzero eigenvalue, not the determinant. **Answer: A, B — eigenvalues 4 and 0, and trace 4.** A: **Correct:** The eigendirections give eigenvalues 4 and 0. B: **Correct:** Adding the two diagonal entries gives 4. C: **Incorrect:** The determinant is 0, not 4. D: **Incorrect:** The zero eigenvalue means the matrix is not invertible.