Machine Learning Foundations · Quiz 2 · Sep 2025

**1. Expand a quadratic form:** For \(v=(x,y,z)^T\), the coefficient of \(xy\) in \(v^TAv\) is \(a_{12}+a_{21}\); similarly, the other mixed coefficients are sums of the two off-diagonal entries. A symmetric representation therefore uses half of each mixed coefficient. **2. Match every coefficient:** The squared terms give diagonal entries \(1,-1,2\). The terms \(2xy,-4yz,4xz\) give symmetric entries \(1,-2,2\), respectively. Thus \[A=\begin{pmatrix}1&1&2\\1&-1&-2\\2&-2&2\end{pmatrix}.\] **3. Verify by multiplication:** Expanding this matrix gives \(x^2-y^2+2z^2+2xy-4yz+4xz\), exactly the stated polynomial. **Answer: A — \(A=\begin{pmatrix}1&1&2\\1&-1&-2\\2&-2&2\end{pmatrix}\).** **Convention note:** A quadratic form does not uniquely determine an unrestricted matrix: adding any skew-symmetric matrix leaves the form unchanged. Of the listed matrices, only A has the required diagonal and off-diagonal sums. A: **Correct:** Its diagonal and all three off-diagonal sums match the polynomial. B: **Incorrect:** The yz coefficient is \(-1-2=-3\), rather than \(-4\). C: **Incorrect:** The y-squared coefficient is 1 rather than -1; the mixed coefficients also differ. D: **Incorrect:** The z-squared coefficient is 1 rather than 2; xy, xz and yz also fail to match.

**1. Find the eigenvalues:** For \(A=\begin{pmatrix}d&\sqrt2\\\sqrt2&d\end{pmatrix}\), vectors \((1,1)^T\) and \((1,-1)^T\) give eigenvalues \(d+\sqrt2\) and \(d-\sqrt2\). **2. Test positive definiteness:** A real symmetric matrix is positive definite exactly when all its eigenvalues are positive. Here this requires \(d-\sqrt2>0\), or \(d>\sqrt2\). When \(d<-\sqrt2\), both eigenvalues are negative instead. **3. Test invertibility:** \(\det A=d^2-2\), so singularity occurs only at \(d=\pm\sqrt2\). In particular, \(d=\sqrt2\) is singular, whereas every \(d>\sqrt2\) is invertible. **Answer: A, C — positive definiteness for d greater than square root of 2, and singularity at equality.** A: **Correct:** Both eigenvalues are positive when \(d>\sqrt2\). B: **Incorrect:** Both eigenvalues are negative in this range, making A negative definite. C: **Correct:** One eigenvalue is zero and the determinant vanishes. D: **Incorrect:** Both eigenvalues are nonzero and positive in this range.

**1. Use the defining SVD relation:** The supplied right and left singular vectors satisfy \(Av_1=\sigma_1u_1=5u_1\). The expression requested is \(v_1^TAv_1\), not \(u_1^TAv_1\). **2. Compute the inner product:** With \(v_1=(2,1)^T/\sqrt5\) and \(u_1=(1,2)^T/\sqrt5\), \(v_1^Tu_1=(2\cdot1+1\cdot2)/5=4/5\). **3. Substitute:** \[v_1^TAv_1=5v_1^Tu_1=5\cdot\frac45=4.\] The second singular value is unnecessary because the action of A on the specified right singular vector is already known. **Answer: 4.**

**1. Use the fixed sum:** The identity \(x^3+y^3=(x+y)^3-3xy(x+y)\) becomes \(4096-48xy\) when \(x+y=16\). Minimizing the cubic sum is therefore equivalent to maximizing xy. **2. Bound the product for real numbers:** Since \((x-y)^2\ge0\), \(4xy\le(x+y)^2=256\), giving \(xy\le64\). Equality is attained at \(x=y=8\). **3. Report the requested quantity:** At this point the cubic sum is \(8^3+8^3=1024\), but the question asks for xy, which is 64. **Answer: 64.**

**1. Recall the rectangular SVD:** Every real \(m\times n\) matrix has a decomposition \(A=U\Sigma V^T\), with full-form dimensions \(m\times m\), \(m\times n\), and \(n\times n\), respectively. Thus A need not be square; for complex matrices, use the conjugate transpose \(V^*\). **Answer: B — False.** A: **Incorrect:** Rectangular matrices also possess an SVD, so the word “only” makes this statement false. B: **Correct:** The SVD exists for rectangular as well as square matrices.

**1. Set up two eigenpairs:** Let \(Au=\lambda u\) and \(Av=\mu v\), where \(\lambda\ne\mu\). Since A is Hermitian, \(A^*=A\) and both eigenvalues are real. **2. Evaluate the same product two ways:** \(u^*Av=\mu u^*v\), while \(u^*Av=(Au)^*v=\lambda u^*v\). Subtracting gives \((\lambda-\mu)u^*v=0\). **3. Conclude orthogonality:** Distinct eigenvalues imply \(u^*v=0\), which is the complex inner-product definition of orthogonality. **Answer: A — True.** A: **Correct:** Hermitian symmetry and distinct real eigenvalues force the inner product to vanish. B: **Incorrect:** The stated orthogonality theorem holds for every Hermitian matrix.

**1. Enforce column orthogonality:** The columns of \(U=\tfrac12\begin{pmatrix}1+i&x\\1-i&y\end{pmatrix}\) must have zero Hermitian inner product. Conjugating the first column gives \((1-i)x+(1+i)y=0\). **2. Solve the ratio:** The second column must have norm 1, so it cannot be zero; the orthogonality equation also shows y cannot be zero. Therefore \[\frac{x}{y}=-\frac{1+i}{1-i}=-i.\] **3. Identify the real parameters:** Comparing \(x/y=a+ib\) with \(-i=0+i(-1)\) gives \(a=0\) and \(b=-1\). **Answer: 0.**

**1. Enforce column orthogonality:** The columns of \(U=\tfrac12\begin{pmatrix}1+i&x\\1-i&y\end{pmatrix}\) must have zero Hermitian inner product. Conjugating the first column gives \((1-i)x+(1+i)y=0\). **2. Solve the ratio:** The second column must have norm 1, so it cannot be zero; the orthogonality equation also shows y cannot be zero. Therefore \[\frac{x}{y}=-\frac{1+i}{1-i}=-i.\] **3. Identify the real parameters:** Comparing \(x/y=a+ib\) with \(-i=0+i(-1)\) gives \(a=0\) and \(b=-1\). **Answer: -1.**

**1. Determine the permissible second columns:** Orthogonality to the fixed first column forces \(x=-iy\). Unit norm then requires \((|x|^2+|y|^2)/4=1\), so \(|y|=\sqrt2\). **2. Exhibit the whole family:** For any real \(\theta\), choose \(y=\sqrt2e^{i\theta}\) and \(x=-i\sqrt2e^{i\theta}\). These satisfy both orthogonality and normalization, so each choice defines a unitary matrix. **3. Count distinct matrices:** Distinct phases modulo \(2\pi\) give infinitely many second columns, even though their ratio x/y is fixed. **Answer: A — S is an infinite set.** A: **Correct:** A continuous phase parameter produces infinitely many valid matrices. B: **Incorrect:** The possible phases are not restricted to a finite list. C: **Incorrect:** For example y equal to square root of 2 and x equal to minus i times square root of 2 gives a valid matrix. D: **Incorrect:** The second column may be multiplied by any unit complex phase.

**1. Find eigenvalues using symmetric directions:** For \(A=\tfrac14\begin{pmatrix}3&-1\\-1&3\end{pmatrix}\), \(A(1,1)^T=\tfrac12(1,1)^T\), while \(A(1,-1)^T=(1,-1)^T\). Thus its eigenvalues are \(1/2,1\). **2. Convert to singular values:** Because A is real symmetric, eigenvalues of \(A^TA=A^2\) are \(1/4,1\). Taking their nonnegative square roots gives singular values \(1/2,1\). **Answer: A — \(1,\tfrac12\).** A: **Correct:** The pair is exactly \(1,1/2\). B: **Incorrect:** The determinant is \(1/2\), so a zero singular value is impossible. C: **Incorrect:** Singular values cannot be negative, and 2 is not a singular value of this matrix.

**1. State the general relation:** A real symmetric matrix has an orthogonal diagonalization \(B=Q\Lambda Q^T\). Its singular values are the absolute values of its eigenvalues, not necessarily the signed eigenvalues themselves. **2. Disprove both statements with one example:** Take the symmetric matrix \(B=[-1]\). Its only eigenvalue is -1 and its only singular value is 1. Thus an eigenvalue need not itself be a singular value (P fails), and a singular value need not itself be an eigenvalue (Q fails). **3. Distinguish the special case:** Both claims would hold for a positive semidefinite symmetric matrix, but the question assumes only symmetry. **Answer: D — both P and Q are false.** A: **Incorrect:** P fails for an eigenvalue of -1. B: **Incorrect:** Q fails because a singular value of 1 need not be an eigenvalue of B. C: **Incorrect:** Symmetry alone allows negative eigenvalues, so both claims cannot be asserted. D: **Correct:** The symmetric one-by-one matrix [-1] disproves P and Q.

**1. Compute the update rule:** For \(f(x,y)=x^2+y^2\), \(\nabla f=(2x,2y)\). Hence gradient descent with constant rate \(\eta\) gives \((x_{t+1},y_{t+1})=(1-2\eta)(x_t,y_t)\). **2. Apply the first step:** At \(\eta=0.2\), the multiplier is \(1-0.4=0.6\), so \((x_1,y_1)=0.6(x_0,y_0)\). **3. Apply the second step:** \((x_2,y_2)=0.6^2(x_0,y_0)=0.36(x_0,y_0)\). Therefore the requested coefficient is \(\alpha=0.36\). **Answer: 0.36.**

**1. Compute the update rule:** For \(f(x,y)=x^2+y^2\), \(\nabla f=(2x,2y)\). Hence gradient descent with constant rate \(\eta\) gives \((x_{t+1},y_{t+1})=(1-2\eta)(x_t,y_t)\). **2. Require arrival at the minimum:** The unique minimum is the origin. Because the initial point is nonzero, one step reaches it precisely when \(1-2\eta=0\). **3. Solve for the rate:** \(2\eta=1\), so \(\eta=0.5\). The initial point is not already optimal, which ensures this is exactly one step. **Answer: 0.5.**

**1. Compute the update rule:** For \(f(x,y)=x^2+y^2\), \(\nabla f=(2x,2y)\). Hence gradient descent with constant rate \(\eta\) gives \((x_{t+1},y_{t+1})=(1-2\eta)(x_t,y_t)\). **2. Derive the convergence condition:** Repeated updates give \(p_t=(1-2\eta)^tp_0\). For nonzero \(p_0\), its norm tends to zero exactly when \(|1-2\eta|<1\), equivalent to \(0<\eta<1\). **3. Check the boundary and oscillation:** Rate 0.9 gives a negative factor but shrinking norm, so oscillation is allowed. Rate 1 gives factor -1 and no decrease in distance; rate 1.3 gives magnitude 1.6 and divergence. **Answer: A, B — learning rates 0.3 and 0.9.** A: **Correct:** The multiplier is 0.4, so distances shrink geometrically. B: **Correct:** The multiplier is -0.8; signs alternate but distances shrink. C: **Incorrect:** The multiplier is -1, preserving the nonzero distance forever. D: **Incorrect:** The multiplier is -1.6, so the distance grows.

**1. Center the five observations:** The points are \((-2,-2),(-1,-1),(0,0),(1,1),(2,2)\). Their mean is \((0,0)\), so they are already centered. **2. Compute the population covariance:** Each diagonal entry and the cross-covariance equal \((4+1+0+1+4)/5=2\). Thus \[C=\begin{pmatrix}2&2\\2&2\end{pmatrix}.\] **3. Read the requested parameter:** Every entry of the stated matrix is a, so \(a=2\). **Answer: 2.** **Convention note:** This paper uses the PCA population covariance with divisor n=5. The unbiased sample covariance with divisor n−1 would have entries 2.5 instead.

**1. Center the five observations:** The points are \((-2,-2),(-1,-1),(0,0),(1,1),(2,2)\). Their mean is \((0,0)\), so they are already centered. **2. Compute the population covariance:** Each diagonal entry and the cross-covariance equal \((4+1+0+1+4)/5=2\). Thus \[C=\begin{pmatrix}2&2\\2&2\end{pmatrix}.\] **3. Find the principal eigenpair:** The unit vector \(v=(1,1)^T/\sqrt2\) has eigenvalue 4; the perpendicular direction has eigenvalue 0. Either sign of v is valid. **4. Evaluate the quadratic form:** Since \(Cv=4v\) and \(v^Tv=1\), \(v^TCv=4v^Tv=4\). **Answer: 4.** **Convention note:** The shared dataset is supplied in the MLF passage of this bundle; covariance is normalized by n=5.

**1. Center the five observations:** The points are \((-2,-2),(-1,-1),(0,0),(1,1),(2,2)\). Their mean is \((0,0)\), so they are already centered. **2. Compute the population covariance:** Each diagonal entry and the cross-covariance equal \((4+1+0+1+4)/5=2\). Thus \[C=\begin{pmatrix}2&2\\2&2\end{pmatrix}.\] **3. Recompute after adding the two points:** Adding \((100,100)\) and \((-100,-100)\) leaves the mean zero. All seven points remain on the same line, and \(C^\prime=\tfrac{20010}{7}\begin{pmatrix}1&1\\1&1\end{pmatrix}\). **4. Compare principal directions:** Both covariance matrices have the same unique leading eigenspace, the span of \((1,1)^T\). A unit vector in this one-dimensional space can have either sign, so \(v^\prime=v\) or \(v^\prime=-v\). The magnitude of the variance changes, but the principal axis does not. **Answer: A — \(v^\prime=v\) or \(v^\prime=-v\).** **Convention note:** The dataset needed for this question is present in the shared MLF passage in this bundle. Eigenvector sign has no effect on the PCA axis. A: **Correct:** Both vectors lie on the same one-dimensional principal axis, with arbitrary sign. B: **Incorrect:** The perpendicular axis has zero variance, so it cannot become the leading direction. C: **Incorrect:** The principal axes coincide; a 45-degree angle is impossible. D: **Incorrect:** The exact line containing every point determines the leading eigenspace.