Machine Learning Practice · End Term · Jan 2026 · FN

**1. Analyze the `cross_val_score` code snippet:** We have `X = np.arange(20).reshape((10, 2))` (which has \(N = 10\) samples and \(2\) features) and `y = np.array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1])` (10 labels: 5 zeros and 5 ones). We run `cross_val_score(model, X, y, cv=5, scoring='accuracy')`. **2. Evaluate each statement:** - **Statement A:** With \(N = 10\) samples and `cv=5` (5-fold cross-validation), the dataset is partitioned into 5 equal folds of \(10 / 5 = 2\) samples each. In each iteration, 4 folds (\(4 \times 2 = 8\) samples) are used for training and 1 fold (\(2\) samples) is used for testing. (Correct!) - **Statement B:** In cross-validation, the model is trained on the training fold and evaluated (`scoring='accuracy'`) on the held-out **test (validation) fold**, not the training set. (Incorrect.) - **Statement C:** If we change `cv=2` (2-fold cross-validation), the 10 samples are split into 2 equal folds of \(10 / 2 = 5\) samples each, so each fold has exactly 5 samples in the test set. (Correct!) - **Statement D:** `cross_val_score` does **not** perform any automatic feature scaling or normalization unless the passed estimator is a `Pipeline` containing a scaler. (Incorrect.) **Answer: A, C — Each fold with cv=5 uses 8 training and 2 testing samples, and changing to cv=2 puts 5 samples in each test fold.** A: **Correct:** Splitting 10 samples into `cv=5` equal folds puts \(10/5 = 2\) samples in the test fold and the remaining \(10 - 2 = 8\) samples in the training fold. B: **Incorrect:** `cross_val_score` computes the evaluation metric on the held-out validation/test fold in each split, never on the training fold. C: **Correct:** With 10 samples and `cv=2`, each of the 2 folds contains exactly \(10/2 = 5\) test samples. D: **Incorrect:** `cross_val_score` passes the raw slices of `X` directly to `model.fit` and `model.score` without normalizing features.

**1. Note that the question asks: "Which of the following are NOT true?"** We are given a `VotingClassifier` with `estimators=[('lr', clf1), ('dt', clf2), ('svc', clf3)]`, `voting='soft'`, and `weights=[2, 1, 2]`, where `clf3 = SVC(probability=False)`. **2. Check the truth value of each statement:** - **Statement A:** With `weights=[2, 1, 2]`, `'lr'` has weight `2` and `'dt'` has weight `1`, so Logistic Regression has twice the weight (influence) of Decision Tree in the weighted average. This statement is **true** (so do not select it). - **Statement B:** Soft voting requires every base estimator to expose `predict_proba`. In `scikit-learn`, `SVC(probability=False)` does **not** enable probability estimation (`predict_proba` is unavailable), so `SVC` with `probability=False` cannot contribute to soft voting (calling `.fit` / `.predict` raises an `AttributeError`). This statement is **true** (so do not select it). - **Statement C:** *"Changing `weights=[1,1,1]` will not affect the predictions if all models have the same accuracy."* Even if three models have the same overall accuracy on a dataset, they can output different class probability distributions on individual test samples, so changing the weights from `[2, 1, 2]` to `[1, 1, 1]` **can** change individual sample predictions! Thus, Statement C is **NOT true** (select it). - **Statement D:** *"Soft voting ignores classifier probabilities and only counts majority vote."* This describes *hard* voting (`voting='hard'`), not soft voting (`voting='soft'` averages the predicted probabilities). Thus, Statement D is **NOT true** (select it). **Answer: C, D — Statements C and D are not true.** A: **Incorrect:** This statement is true (so do not select it): `weights=[2, 1, 2]` gives `'lr'` (weight 2) twice the weight of `'dt'` (weight 1). B: **Incorrect:** This statement is true (so do not select it): `SVC` disables `predict_proba` when `probability=False`, which is required for `voting='soft'`. C: **Correct:** This statement is false (so select it): models with equal overall accuracy can still assign different probabilities to individual test samples, so changing `weights` can alter predictions. D: **Correct:** This statement is false (so select it): soft voting averages predicted class probabilities (`predict_proba`), whereas hard voting counts majority class votes.

**1. Analyze the `Pipeline` with `StandardScaler` and `SGDRegressor`:** The pipeline scales features with `StandardScaler()` and fits `SGDRegressor(loss="squared_error", penalty="l2", alpha=0.01, learning_rate="adaptive", eta0=0.1, max_iter=1000, tol=1e-3, random_state=42)`. **2. Evaluate each statement:** - **Statement A:** Stochastic Gradient Descent (`SGDRegressor`) updates weights using gradients proportional to the feature values \(x_{ij}\). Without feature scaling, features with large scales produce steep gradients and require tiny learning rates, leading to slow or unstable convergence. Thus, feature scaling is essential. (Correct!) - **Statement B:** In `scikit-learn`, `learning_rate="adaptive"` keeps the learning rate **constant** equal to `eta0` as long as the training loss keeps decreasing (by at least `tol`), and only divides the learning rate by 5 when the stopping criterion fails to improve across consecutive epochs. It does *not* decrease at every iteration regardless of performance. (Incorrect.) - **Statement C:** The parameter `tol=1e-3` defines the stopping criterion: if the loss improvement is less than `tol` for `n_iter_no_change` epochs (and under `'adaptive'`, once the learning rate drops below `1e-6`), training stops early before reaching `max_iter=1000`. (Correct!) - **Statement D:** Making the regularization strength `alpha` arbitrarily large forces all weights toward zero, causing severe **underfitting** (high bias) and degrading generalization. (Incorrect.) **Answer: A, C — Feature scaling is essential for gradient-based updates in SGDRegressor, and training may stop early before max_iter=1000 when the tolerance criterion is met.** A: **Correct:** Gradient descent is sensitive to feature scales; standardizing features makes the loss surface better conditioned and speeds up SGD convergence. B: **Incorrect:** `learning_rate='adaptive'` keeps the step size at `eta0` while training loss is improving and only reduces it by a factor of 5 when progress stalls. C: **Correct:** `max_iter=1000` is only the maximum cap on epochs; convergence checking via `tol=1e-3` allows training to terminate early once the loss stabilizes. D: **Incorrect:** Excessively large `alpha` over-penalizes the weights, causing underfitting and poor test generalization.

**1. Recall the formula for the \(R^2\) (coefficient of determination) score:** Given true values \(y_{\text{true}} = [2, 4, 6, 8, 10]\) and predicted values \(y_{\text{pred}} = [3, 5, 5, 9, 11]\), `r2_score(y_true, y_pred)` computes: \[R^2 = 1 - \frac{\text{SS}_{\text{res}}}{\text{SS}_{\text{tot}}} = 1 - \frac{\sum_{i=1}^n (y_i - \hat{y}_i)^2}{\sum_{i=1}^n (y_i - \bar{y})^2}\] **2. Compute the Residual Sum of Squares (\(\text{SS}_{\text{res}}\)):** For each of the 5 samples, the error \(y_i - \hat{y}_i\) has magnitude \(1\): - \(i=1: (2 - 3)^2 = (-1)^2 = 1\) - \(i=2: (4 - 5)^2 = (-1)^2 = 1\) - \(i=3: (6 - 5)^2 = (1)^2 = 1\) - \(i=4: (8 - 9)^2 = (-1)^2 = 1\) - \(i=5: (10 - 11)^2 = (-1)^2 = 1\) Summing these gives \(\text{SS}_{\text{res}} = 1 + 1 + 1 + 1 + 1 = 5\). **3. Compute the Total Sum of Squares (\(\text{SS}_{\text{tot}}\)) and \(R^2\):** The mean of \(y_{\text{true}}\) is \(\bar{y} = \frac{2 + 4 + 6 + 8 + 10}{5} = 6\). \[\text{SS}_{\text{tot}} = (2 - 6)^2 + (4 - 6)^2 + (6 - 6)^2 + (8 - 6)^2 + (10 - 6)^2 = 16 + 4 + 0 + 4 + 16 = 40\] Substituting \(\text{SS}_{\text{res}} = 5\) and \(\text{SS}_{\text{tot}} = 40\): \[R^2 = 1 - \frac{5}{40} = 1 - 0.125 = 0.875 \approx 0.88\] **Answer: 0.88.**

**1. Count the number of hyperparameter combinations in `param_grid`:** We are given: - `"max_depth": [None, 5, 10]` \(\implies 3\) values - `"min_samples_split": [2, 4]` \(\implies 2\) values Taking the Cartesian product gives \(3 \times 2 = 6\) candidate hyperparameter combinations. **2. Account for `cv=4` cross-validation splits:** For each of the \(6\) hyperparameter combinations, `GridSearchCV` performs \(4\)-fold cross-validation, training the decision tree once on each of the \(4\) training folds: \[6 \text{ combinations} \times 4 \text{ folds} = 24 \text{ model trainings during CV}\] **Answer: 24.**

**1. Understand how `GridSearchCV` interacts with a `Pipeline`:** In the given code, `pipe = Pipeline([("scaler", StandardScaler()), ("lr", LinearRegression())])` is passed as the estimator inside `GridSearchCV(pipe, param_grid=param_grid, cv=5)`. **2. Trace what happens during each cross-validation split:** Because `StandardScaler()` is **inside** the `Pipeline`, during each of the 5 cross-validation folds: 1. `GridSearchCV` splits `X_train` into a fold-specific training subset and validation subset. 2. It calls `pipe.fit(X_fold_train, y_fold_train)`, which fits `StandardScaler` **only on `X_fold_train`** (recomputing the mean and standard deviation independently in each CV fold). 3. It then transforms `X_fold_val` using those fold-specific scaling parameters before scoring `LinearRegression`. This completely prevents data leakage between validation folds and training folds. **Answer: B — Scaling parameters are recomputed independently in each CV fold.** A: **Incorrect:** `StandardScaler` is inside the `Pipeline`, so it is fitted separately on each CV training fold rather than once on the full `X_train` beforehand (only the final `refit=True` step fits on the full `X_train` after CV completes). B: **Correct:** Wrapping `StandardScaler` inside the `Pipeline` ensures that in every CV split, the mean and variance are computed solely from that fold's training portion. C: **Incorrect:** Putting `StandardScaler` inside the `Pipeline` passed to `GridSearchCV` is the exact best practice that *prevents* data leakage. D: **Incorrect:** Scaling is applied inside every single cross-validation fold before fitting and evaluating `LinearRegression`.

**1. Trace the image shapes and types through each line of the Python code:** - `image_0 = np.random.randint(0, 256, size=(128, 128, 3), dtype=np.uint8)` Creates a 3-channel RGB NumPy array of height `128`, width `128`, and pixel values in `0..255`. - `image_1 = Image.fromarray(image_0)` Converts the NumPy array into a `PIL.Image.Image` object (not a NumPy `ndarray`). - `image_2 = image_1.resize((32, 64))` Resizes the PIL image to width `32` and height `64` (`32 x 64 = 2048` pixels). - `image_3 = image_2.convert('L')` Converts the RGB image to a single-channel (`'L'` = 8-bit grayscale) PIL image of size `32 x 64` (whose NumPy array shape is `(64, 32)`). - `image_4 = np.array(image_3).flatten()` Converts the `64 x 32` single-channel grayscale image to a 2D NumPy array of shape `(64, 32)` and flattens it into a 1D array of length \(64 \times 32 = 2048\), so `image_4.shape` is `(2048,)`. **2. Check the options:** - Option A is false because Python uses 0-based indexing, so the *first* color channel is `image_0[:, :, 0]`, whereas `image_0[:, :, 1]` is the *second* channel. - Option B is false because `np.random.randint(0, 256)` generates integers up to `255` (not `128`). - Option C is false because `image_1` is a `PIL.Image.Image` object, not a `numpy.ndarray`. - Option D is **true** because `image_4` is a flattened 1D array of \(32 \times 64 = 2048\) grayscale pixels, having shape `(2048,)`. **Answer: D — The shape of image_4 is (2048,).** A: **Incorrect:** Array indexing in NumPy is 0-based, so `image_0[:, :, 1]` extracts the *second* color channel; the first color channel is `image_0[:, :, 0]`. B: **Incorrect:** `np.random.randint(0, 256)` samples integers in `[0, 255]`, so the maximum possible intensity is `255` (`128` is the spatial height/width). C: **Incorrect:** `Image.fromarray(image_0)` returns a `PIL.Image.Image` instance, not a NumPy `ndarray`. D: **Correct:** Resizing to `(32, 64)` and converting to single-channel grayscale `'L'` leaves \(32 \times 64 = 2048\) pixels, so `.flatten()` produces a 1D array of shape `(2048,)`.

**1. Evaluate Assertion (A):** *"In a medical diagnosis system to identify highly fatal diseases, recall is more important than precision. (Presence of a disease is treated as positive class)"* When diagnosing a fatal disease, failing to detect a sick patient is a **False Negative (FN)**, which can cost the patient's life. Since \(\text{Recall} = \frac{\text{TP}}{\text{TP} + \text{FN}}\), achieving high recall is critical to ensure that almost no sick patients are missed. Thus, **Assertion (A) is true**. **2. Evaluate Reason (R):** *"High recall ensures that the model makes fewer false positive predictions."* Recall is \(\frac{\text{TP}}{\text{TP} + \text{FN}}\), which measures how well the model avoids **False Negatives (FN)**, not False Positives (FP)! Avoiding False Positives is measured by **Precision** (\(\frac{\text{TP}}{\text{TP} + \text{FP}}\)). Thus, **Reason (R) is false**. **Answer: C — A is true, but R is false.** A: **Incorrect:** Reason (R) is false because high recall minimizes False Negatives, not False Positives. B: **Incorrect:** Reason (R) is factually incorrect—Precision, not Recall, controls False Positives. C: **Correct:** High recall is vital in fatal disease screening to avoid missing sick patients (A is true), whereas recall reduces False Negatives rather than False Positives (R is false). D: **Incorrect:** Assertion (A) is true and Reason (R) is false. E: **Incorrect:** Assertion (A) is true because missing a fatal disease (False Negative) has severe consequences.

**1. Understand the Augmented Dickey-Fuller (`adfuller`) test:** In time-series econometrics and forecasting (such as before fitting an ARIMA model), the Augmented Dickey-Fuller (ADF) test (`statsmodels.tsa.stattools.adfuller`) is a formal statistical test for a **unit root** in a time series sample. **2. State the null and alternative hypotheses of the ADF test:** - **Null hypothesis (\(H_0\)):** The time series possesses a **unit root**, meaning it is **non-stationary** (has a stochastic trend). - **Alternative hypothesis (\(H_1\)):** The time series does not have a unit root and is **stationary** (or trend-stationary). Thus, the primary objective of `adfuller()` is to determine whether the time series possesses a unit root indicating non-stationarity. **Answer: C — The time series possesses a unit root indicating non-stationarity.** A: **Incorrect:** Testing for autocorrelation across multiple lags is the purpose of the Ljung-Box (`acorr_ljungbox`) test or ACF plot, not the ADF test. B: **Incorrect:** Testing for heteroscedastic (time-varying) variance is done using ARCH/Breusch-Pagan/White tests, not the ADF test. C: **Correct:** The Augmented Dickey-Fuller (`adfuller`) test specifically tests the null hypothesis that a time series has a unit root (non-stationarity). D: **Incorrect:** Normality is tested using Jarque-Bera or Shapiro-Wilk tests, not `adfuller()`.

**1. Understand how `GaussianNB` computes class-conditional likelihoods:** In Gaussian Naive Bayes, the likelihood of a continuous feature \(x_j\) given class \(c\) is modeled using a univariate Gaussian density with class-specific mean \(\mu_{cj}\) and variance \(\sigma_{cj}^2\): \[p(x_j \mid y = c) = \frac{1}{\sqrt{2\pi \sigma_{cj}^2}} \exp\left( -\frac{(x_j - \mu_{cj})^2}{2\sigma_{cj}^2} \right)\] Taking the log-likelihood gives: \[\ln p(x_j \mid y = c) = -\frac{1}{2}\ln(2\pi \sigma_{cj}^2) - \frac{(x_j - \mu_{cj})^2}{2\sigma_{cj}^2}\] **2. Analyze the impact of very small vs. large feature variance:** - Notice that the variance \(\sigma_{cj}^2\) appears in the **denominator** of the quadratic penalty \(-\frac{(x_j - \mu_{cj})^2}{2\sigma_{cj}^2}\) and in the peak height \(\frac{1}{\sqrt{2\pi \sigma_{cj}^2}}\). - When a feature has a **very small variance** (\(\sigma_{cj}^2 \ll 1\)), its Gaussian density has a very tall, sharp likelihood peak at \(\mu_{cj}\), and even a tiny deviation \(x_j - \mu_{cj}\) gets divided by the tiny \(\sigma_{cj}^2\), producing huge swings in the log-likelihood that can dominate the posterior probability calculation (which is why `GaussianNB` includes a `var_smoothing` hyperparameter). **Answer: C — Features with very small variance can dominate due to sharper likelihood peaks.** A: **Incorrect:** `GaussianNB` does not fail on unstandardized features; it estimates \(\mu_{cj}\) and \(\sigma_{cj}^2\) for each feature automatically. B: **Incorrect:** A very large variance \(\sigma_{cj}^2\) makes the Gaussian density very flat and wide, resulting in *smaller* likelihood peak values and weaker influence on log-odds differences. C: **Correct:** Because \(\sigma_{cj}^2\) sits in the denominator of both the density peak \(1/\sqrt{2\pi\sigma_{cj}^2}\) and the exponent \(-(x_j - \mu_{cj})^2/(2\sigma_{cj}^2)\), features with very small variance have sharp peaks and large log-likelihood terms that can dominate predictions. D: **Incorrect:** `GaussianNB` explicitly estimates and uses the variance \(\sigma_{cj}^2\) of every feature in every class (`model.var_`).

**1. Compare the hyperparameters of `rf1` and `rf2`:** - **`rf1`:** `max_depth=None` (deep unpruned trees), `max_features="sqrt"` (at each split, only a random subset of \(\sqrt{d}\) features is considered). - **`rf2`:** `max_depth=5` (shallow trees), `max_features=None` (at each split, **all \(d\) features** are considered; this is pure bagging without random feature subspace selection). **2. Understand how `max_features` affects tree correlation in Random Forests:** - When `max_features=None` (`rf2`), every tree considers **all features** at every split. If the dataset has a few strong predictor features, almost every bootstrap tree will pick the exact same top features near the root, making the trees in `rf2` highly similar (**higher tree correlation**). - By restricting each split to a random subset `max_features="sqrt"`, `rf1` decorrelates the trees so that averaging them achieves a much stronger reduction in variance. **Answer: B — rf2 is more likely to suffer from higher tree correlation.** A: **Incorrect:** Although `rf1` has `max_depth=None`, on small datasets where `max_depth=5` already reaches pure leaves, both can achieve `0` training error (so `rf1` is not strictly guaranteed to *always* have lower training error than `rf2`). B: **Correct:** Setting `max_features=None` in `rf2` allows every tree to evaluate all features at every split, causing trees to repeatedly pick the same dominant features and increasing inter-tree correlation. C: **Incorrect:** `rf1` sets `max_depth=None` (unrestricted depth), and restricting tree depth increases bias rather than reducing it. D: **Incorrect:** `rf2` has *shallower* trees (`max_depth=5`) than `rf1` (`max_depth=None`), not deeper trees.

**1. Trace the pandas DataFrame indexing and assignment:** We start with: ```python df = pd.DataFrame({"A": [10, 20, 30, 40], "B": [1, 2, 3, 4]}) subset = df[df["A"] > 20] subset["B"] = subset["B"] * 2 ``` - The boolean mask `df["A"] > 20` filters rows where column `"A"` is `30` or `40`, returning a new filtered DataFrame assigned to `subset`. - Modifying `subset["B"] = subset["B"] * 2` updates column `"B"` on the separate `subset` object (often triggering a `SettingWithCopyWarning` in older pandas versions, or modifying a copy under Copy-on-Write), while leaving the original DataFrame `df` **completely unchanged** (`df["B"]` is still `[1, 2, 3, 4]`). **Answer: B — Column B in df remains unchanged.** A: **Incorrect:** Boolean indexing `df[df['A'] > 20]` creates a separate filtered object `subset`, so modifying `subset['B']` does not mutate `df['B']`. B: **Correct:** Because `subset` is the result of boolean filtering rather than an in-place `.loc` assignment on `df`, modifying `subset['B']` leaves `df['B']` unchanged. C: **Incorrect:** Pandas may emit a `SettingWithCopyWarning` (warning, not an exception), so execution does not stop with an error. D: **Incorrect:** Option B accurately describes the behavior.

**1. Evaluate each of the 4 numbered statements on bias and variance:** - **Statement 1:** *"Increasing the training dataset size generally reduces variance but may not significantly reduce bias."* **True!** More training samples reduce estimation variance (since sample averages converge), but if a model family is fundamentally too simple (e.g., fitting a line to a quadratic curve), its asymptotic approximation bias remains high. - **Statement 2:** *"A model with low training error and high test error necessarily has high bias."* **False!** Low training error combined with high test error is the textbook definition of **overfitting (high variance, low bias)**, not high bias. - **Statement 3:** *"Regularization techniques primarily aim to reduce variance at the cost of introducing some bias."* **True!** Penalizing model complexity (e.g., Ridge/Lasso) shrinks weights, trading a small increase in bias for a substantial drop in variance. - **Statement 4:** *"An ensemble of high-bias models (e.g., shallow trees) always results in a low-bias predictor."* **False!** Simply averaging high-bias models (as in Bagging) keeps the bias high; only sequential boosting methods reduce bias, so an ensemble of high-bias models does not *always* have low bias. **2. Conclude:** Statements **1 and 3** are correct. **Answer: A — 1 and 3.** A: **Correct:** Statement 1 (more data reduces variance, not model bias) and Statement 3 (regularization reduces variance at the cost of some bias) are both true. B: **Incorrect:** Statement 2 is false because low training error and high test error indicates high variance (overfitting), not high bias. C: **Incorrect:** Both Statement 2 and Statement 4 are false (averaging high-bias models in bagging retains high bias). D: **Incorrect:** Statement 2 is false. E: **Incorrect:** Statement 3 is also true in addition to Statement 1.

**1. Examine the formula implemented in the code snippet:** The code computes: ```python numerator = np.var(y_true - y_pred) denominator = np.var(y_true) metric = 1 - numerator / denominator ``` Writing this in mathematical notation gives: \[\text{metric} = 1 - \frac{\text{Var}(y_{\text{true}} - y_{\text{pred}})}{\text{Var}(y_{\text{true}})}\] **2. Distinguish between Explained Variance Score and \(R^2\) Score:** - **Explained Variance Score (`explained_variance_score`):** Uses the **variance** of the residuals in the numerator: \[\text{EVS} = 1 - \frac{\text{Var}(y - \hat{y})}{\text{Var}(y)}\] Because `np.var(y_true - y_pred)` subtracts the mean residual \(\overline{y - \hat{y}}\), this is the exact formula for the **Explained Variance Score**. - **\(R^2\) Score (`r2_score`):** Uses the uncentered mean squared error \(\frac{1}{n}\sum (y_i - \hat{y}_i)^2\) in the numerator: \(1 - \frac{\text{MSE}(y, \hat{y})}{\text{Var}(y)}\). They only coincide when the mean residual is zero. **Answer: C — Explained Variance Score.** A: **Incorrect:** Recall score is a classification metric (\(\text{TP}/(\text{TP}+\text{FN})\)), not a variance ratio on continuous targets. B: **Incorrect:** The \(R^2\) score uses `np.mean((y_true - y_pred)**2)` in the numerator rather than `np.var(y_true - y_pred)`. C: **Correct:** The formula \(1 - \frac{\text{Var}(y_{\text{true}} - y_{\text{pred}})}{\text{Var}(y_{\text{true}})}\) is the exact definition of `explained_variance_score` in `scikit-learn`. D: **Incorrect:** Silhouette score is an unsupervised clustering metric based on intra-cluster and nearest-cluster distances.

**1. Trace the DataFrame creation and `.iloc` indexing:** The DataFrame `df` is constructed from three dictionary rows with columns `'a'` (index 0), `'b'` (index 1), `'c'` (index 2), and `'d'` (index 3): \[\begin{array}{c|cccc} \text{Row} & \text{'a' (0)} & \text{'b' (1)} & \text{'c' (2)} & \text{'d' (3)} \\ \hline 0 & 1 & 2 & 3 & 4 \\ 1 & 100 & 200 & 300 & 400 \\ 2 & 1000 & 2000 & 3000 & 4000 \end{array}\] **2. Evaluate `result = df.iloc[:, lambda df: [0, 2]]`:** - `.iloc[:, [0, 2]]` selects all rows (`:`) and the columns at integer positions `0` (`'a'`) and `2` (`'c'`). - The values stored in `result` are therefore: - Column `'a'`: `1, 100, 1000` - Column `'c'`: `3, 300, 3000` **3. Find the maximum value in `result`:** The largest number across columns `'a'` and `'c'` is **3000** (located at row 2, column `'c'`). **Answer: D — 3000.** A: **Incorrect:** `1000` is the maximum of column `'a'` (index 0), but column `'c'` (index 2) is also selected and contains `3000`. B: **Incorrect:** `4000` is in column `'d'` (index 3), which is excluded by `[0, 2]`. C: **Incorrect:** `400` is in column `'d'` (index 3), row 1, which is not selected. D: **Correct:** Selecting columns at integer positions `0` (`'a'`) and `2` (`'c'`) yields values up to `3000` in row 2, column `'c'`.

**1. Recall the mathematical definition of Cosine Similarity:** To compare the semantic similarity of two word embedding vectors \(x\) and \(y\), we use **cosine similarity**, defined as the dot product of the two vectors divided by the product of their Euclidean (L2) norms: \[\text{cosine\_similarity}(x, y) = \frac{x \cdot y}{\|x\|_2 \,\|y\|_2} = \frac{\sum_i x_i y_i}{\sqrt{\sum_i x_i^2} \, \sqrt{\sum_i y_i^2}}\] **2. Compare the four NumPy implementations:** - **Option A:** Divides `np.dot(x, y)` by the vector sum `(x + y)`, which is elementwise division by a vector, not a scalar similarity metric. - **Option B:** Computes `dot_product = np.sum(x * y)`, `x_norm = np.sqrt(np.sum(x ** 2))`, `y_norm = np.sqrt(np.sum(y ** 2))`, and returns `dot_product / (x_norm * y_norm)`. This matches the exact cosine similarity formula! - **Option C:** Divides by the sum of squared norms `np.sum(x ** 2) + np.sum(y ** 2)` without square roots or multiplication. - **Option D:** Omits `np.sqrt(...)` when computing `x_norm` and `y_norm`, dividing by the product of squared norms \(\|x\|_2^2 \|y\|_2^2\) instead of \(\|x\|_2 \|y\|_2\). **Answer: B — dot_product / (np.sqrt(np.sum(x ** 2)) * np.sqrt(np.sum(y ** 2))).** A: **Incorrect:** Dividing the scalar dot product by the vector `(x + y)` produces a vector rather than the cosine of the angle between `x` and `y`. B: **Correct:** This code accurately computes the inner product \(\sum x_i y_i\) divided by the product of the L2 norms \(\sqrt{\sum x_i^2}\sqrt{\sum y_i^2}\). C: **Incorrect:** The denominator adds the squared norms instead of multiplying the square roots of the squared norms. D: **Incorrect:** `x_norm` and `y_norm` are missing `np.sqrt(...)`, so this divides by \(\|x\|^2 \|y\|^2\) instead of \(\|x\| \|y\|\).

**1. Understand how `KNeighborsRegressor(n_neighbors=3, weights='uniform')` predicts:** We are given 4 training samples in 1D: - \(x_1 = 1,\; y_1 = 10\) - \(x_2 = 3,\; y_2 = 30\) - \(x_3 = 4,\; y_3 = 40\) - \(x_4 = 6,\; y_4 = 60\) We want to predict `y_pred` for the test point \(x_{\text{test}} = 5\) using the \(k = 3\) nearest neighbors with uniform weights. **2. Compute the Euclidean distance from \(x_{\text{test}} = 5\) to each training point:** - \(d(5, 1) = |5 - 1| = 4\) - \(d(5, 3) = |5 - 3| = 2\) - \(d(5, 4) = |5 - 4| = 1\) - \(d(5, 6) = |5 - 6| = 1\) The 3 nearest neighbors to \(x_{\text{test}} = 5\) are \(x_3 = 4\) (distance 1), \(x_4 = 6\) (distance 1), and \(x_2 = 3\) (distance 2). **3. Average the target values of the 3 nearest neighbors:** Their target values are \(y_3 = 40\), \(y_4 = 60\), and \(y_2 = 30\). Taking the unweighted mean: \[y_{\text{pred}} = \frac{30 + 40 + 60}{3} = \frac{130}{3} \approx 43.3333 \approx 43.33\] **Answer: 43.33.**

**1. Tokenize the two documents and lowercase all words:** We are given two documents and `TfidfVectorizer(ngram_range=(1, 2), smooth_idf=True, lowercase=True)`: - Document 1: `"machine learning models are useful"` (5 tokens) - Document 2: `"machine learning techniques are powerful"` (5 tokens) With `ngram_range=(1, 2)`, the vectorizer extracts all unique **unigrams** (1-word tokens) and **bigrams** (2-word consecutive sequences) across the corpus. **2. List all unique unigrams:** From Document 1: `{"machine", "learning", "models", "are", "useful"}` (5 unigrams). From Document 2: `"machine"`, `"learning"`, and `"are"` are already counted; new unigrams are `{"techniques", "powerful"}` (2 new unigrams). Total unique unigrams \(= 5 + 2 = 7\). **3. List all unique bigrams:** - Document 1 has 4 bigrams: `"machine learning"`, `"learning models"`, `"models are"`, `"are useful"`. - Document 2 has 4 bigrams: `"machine learning"` (already counted), `"learning techniques"`, `"techniques are"`, `"are powerful"` (3 new bigrams). Total unique bigrams \(= 4 + 3 = 7\). **4. Sum the unique unigrams and bigrams to find the number of columns in `X_tfidf`:** \[\text{Total columns} = 7 + 7 = 14\] **Answer: 14.**

**1. Identify the layer dimensions of the `MLPRegressor`:** We have a fully connected neural network with: - Input layer: \(n_0 = 4\) features - Hidden layer 1: \(n_1 = 3\) neurons (from `hidden_layer_sizes=(3, 2)`) - Hidden layer 2: \(n_2 = 2\) neurons (from `hidden_layer_sizes=(3, 2)`) - Output layer: \(n_3 = 1\) continuous target neuron **2. Calculate the number of weights between each pair of consecutive layers (ignoring biases):** - Input layer to Hidden layer 1: \(4 \times 3 = 12\) weights - Hidden layer 1 to Hidden layer 2: \(3 \times 2 = 6\) weights - Hidden layer 2 to Output layer: \(2 \times 1 = 2\) weights **3. Sum all weights:** \[\text{Total weights} = 12 + 6 + 2 = 20\] **Answer: 20.**

**1. Recall the definition of Recall for a specific class in multi-class classification:** For class `1`, Recall is the fraction of actual class `1` samples (`y_true == 1`) that are correctly predicted as class `1` (`y_pred == 1`): \[\text{Recall}_{1} = \frac{\text{TP}_1}{\text{TP}_1 + \text{FN}_1} = \frac{\text{Number of samples with } y_{\text{true}} = 1 \text{ and } y_{\text{pred}} = 1}{\text{Total number of samples with } y_{\text{true}} = 1}\] **2. Compare `y_true` and `y_pred` on the middle row (where `y_true == 1`):** Looking at the three rows of `y_true` and `y_pred`: - Row 1 has seven `0`s in `y_true`. - Row 2 has six `1`s in `y_true`: `1, 1, 1, 1, 1, 1` (so there are \(6\) actual class `1` samples in total). - The corresponding predictions in Row 2 of `y_pred` are: `1, 1, 0, 1, 2, 1`. - Row 3 has seven `2`s in `y_true`. **3. Count True Positives for class `1` and compute Recall:** Out of the \(6\) actual class `1` samples in Row 2, the predictions are `1` at positions 1, 2, 4, and 6 (\(\text{TP}_1 = 4\)), while position 3 is predicted as `0` and position 5 is predicted as `2` (\(\text{FN}_1 = 2\)). Therefore: \[\text{Recall}_{1} = \frac{4}{6} = \frac{2}{3} \approx 0.6667 \approx 0.67\] **Answer: 0.67.**

**1. Inspect the 4-sample, 4-feature matrix `X`:** From the comprehension code block, the \(4 \times 4\) data matrix is: \[X = \begin{bmatrix} 2 & 4 & 30 & 1 \\ 4 & 6 & 20 & 1 \\ 6 & 4 & 30 & -1 \\ 8 & 6 & 20 & -1 \end{bmatrix}\] `StandardScaler` standardizes each column \(j\) independently using its sample mean \(\mu_j\) and population standard deviation \(\sigma_j = \sqrt{\frac{1}{4}\sum_{i=1}^4 (x_{ij} - \mu_j)^2}\): \[z_{ij} = \frac{x_{ij} - \mu_j}{\sigma_j}\] **2. Standardize each of the 4 columns:** - **Column 0 (`[2, 4, 6, 8]`):** - Mean \(\mu_0 = \frac{2 + 4 + 6 + 8}{4} = 5\). - Deviations from mean: \([-3, -1, 1, 3]\). - Population variance: \(\sigma_0^2 = \frac{(-3)^2 + (-1)^2 + 1^2 + 3^2}{4} = \frac{9 + 1 + 1 + 9}{4} = \frac{20}{4} = 5 \implies \sigma_0 = \sqrt{5} \approx 2.236068\). - Standardized values: \(\left[\frac{-3}{\sqrt{5}}, \frac{-1}{\sqrt{5}}, \frac{1}{\sqrt{5}}, \frac{3}{\sqrt{5}}\right] \approx [-1.3416, -0.4472, 0.4472, 1.3416]\). - **Column 1 (`[4, 6, 4, 6]`):** - Mean \(\mu_1 = 5\), deviations \([-1, 1, -1, 1]\), \(\sigma_1 = 1\), so standardized values are \([-1, 1, -1, 1]\). - **Column 2 (`[30, 20, 30, 20]`):** - Mean \(\mu_2 = 25\), deviations \([5, -5, 5, -5]\), \(\sigma_2 = 5\), so standardized values are \([1, -1, 1, -1]\). - **Column 3 (`[1, 1, -1, -1]`):** - Mean \(\mu_3 = 0\), deviations \([1, 1, -1, -1]\), \(\sigma_3 = 1\), so standardized values are \([1, 1, -1, -1]\). **3. Find the maximum value in `X_scaled`:** Comparing the maximum of Column 0 (\(\frac{3}{\sqrt{5}} \approx 1.3416\)) with the maximum of Columns 1, 2, and 3 (\(1.0\)), the overall maximum value in `X_scaled` is: \[\max(X_{\text{scaled}}) = \frac{3}{\sqrt{5}} \approx 1.34\] **Answer: 1.34.**

**1. Recall the population variances of the 4 columns of `X` from the previous step:** For the dataset: \[X = \begin{bmatrix} 2 & 4 & 30 & 1 \\ 4 & 6 & 20 & 1 \\ 6 & 4 & 30 & -1 \\ 8 & 6 & 20 & -1 \end{bmatrix}\] we found the population variance \(\sigma_j^2 = \frac{1}{4}\sum_{i=1}^4 (x_{ij} - \mu_j)^2\) of each column: - **Column 0 (`[2, 4, 6, 8]`):** \(\sigma_0^2 = 5.0\) - **Column 1 (`[4, 6, 4, 6]`):** \(\sigma_1^2 = 1.0\) - **Column 2 (`[30, 20, 30, 20]`):** \(\sigma_2^2 = 25.0\) - **Column 3 (`[1, 1, -1, -1]`):** \(\sigma_3^2 = 1.0\) **2. Apply `VarianceThreshold(threshold=0.3)`:** `VarianceThreshold(threshold=0.3)` keeps every feature whose variance is strictly greater than `0.3`. Since \(5.0 > 0.3\), \(1.0 > 0.3\), \(25.0 > 0.3\), and \(1.0 > 0.3\), **all 4 features** exceed the threshold and are retained in `X_selected`. **Answer: 4.**

**1. Understand the purpose and mechanism of Data Augmentation:** In machine learning (especially computer vision), **data augmentation** generates new synthetic training examples by applying label-preserving transformations—such as random rotations, scaling, cropping, horizontal/vertical flipping, or color jittering—to existing training images. **2. Evaluate each statement:** - **Statement A:** True — data augmentation creates modified copies of existing data using transformations like rotation, scaling, and cropping. - **Statement B:** True — expanding the diversity and effective size of the training set prevents the model from memorizing exact pixel layouts, helping it generalize better on unseen test data and reducing overfitting. - **Statement C:** False — data augmentation increases the number/variety of training examples (which typically increases training time rather than shrinking image dimensions for speed). - **Statement D:** False — noise removal and grayscale conversion are standard image preprocessing/denoising steps, not data augmentation. **Answer: A, B — Data augmentation creates modified copies via transformations like rotation/scaling/cropping, and artificially expands the dataset to improve generalization and reduce overfitting.** A: **Correct:** Standard image augmentation applies geometric and photometric transforms (rotation, scaling, cropping, flipping) to create varied training samples. B: **Correct:** Increasing training set diversity via augmentation acts as a regularizer that reduces overfitting and improves unseen-data generalization. C: **Incorrect:** Downsizing images for faster training is dimensionality reduction/resizing, whereas data augmentation expands dataset variety. D: **Incorrect:** Denoising and grayscale conversion are deterministic preprocessing steps rather than data augmentation.

**1. Analyze the `CountVectorizer` parameters in each statement:** - **Statement A:** Setting `max_df=3` (an integer) tells `CountVectorizer` to ignore terms that appear in **strictly more than 3 documents** (document frequency \(> 3\)) when building the vocabulary. (Correct!) - **Statement B:** In `CountVectorizer`, `ngram_range` must be a **2-tuple** `(min_n, max_n)` such as `ngram_range=(1, 3)` to extract unigrams, bigrams, and trigrams. Passing a 3-tuple `ngram_range=(1, 2, 3)` is invalid (`ValueError: too many values to unpack`). (Incorrect.) - **Statement C:** Setting `stop_words='english'` **removes** common English stop words (like `"the"`, `"is"`, `"and"`), which **decreases** (or keeps equal) the size of the vocabulary, never increases it. (Incorrect.) - **Statement D:** Setting `binary=True` makes `CountVectorizer` record presence/absence (`1` if a token occurs at least once in the document, `0` otherwise), changing all non-zero term counts to `1`. (Correct!) **Answer: A, D — Setting max_df=3 ignores terms with document frequency > 3, and setting binary=True sets all non-zero counts to 1.** A: **Correct:** When `max_df` is an integer (e.g., `max_df=3`), terms with document frequency strictly greater than `3` are excluded from the vocabulary. B: **Incorrect:** `ngram_range` expects a 2-tuple `(min_n, max_n)`; to extract unigrams, bigrams, and trigrams, you must set `ngram_range=(1, 3)`, not `(1, 2, 3)`. C: **Incorrect:** Filtering out stop words (`stop_words='english'`) removes words from the vocabulary, reducing its size rather than increasing it. D: **Correct:** With `binary=True`, all non-zero token frequency counts are clipped to `1` to indicate binary presence/absence.

**1. Review Hierarchical Agglomerative Clustering and distance metrics:** - **Statement A:** Agglomerative clustering is a **bottom-up** hierarchical clustering method: it begins with each point as its own singleton cluster and iteratively merges the closest pair of clusters according to a chosen linkage criterion. (Correct!) - **Statement B:** When `linkage='complete'` (complete linkage), the distance between two clusters is defined as the **maximum** distance between any observation in the first cluster and any observation in the second cluster—not the average distance (which is `linkage='average'`). (Incorrect.) - **Statement C:** In general (with single, complete, or average linkage), distance/dissimilarity measures such as `"manhattan"`, `"cosine"`, and `"euclidean"` can be used to compute inter-cluster linkage distances. (Correct!) - **Statement D:** Mathematically, a valid distance metric \(d(x, y)\) must satisfy non-negativity (\(d(x, y) \ge 0\) with \(d(x,y)=0 \iff x=y\)), symmetry (\(d(x, y) = d(y, x)\)), and the triangle inequality (\(d(x, z) \le d(x, y) + d(y, z)\)). (Correct!) **Answer: A, C, D — Agglomerative clustering is a bottom-up merging approach, metrics like manhattan/cosine/euclidean can be used, and a valid metric satisfies non-negativity, symmetry, and the triangle inequality.** A: **Correct:** Agglomerative clustering starts with individual points as singleton clusters and merges them bottom-up based on the linkage criterion. B: **Incorrect:** `'complete'` linkage uses the **maximum** pairwise distance between points in the two clusters; the average pairwise distance corresponds to `'average'` linkage. C: **Correct:** `AgglomerativeClustering` supports `'euclidean'`, `'manhattan'`, and `'cosine'` distance metrics for single, complete, and average linkage. D: **Correct:** By definition, any mathematical metric space distance function must be non-negative, symmetric, and obey the triangle inequality.