Machine Learning Practice · End Term · May 2026 · FN

**1. Understand the problem requirement:** We have a pandas DataFrame `df` containing columns `'ID'`, `'Department'` (`'IT'`, `'HR'`, `'Finance'`), `'Score'`, and `'Completed_Hours'`. We need to select all rows belonging to departments whose **average (mean) `'Score'` is strictly greater than 75**, while preserving the original DataFrame structure and index. **2. Check the department mean scores:** - `'IT'`: scores are `85` and `90`, mean \(= 87.5 > 75\) (keep all `'IT'` rows). - `'HR'`: scores are `70` and `75`, mean \(= 72.5 \le 75\) (drop). - `'Finance'`: scores are `60` and `80`, mean \(= 70.0 \le 75\) (drop). **3. Compare the pandas expressions:** - `df.groupby('Department').filter(lambda x: x['Score'].mean() > 75)` groups the DataFrame by `'Department'`, evaluates whether each group's mean `'Score'` is strictly greater than `75`, and returns a subset of the original DataFrame containing all rows from the matching groups with the original structure and index preserved. **Answer: B — df.groupby('Department').filter(lambda x: x['Score'].mean() > 75).** A: **Incorrect:** `df[df['Score'] > 75]` filters individual employee rows with `'Score' > 75` (which would include the Finance employee with score 80) instead of filtering by the department's mean score. B: **Correct:** `groupby('Department').filter(...)` evaluates the condition `x['Score'].mean() > 75` on each department group as a whole and returns all rows of qualifying departments with the original index preserved. C: **Incorrect:** This uses `>= 75` instead of strictly greater than `75` (`> 75`). D: **Incorrect:** Using `.apply(...)` filters individual rows > 75 within each group and adds a multi-index of `'Department'` rather than preserving the original DataFrame structure and filtering by group mean.

**1. Understand how `VarianceThreshold(threshold=0.5)` works:** In `scikit-learn`, `VarianceThreshold(threshold=0.5)` computes the population variance \(\sigma^2 = \frac{1}{n}\sum_{i=1}^n (x_i - \mu)^2\) of each column (feature) across the \(n = 3\) samples and retains only those columns whose variance is **strictly greater than** `0.5`. **2. Compute the population variance of each of the 4 columns in `X`:** We are given: \[X = \begin{bmatrix} 0 & 2 & 0 & 3 \\ 0 & 1 & 4 & 3 \\ 0 & 1 & 1 & 3 \end{bmatrix}\] - **Column 0 (`[0, 0, 0]`):** All values are equal to `0`, so variance \(\sigma_0^2 = 0 \le 0.5\) (removed). - **Column 1 (`[2, 1, 1]`):** Mean \(\mu_1 = \frac{2 + 1 + 1}{3} = \frac{4}{3}\). Variance: \[\sigma_1^2 = \frac{1}{3}\left[\left(2 - \frac{4}{3}\right)^2 + \left(1 - \frac{4}{3}\right)^2 + \left(1 - \frac{4}{3}\right)^2\right] = \frac{1}{3}\left(\frac{4}{9} + \frac{1}{9} + \frac{1}{9}\right) = \frac{6}{27} = \frac{2}{9} \approx 0.222 \le 0.5 \quad (\text{removed})\] - **Column 2 (`[0, 4, 1]`):** Mean \(\mu_2 = \frac{0 + 4 + 1}{3} = \frac{5}{3}\). Variance: \[\sigma_2^2 = \frac{1}{3}\left[\left(0 - \frac{5}{3}\right)^2 + \left(4 - \frac{5}{3}\right)^2 + \left(1 - \frac{5}{3}\right)^2\right] = \frac{1}{3}\left(\frac{25}{9} + \frac{49}{9} + \frac{4}{9}\right) = \frac{78}{27} = \frac{26}{9} \approx 2.889 > 0.5 \quad (\text{kept!})\] - **Column 3 (`[3, 3, 3]`):** All values are equal to `3`, so variance \(\sigma_3^2 = 0 \le 0.5\) (removed). **3. Conclude:** Only **1** feature (Column 2) has variance strictly greater than `0.5`. **Answer: B — 1.** A: **Incorrect:** Column 2 (`[0, 4, 1]`) has a variance of \(26/9 \approx 2.89 > 0.5\), so it is not removed. B: **Correct:** Only Column 2 has variance \(\approx 2.89 > 0.5\); Columns 0 and 3 have variance \(0\) and Column 1 has variance \(2/9 \approx 0.222 \le 0.5\), leaving 1 feature. C: **Incorrect:** Column 1 (`[2, 1, 1]`) has population variance \(2/9 \approx 0.222\), which is below the `0.5` threshold, so only 1 feature remains, not 2. D: **Incorrect:** Three of the four columns have variance \(\le 0.5\) and are dropped.

**1. Understand `TfidfVectorizer(min_df=2)`:** We are given three documents: - Document 1: `"red blue green"` - Document 2: `"red blue"` - Document 3: `"red yellow"` When `min_df=2` (an integer) is passed to `TfidfVectorizer`, it keeps only those words (tokens) that appear in **at least 2 documents** (document frequency \(\ge 2\)). **2. Count the document frequency (DF) of each unique word:** - `"red"`: appears in Document 1, Document 2, and Document 3 (\(\text{DF} = 3 \ge 2\) — **kept**). - `"blue"`: appears in Document 1 and Document 2 (\(\text{DF} = 2 \ge 2\) — **kept**). - `"green"`: appears only in Document 1 (\(\text{DF} = 1 < 2\) — **ignored**). - `"yellow"`: appears only in Document 3 (\(\text{DF} = 1 < 2\) — **ignored**). **3. Determine the output of `vectorizer.get_feature_names_out()`:** `get_feature_names_out()` returns a NumPy array of the retained vocabulary terms sorted in **alphabetical order**: `['blue' 'red']`. **Answer: C — ['blue' 'red'].** A: **Incorrect:** Both `'green'` and `'yellow'` appear in only 1 document and are filtered out by `min_df=2`. B: **Incorrect:** `'yellow'` only appears in the third document (\(\text{DF} = 1 < 2\)), so it is excluded. C: **Correct:** Only `'blue'` (\(\text{DF}=2\)) and `'red'` (\(\text{DF}=3\)) meet `min_df=2`, and `get_feature_names_out()` returns them as a NumPy array sorted alphabetically: `['blue' 'red']`. D: **Incorrect:** `'green'` only appears in Document 1 (\(\text{DF} = 1 < 2\)), so it is removed.

**1. Understand the first merge step in Agglomerative Clustering:** In bottom-up hierarchical agglomerative clustering, each data point starts in its own singleton cluster. At the very first step, the algorithm merges the two singleton clusters that have the **smallest pairwise Euclidean distance**. **2. Compute the Euclidean distances between the given points:** We are given \(A = (1, 1)\), \(B = (2, 2)\), \(C = (8, 8)\), and \(D = (10, 10)\): - Distance between \(A(1, 1)\) and \(B(2, 2)\): \[d(A, B) = \sqrt{(2 - 1)^2 + (2 - 1)^2} = \sqrt{1 + 1} = \sqrt{2} \approx 1.414\] - Distance between \(C(8, 8)\) and \(D(10, 10)\): \[d(C, D) = \sqrt{(10 - 8)^2 + (10 - 8)^2} = \sqrt{4 + 4} = \sqrt{8} = 2\sqrt{2} \approx 2.828\] - Distance between \(B(2, 2)\) and \(C(8, 8)\): \[d(B, C) = \sqrt{(8 - 2)^2 + (8 - 2)^2} = \sqrt{36 + 36} = 6\sqrt{2} \approx 8.485\] - Distance between \(A(1, 1)\) and \(D(10, 10)\): \[d(A, D) = 9\sqrt{2} \approx 12.728\] **3. Conclude:** Since \(d(A, B) = \sqrt{2} \approx 1.414\) is the smallest pairwise distance, **A and B** are merged first. **Answer: A — A and B.** A: **Correct:** The Euclidean distance between \(A(1,1)\) and \(B(2,2)\) is \(\sqrt{2} \approx 1.414\), which is the minimum distance among all pairs. B: **Incorrect:** The distance between \(C(8,8)\) and \(D(10,10)\) is \(\sqrt{8} \approx 2.828\), which is twice as large as \(d(A, B) = \sqrt{2}\). C: **Incorrect:** \(A(1,1)\) and \(D(10,10)\) are the farthest apart, with distance \(9\sqrt{2} \approx 12.73\). D: **Incorrect:** \(B(2,2)\) and \(C(8,8)\) have distance \(6\sqrt{2} \approx 8.49\), much larger than \(d(A, B)\).

**1. Understand the `NearestNeighbors` recommender code:** The code fits `NearestNeighbors(n_neighbors=11, metric='cosine')` on the 19 binary genre columns of `item`, queries `nn.kneighbors(query)` for a target movie in the dataset, and then extracts recommendations via: `recs = item.iloc[indices.flatten()[1:]]['movie_title']` **2. Analyze why `n_neighbors=11` and `[1:]` are used:** Because the query movie is itself part of the training dataset `X` on which `nn` was fitted, the movie with the smallest cosine distance (distance \(= 0\)) to the query movie is **the query movie itself** (at index position `0` of `indices.flatten()`). To return **10 other movies** as recommendations, we must request `n_neighbors=11` neighbors and slice off the first entry (`[1:]`) to exclude the query movie itself. **Answer: B — n_neighbors=11 is used instead of 10 because the nearest neighbour of any movie in the dataset is the movie itself, which is then dropped by [1:].** A: **Incorrect:** `kneighbors` returns **cosine distances** (\(1 - \text{cosine similarity}\)) sorted in **ascending** order (smallest distance first), not cosine similarities in descending order. B: **Correct:** Because the query movie is in the fitted dataset, it appears at index `0` as its own closest neighbor (distance 0); requesting `11` neighbors and slicing `[1:]` yields 10 distinct recommended movies. C: **Incorrect:** If two binary vectors share no genres, their dot product is \(0\), so their cosine similarity is \(0\) and their cosine distance is \(1 - 0 = 1.0\), not less than \(1\). D: **Incorrect:** Cosine distance is inherently scale-invariant (\(1 - \frac{u \cdot v}{\|u\|\|v\|}\)) and `StandardScaler` is not required prior to using `metric='cosine'` on binary genre vectors.

**1. Understand how `MinMaxScaler(feature_range=(0, 1))` transforms data:** For a feature column \(X\), `MinMaxScaler(feature_range=(0, 1))` scales each value \(x_i\) using the formula: \[x_{i,\text{scaled}} = \frac{x_i - x_{\min}}{x_{\max} - x_{\min}}\] **2. Compute the scaled values for `X = [[10], [20], [30], [40], [50]]`:** Here, \(x_{\min} = 10\) and \(x_{\max} = 50\), so the range is \(x_{\max} - x_{\min} = 40\): - For \(x = 10\): \(\frac{10 - 10}{40} = 0.00\) - For \(x = 20\): \(\frac{20 - 10}{40} = 0.25\) - For \(x = 30\): \(\frac{30 - 10}{40} = 0.50\) - For \(x = 40\): \(\frac{40 - 10}{40} = 0.75\) - For \(x = 50\): \(\frac{50 - 10}{40} = 1.00\) **3. Compute `X_scaled.sum()`:** Summing all 5 scaled values: \[0.00 + 0.25 + 0.50 + 0.75 + 1.00 = 2.50\] **Answer: 2.50.**

**1. Trace the chained `.head()` and `.tail()` calls step by step:** Let the original DataFrame `df` have 10 rows numbered `[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]`. Recall that when no argument is passed, `.head()` and `.tail()` default to `n = 5` rows: 1. `df.head(6)` selects the first 6 rows: `[1, 2, 3, 4, 5, 6]`. 2. `.tail(5)` selects the last 5 rows of that result: `[2, 3, 4, 5, 6]`. 3. `.head(3)` selects the first 3 rows of that result: `[2, 3, 4]`. 4. `.tail(2)` selects the last 2 rows of that result: `[3, 4]`. 5. `.head()` (default `n = 5`) on a 2-row DataFrame returns all 2 rows: `[3, 4]`. 6. `.tail()` (default `n = 5`) on that 2-row DataFrame returns all 2 rows: `[3, 4]`. **2. Sum the remaining row numbers:** The rows printed are row `3` and row `4`. Their sum is: \[3 + 4 = 7\] **Answer: 7.**

**1. Step 1 — Impute the missing value using `SimpleImputer(strategy='mean')`:** The input column is `X = [[2.0], [4.0], [np.nan], [8.0], [6.0]]`. The mean of the four observed values is: \[\mu = \frac{2.0 + 4.0 + 8.0 + 6.0}{4} = \frac{20.0}{4} = 5.0\] Replacing `np.nan` with `5.0` gives the imputed array: \[X_{\text{imp}} = [2.0,\; 4.0,\; 5.0,\; 8.0,\; 6.0]^T\] **2. Step 2 — Standardize using `StandardScaler()`:** `StandardScaler` computes the mean \(\mu\) and population standard deviation \(\sigma = \sqrt{\frac{1}{n}\sum_{i=1}^n (x_i - \mu)^2}\) across all \(n = 5\) samples: - Mean: \(\mu = \frac{2 + 4 + 5 + 8 + 6}{5} = 5.0\) - Population variance: \[\sigma^2 = \frac{(2 - 5)^2 + (4 - 5)^2 + (5 - 5)^2 + (8 - 5)^2 + (6 - 5)^2}{5} = \frac{9 + 1 + 0 + 9 + 1}{5} = \frac{20}{5} = 4.0\] - Standard deviation: \(\sigma = \sqrt{4.0} = 2.0\) **3. Compute the transformed value of the first sample (`x_1 = 2.0`):** \[z_1 = \frac{x_1 - \mu}{\sigma} = \frac{2.0 - 5.0}{2.0} = \frac{-3.0}{2.0} = -1.5\] **Answer: -1.5.**

**1. Understand `MLPClassifier` architecture and the `n_layers_` attribute:** In `scikit-learn`, `MLPClassifier(hidden_layer_sizes=(100, 50), random_state=0)` constructs a feed-forward neural network with: - **1 Input layer** (size = 10 features) - **2 Hidden layers** (specified by the tuple `(100, 50)`: first hidden layer with 100 units, second hidden layer with 50 units) - **1 Output layer** (size = 3 units for 3-class classification) **2. Count the total number of layers (`model.n_layers_`):** In `scikit-learn`, the attribute `model.n_layers_` counts the input layer, all hidden layers, and the output layer: \[\text{n\_layers\_} = 1 + 2 + 1 = 4\] **Answer: 4.**

**1. Count the learned parameters in `model_1` (`fit_intercept=True`):** When `LinearRegression(fit_intercept=True)` is fit on a dataset `X_train` with \(d = 10\) features, the model learns: - \(10\) feature weights (stored in `model_1.coef_`) - \(1\) bias/intercept term (stored in `model_1.intercept_`) Total parameters learned by `model_1` \(= 10 + 1 = 11\). **2. Count the learned parameters in `model_2` (`fit_intercept=False`):** When `LinearRegression(fit_intercept=False)` is fit on the same dataset with \(10\) features, no intercept is estimated (the hyperplane is forced through the origin), so the model learns only: - \(10\) feature weights (stored in `model_2.coef_`) Total parameters learned by `model_2` \(= 10\). **3. Find the difference:** \[11 - 10 = 1\] **Answer: 1.**

**1. Understand how `RadiusNeighborsRegressor(radius=1.0)` predicts:** We have 4 training points in 1D: - \(x_1 = 0,\; y_1 = 0\) - \(x_2 = 1,\; y_2 = 0\) - \(x_3 = 2,\; y_3 = 1\) - \(x_4 = 3,\; y_4 = 1\) When predicting for the query point \(x_q = 1.5\) with `radius=1.0`, the regressor finds all training points \(x_i\) whose distance to \(1.5\) satisfies \(|x_i - 1.5| \le 1.0\), and averages their target values \(y_i\). **2. Find all training points within radius \(1.0\) of \(1.5\):** - \(|0 - 1.5| = 1.5 > 1.0\) (outside radius) - \(|1 - 1.5| = 0.5 \le 1.0\) (inside radius, \(y_2 = 0\)) - \(|2 - 1.5| = 0.5 \le 1.0\) (inside radius, \(y_3 = 1\)) - \(|3 - 1.5| = 1.5 > 1.0\) (outside radius) **3. Compute the average target value of the neighbors:** The two neighbors within radius \(1.0\) are \(x_2 = 1\) (\(y_2 = 0\)) and \(x_3 = 2\) (\(y_3 = 1\)). Their uniform average is: \[\hat{y} = \frac{0 + 1}{2} = 0.5\] **Answer: 0.5.**

**1. Understand how `KNNImputer(n_neighbors=2, weights='uniform')` works:** We want to impute the missing value `np.nan` in row 0, column 2 of the matrix: - Row 0: `[1.0, 2.0, np.nan]` - Row 1: `[2.0, 4.0, 6.0]` - Row 2: `[1.0, 3.0, 5.0]` - Row 3: `[8.0, 9.0, 4.0]` - Row 4: `[3.0, 5.0, 7.0]` `KNNImputer` measures the distance between Row 0 and every other row that has a non-missing value in column 2, using the shared observed coordinates (columns 0 and 1): \[d_{\text{nan}}(u, v) = \sqrt{\frac{3}{2} \big((u_0 - v_0)^2 + (u_1 - v_1)^2\big)}\] **2. Compute the squared differences on columns 0 and 1 between Row 0 `(1.0, 2.0)` and Rows 1–4:** - **Row 1 `(2.0, 4.0)`:** \((1 - 2)^2 + (2 - 4)^2 = 1 + 4 = 5\) - **Row 2 `(1.0, 3.0)`:** \((1 - 1)^2 + (2 - 3)^2 = 0 + 1 = 1\) - **Row 3 `(8.0, 9.0)`:** \((1 - 8)^2 + (2 - 9)^2 = 49 + 49 = 98\) - **Row 4 `(3.0, 5.0)`:** \((1 - 3)^2 + (2 - 5)^2 = 4 + 9 = 13\) **3. Identify the 2 nearest neighbors and average their column-2 values:** The two rows with the smallest distances to Row 0 are **Row 2** (squared diff \(= 1\), column-2 value \(= 5.0\)) and **Row 1** (squared diff \(= 5\), column-2 value \(= 6.0\)). Using `weights='uniform'`, the imputed value is the simple average of their column-2 values: \[\text{Imputed value} = \frac{5.0 + 6.0}{2} = 5.50\] **Answer: 5.50.**

**1. Extract the confusion matrix counts from the problem statement:** We are given a test set of \(N = 500\) samples with: - True Positives: \(\text{TP} = 90\) - True Negatives: \(\text{TN} = 360\) - Total predicted positive (\(\text{TP} + \text{FP}\)): \(120 \implies \text{FP} = 120 - 90 = 30\) - Total actual positive (\(\text{TP} + \text{FN}\)): \(100 \implies \text{FN} = 100 - 90 = 10\) **2. Calculate Precision and Recall:** - **Precision:** \[\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}} = \frac{90}{120} = 0.75\] - **Recall:** \[\text{Recall} = \frac{\text{TP}}{\text{TP} + \text{FN}} = \frac{90}{100} = 0.90\] **3. Calculate the F1-Score:** The F1-score is the harmonic mean of Precision and Recall: \[F_1 = \frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2 \times 0.75 \times 0.90}{0.75 + 0.90} = \frac{1.35}{1.65} = \frac{9}{11} \approx 0.81818\] Rounding to two decimal places gives \(0.82\). **Answer: 0.82.**

**1. Note that the question asks us to select the FALSE statements:** Let us check the truth value of each statement regarding `scikit-learn` preprocessing tools: - **Statement A:** `StandardScaler` centers features to mean \(0\) and scales them to unit variance (\(1\)) via a linear shift-and-scale \(z = (x - \mu)/\sigma\). It does **not** change the shape of the distribution into a uniform distribution. Thus, Statement A is **false** (and should be selected). - **Statement B:** `MinMaxScaler` linearly rescales features to a fixed range (such as \([0, 1]\)). It preserves the original distribution shape and does **not** make features follow a Gaussian distribution. Thus, Statement B is **false** (and should be selected). - **Statement C:** `OrdinalEncoder` is designed for **categorical features with a natural ordered ranking** (such as `'low' < 'medium' < 'high'`), not continuous data from a Normal distribution. Thus, Statement C is **false** (and should be selected). - **Statement D:** `Pipeline` applies a sequential chain of transformations to the *entire* input matrix; to apply *separate* transformations to numerical and categorical columns in parallel, `ColumnTransformer` is used, not `Pipeline` alone. Thus, Statement D is **false** (and should be selected). **Answer: A, B, C, D — All four statements are false and should be selected.** A: **Correct:** This statement is false (so select it): `StandardScaler` only shifts and scales features to zero mean and unit variance without changing their distribution shape to uniform. B: **Correct:** This statement is false (so select it): `MinMaxScaler` linearly maps values to `[0, 1]` and does not transform the distribution into a Gaussian. C: **Correct:** This statement is false (so select it): `OrdinalEncoder` encodes ordered categorical variables, not continuous normally distributed data. D: **Correct:** This statement is false (so select it): `ColumnTransformer` (not `Pipeline`) is the tool that applies separate transformations to distinct subsets of columns in parallel.

**1. Check the lower bound (minimum possible value) of each evaluation metric:** - **A. Accuracy:** Defined as \(\frac{\text{TP} + \text{TN}}{N}\). Since counts are non-negative, the minimum possible value of Accuracy is **\(0\)** (achieved when every prediction is wrong). - **B. F1-Score:** Defined as \(\frac{2 \cdot \text{TP}}{2 \cdot \text{TP} + \text{FP} + \text{FN}}\). Since counts are non-negative, its lower bound is **\(0\)** (when \(\text{TP} = 0\)). - **C. R2-Score (\(R^2\)):** Defined as \(1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2}\). Because a bad regression model can have arbitrarily large squared errors \(\sum (y_i - \hat{y}_i)^2\), \(R^2\) can be **negative** and has no lower bound (\(-\infty\)). - **D. Mean Squared Error (MSE):** Defined as \(\frac{1}{n}\sum_{i=1}^n (y_i - \hat{y}_i)^2\). Since squares of real numbers are always \(\ge 0\), the lower bound of MSE is **\(0\)**. - **E. Precision:** Defined as \(\frac{\text{TP}}{\text{TP} + \text{FP}}\). Since counts are non-negative, its lower bound is **\(0\)**. **2. Conclude:** Accuracy, F1-Score, Mean_Squared_Error, and Precision all have \(0\) as their lower bound, whereas R2-Score can take negative values down to \(-\infty\). **Answer: A, B, D, E — Accuracy, F1-Score, Mean_Squared_Error, and Precision.** A: **Correct:** Accuracy is the fraction of correct predictions \(\in [0, 1]\), so its lower bound is \(0\). B: **Correct:** F1-Score is the harmonic mean of precision and recall \(\in [0, 1]\), so its lower bound is \(0\). C: **Incorrect:** \(R^2 = 1 - \text{SS}_{\text{res}}/\text{SS}_{\text{tot}}\) can be negative (down to \(-\infty\)) whenever the model's predictions are worse than predicting the sample mean. D: **Correct:** Mean Squared Error is an average of squared residuals \((y_i - \hat{y}_i)^2 \ge 0\), so its lower bound is \(0\). E: **Correct:** Precision is a ratio of non-negative counts \(\text{TP}/(\text{TP}+\text{FP}) \in [0, 1]\), so its lower bound is \(0\).

**1. Review the rules of `Pipeline` and `make_pipeline` in `scikit-learn`:** - **Intermediate steps:** Every step in a `Pipeline` except the last one must be a **transformer**, meaning it must implement both `fit` and `transform` (or `fit_transform`) so it can pass transformed data to the next step. (Statement A is **correct**.) - **Final estimator:** The last step in a `Pipeline` can be any estimator (classifier, regressor, transformer, or clusterer) and only needs to implement `fit` (plus whatever methods like `predict` or `transform` you call on the pipeline). (Statement B is **correct**.) - **`make_pipeline` naming:** Unlike `Pipeline([('name', Estimator())])`, the convenience function `make_pipeline(StandardScaler(), LogisticRegression())` **automatically generates** lowercase class names (`'standardscaler'`, `'logisticregression'`) and does **not** allow or require the user to pass explicit step names. (Statement C is **incorrect**.) - **Pipeline interface:** A `Pipeline` object exposes the exact same interface (`predict`, `predict_proba`, `score`, `transform`, etc.) as its final estimator. (Statement D is **correct**.) **Answer: A, B, D — Intermediate steps must implement fit and transform, the final estimator only needs to implement fit, and Pipeline exposes the interface of its final estimator.** A: **Correct:** All steps prior to the final step must transform the data for the next step, so they must implement `fit` and `transform`. B: **Correct:** The final step only needs to implement `fit` (and any prediction/scoring methods invoked on the pipeline). C: **Incorrect:** `make_pipeline` is specifically a shorthand that automatically names steps using their lowercase class names without taking user-supplied `(name, estimator)` tuples. D: **Correct:** Calling `.predict()`, `.score()`, or `.predict_proba()` on a `Pipeline` transforms the input through the intermediate steps and delegates to the final estimator's corresponding method.

**1. Understand `PCA(n_components=0.95)` in `scikit-learn`:** When `n_components` is a float strictly between `0.0` and `1.0` (here `0.95`), `PCA` automatically selects the **minimum number of principal components \(k\)** such that the cumulative proportion of variance explained by those \(k\) components is at least `0.95` (95%). **2. Evaluate each statement:** - **Statement A:** Since the wine dataset has 13 correlated features, 95% of the variance is captured by fewer than 13 components (\(k < 13\)), so `X_pca.shape[1]` is strictly less than the original number of features. (Incorrect.) - **Statement B:** By definition of `n_components=0.95`, `PCA` retains enough components so that `np.sum(pca.explained_variance_ratio_) >= 0.95`. (Correct!) - **Statement C:** Each retained principal component corresponds to one column in `X_pca` and one entry in `pca.explained_variance_ratio_`, so `len(pca.explained_variance_ratio_) == X_pca.shape[1]`. (Correct!) - **Statement D:** Principal components are orthogonal **linear combinations** of all original features (eigenvectors of the covariance matrix), not a subset of individual original features. (Incorrect.) **Answer: B, C — The sum of pca.explained_variance_ratio_ is at least 0.95, and len(pca.explained_variance_ratio_) equals X_pca.shape[1].** A: **Incorrect:** Setting `n_components=0.95` drops the trailing low-variance components, so `X_pca.shape[1]` is smaller than the original feature count. B: **Correct:** Passing a float `0 < n_components < 1` instructs `PCA` to keep the smallest number of components whose cumulative explained variance ratio is at least `0.95`. C: **Correct:** Both `len(pca.explained_variance_ratio_)` and `X_pca.shape[1]` equal the number of retained principal components \(k\). D: **Incorrect:** Principal components are orthogonal linear combinations of the original features, not individual original features selected by variance.

**1. Compare `RandomForestRegressor` (`ensemble1`) and `GradientBoostingRegressor` (`ensemble2`):** - **`ensemble1 = RandomForestRegressor(n_estimators=100, random_state=42)`:** Uses **bagging** (bootstrap aggregating) with randomized feature subsets. All 100 trees are trained **independently in parallel** and their predictions are combined by an unweighted average, which primarily **reduces variance**. - **`ensemble2 = GradientBoostingRegressor(n_estimators=100, random_state=42)`:** Uses **boosting**. Trees are trained **sequentially**, where each tree is fit to the negative gradient (pseudo-residuals) of the loss function from the preceding trees, which primarily **reduces bias**. **2. Check the options:** - Option A is false because `RandomForestRegressor` averages trees equally (no sequential reweighting). - Option B is **true** because `ensemble1` reduces variance by averaging independent trees, whereas `ensemble2` builds trees sequentially to correct errors (residuals) of prior trees. - Option C is false because increasing `n_estimators` too much in gradient boosting can lead to **overfitting** on noisy training data. - Option D is false because these are **regression** models (`Regressor`), which combine tree outputs via averaging or additive weighted sums, not majority voting. **Answer: B — ensemble1 reduces variance by averaging independent predictions from 100 trees, meanwhile ensemble2 builds trees sequentially to correct errors of prior trees.** A: **Incorrect:** `RandomForestRegressor` assigns equal weight (\(1/100\)) to all trees trained independently in parallel. B: **Correct:** Random Forest averages independent bootstrap trees to reduce variance, while Gradient Boosting trains trees sequentially on the residuals of previous trees to reduce bias. C: **Incorrect:** Unlike Random Forests, adding too many boosting stages (`n_estimators=500`) in Gradient Boosting can overfit the training data. D: **Incorrect:** Regression ensembles combine continuous predictions by averaging (Random Forest) or additive summation (Gradient Boosting), whereas majority voting is used only in classification.

**1. Analyze the `SGDRegressor` hyperparameters in the given code:** The model is initialized as: `SGDRegressor(loss="squared_error", penalty="elasticnet", alpha=0.0001, l1_ratio=0.3, max_iter=1000, random_state=42)` **2. Understand `penalty="elasticnet"` and `l1_ratio=0.3`:** - Because `penalty="elasticnet"` and `alpha=0.0001 > 0` are specified, the model **does perform regularization**. - Specifically, Elastic Net regularization combines both the **L1 (Lasso)** penalty and the **L2 (Ridge)** penalty: \[\text{Penalty} = \alpha \Big( \text{l1\_ratio} \cdot \|w\|_1 + \frac{1 - \text{l1\_ratio}}{2} \|w\|_2^2 \Big)\] - Since `l1_ratio=0.3 > 0`, the penalty includes a non-zero L1 component (\(0.3 \alpha \|w\|_1\)). The L1 penalty drives weak or redundant feature coefficients to **exactly zero**, which performs **automatic feature selection**. **Answer: B — The model performs regularization and the regularization can also perform some feature selection.** A: **Incorrect:** With `penalty='elasticnet'` and `alpha=0.0001 > 0`, Elastic Net regularization is active. B: **Correct:** Elastic Net combines L1 and L2 penalties; because `l1_ratio=0.3 > 0`, the L1 term can shrink irrelevant feature weights to zero, performing feature selection. C: **Incorrect:** Unlike pure L2 (`'l2'`) regularization, `'elasticnet'` with `l1_ratio > 0` includes an L1 penalty capable of setting coefficients to zero (feature selection). D: **Incorrect:** Regularization constrains model complexity and helps prevent overfitting rather than causing it.

**1. Evaluate Assertion (A) and Reason (R):** - **Assertion (A):** *"Adding more features to a model always increases training accuracy (or reduces training error)."* In standard unregularized models (such as Ordinary Least Squares linear regression or unpruned decision trees), adding extra features expands the hypothesis space \(\mathcal{H}_{\text{new}} \supseteq \mathcal{H}_{\text{old}}\) (since the model can always set the new feature's weight to \(0\) and at least match the previous training performance, and typically fits the training data even closer). Thus, Assertion (A) is treated as **true**. - **Reason (R):** *"Additional features always provide new useful information that helps the model generalize better on unseen data."* This is **false**! Adding irrelevant, noisy, or redundant features often causes the model to fit noise in the training set (**overfitting** / the curse of dimensionality), which **degrades** generalization performance on unseen test data. **2. Conclude:** Assertion (A) is true, but Reason (R) is false. **Answer: C — A is true, but R is false.** A: **Incorrect:** Reason (R) is false because extra features can be pure noise or redundant, hurting test-set generalization via overfitting. B: **Incorrect:** Reason (R) is false because adding features does not always improve generalization on unseen data. C: **Correct:** Adding features increases model flexibility to fit the training set (A is true), but extra features do not always carry useful signal and can hurt test generalization due to overfitting (R is false). D: **Incorrect:** Assertion (A) is true while Reason (R) is false.

**1. Translate the business requirement into confusion matrix terms:** In this phishing detection problem: - **Positive class (`1`):** Phishing email - **Negative class (`0`):** Legitimate (normal) email - **False Positive (FP):** A legitimate email is wrongly classified as a phishing email (and blocked/quarantined). - **False Negative (FN):** A phishing email is missed and allowed into the inbox. **2. Identify the metric that penalizes False Positives:** The problem states that *legitimate emails must not be wrongly flagged as phishing*—that is, we must minimize **False Positives (FP)** so that when the model flags an email as phishing, we can be highly confident it truly is phishing. The metric that measures the fraction of predicted positive emails that are truly phishing is **Precision**: \[\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}}\] Maximizing Precision directly minimizes False Positives. **Answer: B — Precision.** A: **Incorrect:** Accuracy treats False Positives and False Negatives equally and can be misleading when phishing emails are rare. B: **Correct:** Precision (\(\text{TP}/(\text{TP}+\text{FP})\)) measures how reliable positive (phishing) predictions are; maximizing precision minimizes false alarms on legitimate emails. C: **Incorrect:** Recall (\(\text{TP}/(\text{TP}+\text{FN})\)) focuses on catching all phishing emails (minimizing False Negatives), even at the cost of flagging many legitimate emails. D: **Incorrect:** F1-score balances Precision and Recall equally rather than specifically prioritizing the avoidance of False Positives.

**1. Analyze the NumPy slicing operation on the image:** We are given a \(3 \times 3\) grayscale image array: \[\text{image} = \begin{bmatrix} 10 & 20 & 30 \\ 40 & 50 & 60 \\ 70 & 80 & 90 \end{bmatrix}\] The code computes: `processed = image[1:, 1:]` **2. Determine what `image[1:, 1:]` extracts:** - `1:` along axis 0 (rows) selects row indices `1` and `2` (dropping row `0`). - `1:` along axis 1 (columns) selects column indices `1` and `2` (dropping column `0`). The resulting \(2 \times 2\) sub-image is: \[\text{processed} = \begin{bmatrix} 50 & 60 \\ 80 & 90 \end{bmatrix}\] Extracting a contiguous rectangular sub-region (patch) from a larger image is the definition of **Cropping**. **Answer: B — Cropping.** A: **Incorrect:** Rotation rearranges all pixels around a center angle (e.g., via `np.rot90`) without reducing the \(3 \times 3\) dimensions to \(2 \times 2\). B: **Correct:** Slicing a sub-region `image[1:, 1:]` extracts the bottom-right \(2 \times 2\) patch from the \(3 \times 3\) image, which is image cropping. C: **Incorrect:** Magnification (zooming/upsampling) interpolates pixels to a larger resolution rather than slicing a smaller subarray. D: **Incorrect:** Flipping reverses pixel order along an axis (such as `image[::-1, :]` or `image[:, ::-1]`) while keeping the \(3 \times 3\) shape.

**1. Compare the initialization methods of the three `KMeans` models:** - `model1 = KMeans(n_clusters=3, init='k-means++')` - `model2 = KMeans(n_clusters=3, init='random')` - `model3 = KMeans(n_clusters=3)` — in `scikit-learn`, the default value of `init` is `'k-means++'`. **2. Understand why `init='random'` converges slower than `init='k-means++'`:** - **`'k-means++'` (`model1` and `model3`):** Selects initial cluster centroids sequentially with probability proportional to the squared distance \(D(x)^2\) from existing centroids. Spreading out the initial centroids near the true cluster structure allows Lloyd's algorithm to converge in significantly **fewer iterations**. - **`'random'` (`model2`):** Picks \(k\) random data points uniformly as initial centroids. Because random points can easily start close together inside the same cluster, `model2` typically requires **more iterations to converge** (converges slowest). **Answer: B — model2.** A: **Incorrect:** `model1` uses `'k-means++'` initialization, which spreads out initial centroids for fast convergence. B: **Correct:** `model2` uses `init='random'`, which chooses initial centroids uniformly at random and typically takes more iterations to converge than `'k-means++'`. C: **Incorrect:** `model3` uses the default `init='k-means++'` in `scikit-learn`, giving it the same fast convergence as `model1`. D: **Incorrect:** `KMeans(n_clusters=3)` is completely valid syntax in `scikit-learn` (`init` defaults to `'k-means++'`).

**1. Distinguish between Additive and Multiplicative seasonal decomposition:** In time-series analysis (`statsmodels.tsa.seasonal.seasonal_decompose`): - **Additive model (`model="additive"`):** \(Y_t = T_t + S_t + R_t\). Used when the amplitude (height) of seasonal fluctuations remains roughly **constant** regardless of the overall trend level \(T_t\). - **Multiplicative model (`model="multiplicative"`):** \(Y_t = T_t \times S_t \times R_t\). Used when the seasonal swings grow **proportionally larger** as the overall level/trend of the series increases (as in the classic monthly airline-passenger dataset). **2. Match with the given options:** We should fill the blank with `"multiplicative"`, because the seasonal amplitude scales with the level of the series. **Answer: B — "multiplicative", because the seasonal amplitude scales with the level of the series.** A: **Incorrect:** An increasing trend alone does not dictate an additive model; when seasonal swings grow with the trend, an additive model fails to capture the expanding amplitude. B: **Correct:** In a multiplicative decomposition (\(Y_t = T_t \times S_t \times R_t\)), the seasonal component multiplies the trend, so seasonal swings grow proportionally with the level of the series. C: **Incorrect:** `seasonal_decompose` supports both `'additive'` and `'multiplicative'` models for any positive integer `period`. D: **Incorrect:** In a multiplicative model, the seasonal and residual components are centered around `1.0` (not summing to `0`), and `model='multiplicative'` is chosen because of the scaling seasonal swings.

**1. Inspect each numbered line of the given `Pipeline` / `ColumnTransformer` code snippet:** Let us check the code line by line: - **`# Line1`:** `preprocessor = ColumnTransformer([('scale', StandardScaler, num_cols)])` In `ColumnTransformer`, the second element of each tuple must be an **instantiated** transformer object `StandardScaler()`, not the uninstantiated class `StandardScaler`. Passing the class itself raises a `TypeError` during fitting. Thus, **Line1 has an error**. - **`# Line2`:** `pipe = Pipeline([('prep', preprocessor), ('clf', LogisticRegression())])` This is valid syntax for constructing a `Pipeline`. - **`# Line3`:** `pipe.fit(X_train, X_test)` `pipe.fit(...)` expects the training features and the **training target labels** `(X_train, y_train)`, not `(X_train, X_test)` (which have different sample counts and wrong semantics). Thus, **Line3 has an error**. **Answer: B, D — Line1 and Line3.** A: **Incorrect:** Both Line1 and Line3 contain bugs that will cause execution to fail. B: **Correct:** In Line1, `StandardScaler` is passed as a class reference instead of an instantiated object `StandardScaler()`. C: **Incorrect:** Line2 correctly constructs a `Pipeline` from `(name, step)` tuples. D: **Correct:** In Line3, `pipe.fit` is called with `(X_train, X_test)` instead of `(X_train, y_train)`.

**1. Understand the role of the regularization parameter `C` in `SVC(kernel='rbf')`:** In `scikit-learn`'s Support Vector Classifier (`SVC`), the parameter `C` controls the penalty on slack variables \(C \sum_i \xi_i\) (it is inversely proportional to regularization strength): - **`model1 = SVC(kernel='rbf', C=0.1, gamma='scale')`:** Small `C = 0.1` applies strong regularization, allowing a wider margin and tolerating more training misclassifications. This results in a smoother boundary (**higher bias, lower variance**) and lower training accuracy (`0.81`). - **`model2 = SVC(kernel='rbf', C=100, gamma='scale')`:** Large `C = 100` penalizes training misclassifications very heavily, forcing a narrower margin and a more complex boundary that fits training points tightly (**lower bias, higher variance**) and achieves higher training accuracy (`0.98`). **2. Evaluate each statement:** - **Statement A:** True — `model2` (`C=100`) is less regularized, giving it lower bias and higher variance than `model1` (`C=0.1`). - **Statement B:** True — increasing `C` from `0.1` to `100` penalizes margin violations/misclassifications more strictly on the training data, increasing training accuracy. - **Statement C:** False — lower training accuracy does not guarantee better test-set performance (for example, `model1` might be underfitting). - **Statement D:** False — `gamma` is the key bandwidth hyperparameter specifically used by the `'rbf'` kernel \(\exp(-\gamma \|x - x'\|^2)\). **Answer: A, B — model2 is more likely to have lower bias and higher variance compared to model1, and increasing C allowed model2 to penalize misclassification errors more strictly.** A: **Correct:** A larger `C` (`100` vs `0.1`) reduces regularization strength, making `model2` more flexible (lower bias, higher variance). B: **Correct:** `C` multiplies the hinge-loss slack penalty \(C \sum \xi_i\), so increasing `C` forces the SVM to fit training points more strictly. C: **Incorrect:** Lower training accuracy does not automatically imply better test performance; `C=0.1` could be underfitting the data. D: **Incorrect:** The `gamma` parameter directly controls the width of the Gaussian RBF kernel \(K(x, x') = \exp(-\gamma \|x - x'\|^2)\).

**1. Review the properties of K-Means clustering:** - **Statement A:** K-Means uses Lloyd's iterative coordinate-descent algorithm, which converges to a **local minimum** of the within-cluster sum of squares (WCSS / inertia) that depends on the initial placement of centroids. Running K-Means with different random seeds (`random_state`) can therefore yield different clusterings. (Correct!) - **Statement B:** If one cluster contains points that lie very far from its centroid, those points may be **outliers** or may actually belong to distinct sub-clusters, indicating that the chosen number of clusters \(K = 4\) is suboptimal. (Correct!) - **Statement C:** Finding the global minimum of the K-Means objective is NP-hard; K-Means only guarantees convergence to a *local* optimum, not the global optimum. (Incorrect.) - **Statement D:** K-Means assigns points to clusters to **minimize** (not maximize) the squared Euclidean distance between points and their assigned cluster centroid. (Incorrect.) **Answer: A, B — K-Means can converge to different solutions depending on initial centroid placement, and a cluster with distant points may indicate outliers or a suboptimal K.** A: **Correct:** Because the K-Means objective is non-convex, different initial centroid placements can lead the algorithm to converge to different local minima. B: **Correct:** High within-cluster distances suggest either the presence of extreme outliers or that a larger \(K\) is needed to split a broad group. C: **Incorrect:** K-Means is a greedy local search algorithm and does not guarantee finding the global minimum across all possible partitions. D: **Incorrect:** K-Means minimizes within-cluster distances (inertia) to make clusters compact, rather than maximizing them.

**1. Inspect each marked line in the time-series stationarity testing code:** - **`# Line1`:** `diff = series.diff()` Computes the first difference \(y_t - y_{t-1}\) of the pandas Series. Note that the very first element `diff.iloc[0]` becomes `NaN`. By itself, calling `series.diff()` is valid pandas code. - **`# Line2`:** `adf_p = adfuller(diff)[1]` Passing `diff` directly to `statsmodels.tsa.stattools.adfuller` without calling `.dropna()` first (`adfuller(diff.dropna())[1]`) raises a `MissingDataError` / `ValueError: exog contains inf or nans` because `diff` has a leading `NaN` from `.diff()`! Thus, **Line2 contains an error**. - **`# Line3`:** `kpss_p = kpss(series, regression='ct')[1]` Valid syntax for running the KPSS test with constant and trend (`'ct'`) and extracting the p-value at index `1`. - **`# Line4`:** `is_stationary = (adf_p < 0.05) and (kpss_p < 0.05)` Recall the null hypotheses of the two tests: - **ADF test:** Null hypothesis \(H_0\) is **non-stationary** (has a unit root), so stationarity requires rejecting \(H_0\) (`adf_p < 0.05`). - **KPSS test:** Null hypothesis \(H_0\) is **stationary**, so stationarity requires *failing to reject* \(H_0\) (`kpss_p >= 0.05`, **not** `kpss_p < 0.05`)! Thus, **Line4 contains a logical error**. **Answer: B, D — Line2 and Line4.** A: **Incorrect:** `series.diff()` is valid pandas syntax for computing the first difference of a series. B: **Correct:** Because `series.diff()` introduces a `NaN` at the first index, passing `diff` directly to `adfuller(diff)` without `.dropna()` raises a missing-value error. C: **Incorrect:** `kpss(series, regression='ct')[1]` is valid syntax for extracting the KPSS p-value around a deterministic trend. D: **Correct:** The null hypothesis of the KPSS test is that the series *is* stationary, so `kpss_p < 0.05` indicates *non-stationarity*; confirming stationarity requires `kpss_p >= 0.05`.

**1. Understand the User-Based Collaborative Filtering problem:** We are given the user-item rating matrix \(R\) (where \(0\) denotes an unrated item): \[\begin{array}{c|ccccc} \text{User} & I_1 & I_2 & I_3 & I_4 & I_5 \\ \hline U_1 & 4 & 5 & 0 & 3 & 4 \\ U_2 & 5 & 4 & 4 & 0 & 5 \\ U_3 & 1 & 0 & 5 & 4 & 1 \\ U_4 & 4 & 4 & 0 & 2 & 0 \end{array}\] We want to predict \(U_4\)'s rating for item \(I_5\) using cosine similarity over the full row vectors (treating \(0\) as \(0\)): \[\hat{r}(U_4, I_5) = \frac{\sum_{v \in N} \text{sim}(U_4, v) \cdot r(v, I_5)}{\sum_{v \in N} \text{sim}(U_4, v)}\] where \(N = \{U_1, U_2, U_3\}\) are the users who have rated \(I_5\) (with ratings \(r(U_1, I_5) = 4\), \(r(U_2, I_5) = 5\), and \(r(U_3, I_5) = 1\)). **2. Compute the L2 norms of each user row vector:** - \(\|U_4\| = \sqrt{4^2 + 4^2 + 0^2 + 2^2 + 0^2} = \sqrt{16 + 16 + 4} = \sqrt{36} = 6\) - \(\|U_1\| = \sqrt{4^2 + 5^2 + 0^2 + 3^2 + 4^2} = \sqrt{16 + 25 + 9 + 16} = \sqrt{66} \approx 8.1240\) - \(\|U_2\| = \sqrt{5^2 + 4^2 + 4^2 + 0^2 + 5^2} = \sqrt{25 + 16 + 16 + 25} = \sqrt{82} \approx 9.0554\) - \(\|U_3\| = \sqrt{1^2 + 0^2 + 5^2 + 4^2 + 1^2} = \sqrt{1 + 25 + 16 + 1} = \sqrt{43} \approx 6.5574\) **3. Compute the cosine similarities \(\text{sim}(U_4, v)\):** - \(\text{sim}(U_4, U_1) = \frac{4(4) + 4(5) + 0(0) + 2(3) + 0(4)}{6 \sqrt{66}} = \frac{16 + 20 + 6}{6 \sqrt{66}} = \frac{42}{6\sqrt{66}} = \frac{7}{\sqrt{66}} \approx 0.86164\) - \(\text{sim}(U_4, U_2) = \frac{4(5) + 4(4) + 0(4) + 2(0) + 0(5)}{6 \sqrt{82}} = \frac{20 + 16}{6 \sqrt{82}} = \frac{36}{6\sqrt{82}} = \frac{6}{\sqrt{82}} \approx 0.66259\) - \(\text{sim}(U_4, U_3) = \frac{4(1) + 4(0) + 0(5) + 2(4) + 0(1)}{6 \sqrt{43}} = \frac{4 + 8}{6 \sqrt{43}} = \frac{12}{6\sqrt{43}} = \frac{2}{\sqrt{43}} \approx 0.30500\) **4. Compute the weighted average prediction \(\hat{r}(U_4, I_5)\):** - Numerator: \(0.86164(4) + 0.66259(5) + 0.30500(1) = 3.44656 + 3.31295 + 0.30500 = 7.06451\) - Denominator: \(0.86164 + 0.66259 + 0.30500 = 1.82923\) - Prediction: \[\hat{r}(U_4, I_5) = \frac{7.06451}{1.82923} \approx 3.8620 \approx 3.86\] **Answer: 3.86.**

**1. Understand the flaw in the given feature engineering setup:** The forecasting model uses a **future rolling average** (computed using values *after* the prediction timestamp \(t\)) as an input feature to predict the target at time \(t\). **2. Identify the phenomenon:** In real-world deployment, future values after time \(t\) are not yet available when making a prediction at time \(t\). Including information from the future (or from the target/test set) inside the training/evaluation features is a classic example of **target/temporal data leakage**. Because the future rolling average already contains the future outcomes being predicted, the model achieves an artificially inflated test \(R^2\) of `0.98` that will completely fail in production. **Answer: C — The evaluation is unreliable due to data leakage.** A: **Incorrect:** The high \(R^2\) is an artifact of cheating with future information rather than genuine low-bias modeling. B: **Incorrect:** Using future rolling averages leaks future target values rather than legitimately modeling seasonal patterns. C: **Correct:** Using future values that would not be available at prediction time introduces temporal data leakage, making the `0.98` test \(R^2\) completely invalid. D: **Incorrect:** An \(R^2\) of `0.98` is extremely high (not underfitting), caused here by data leakage.