Question

Consider that Linear Ridge Regression is being used to learn a prediction function \(y_{pred} = w^T x\), where \(w, x \in \mathbb{R}^2\) and Mean Absolute Error (MAE) is used to measure the prediction error. A weight of 0.20 is associated with the regularizer.

At an intermediate step of the training process, assume that the parameter \(w = [-3.00,\ 4.00]^T\). In the next step, for the input \(x = [1.00,\ 2.00]^T\), the predicted value of \(y\) is noted. Let the relation between \(x = [x_1,\ x_2]^T\) and the true value of \(y\) be \(y_{true} = x_1 + x_2\).

The value of the overall regularized loss function for this instance is _______ . (Rounded off to two decimal places)

Hint:

Compute the MAE term |w^T x - y_true| and the L2 penalty lambda times ||w||^2 separately, then add them together.

Answer 1

Correct Answer: 7

Step 1: Write down the Ridge-regularized MAE loss for one sample.
$$L(w) = \underbrace{|w^Tx - y_{true}|}_{\text{MAE term}} + \underbrace{\lambda(w_1^2+w_2^2)}_{\text{L2 penalty}}$$
with $\lambda = 0.20$ given as the regularizer weight.

Step 2: Plug in the numbers for the prediction error piece first.
$w^Tx = (-3.00)(1.00) + (4.00)(2.00) = 5.00$, and separately $y_{true} = x_1+x_2 = 1.00+2.00 = 3.00$.
The absolute error is $|5.00 - 3.00| = 2.00$.

Step 3: Work out the penalty piece independently.
The squared norm of the weight vector is $w_1^2 + w_2^2 = (-3.00)^2 + (4.00)^2 = 9.00 + 16.00 = 25.00$.
Scaling by the regularizer weight: $0.20 \times 25.00 = 5.00$.

Step 4: Combine the two independent pieces.
$$L = 2.00 + 5.00 = 7.00$$
Since the MAE term and the L2 penalty term are computed from completely separate quantities (prediction error vs. weight magnitude), they simply add.

Step 5: Sanity check the magnitude.
$w = [-3, 4]$ is a fairly large weight vector (norm 5), so a regularizer weight of 0.20 on its squared norm contributing 5.00 to a total loss of 7.00 makes sense, the penalty dominates the raw prediction error here.
$$\boxed{L = 7.00}$$