Question

Consider a fully-connected feed-forward multi-layer perceptron. It has 30 neurons in the input layer, followed by two hidden layers and an output layer. The first hidden layer has 4 neurons and the second hidden layer has 3 neurons. The output layer has only one neuron. Assume that no bias parameters are used.

The number of learnable parameters in the multi-layer perceptron is _______ . (Answer in integer)

Hint:

With no bias terms, count only the weight connections between each pair of consecutive layers (input to output) and add them up: 30x4 + 4x3 + 3x1.

Answer 1

Correct Answer: 135

Step 1: Model each layer transition as a weight matrix.
A fully-connected layer that maps from a layer with $p$ neurons to a layer with $q$ neurons (no bias) has a weight matrix of shape $q \times p$, contributing exactly $p \times q$ learnable numbers.

Step 2: List the layer sizes in order.
$$30 \to 4 \to 3 \to 1$$
This network has three transitions: input to hidden-1, hidden-1 to hidden-2, and hidden-2 to output.

Step 3: Compute the parameter count for each transition as a matrix size.
Transition 1 (input to hidden-1): weight matrix of shape $4 \times 30$, giving $4 \times 30 = 120$ entries.
Transition 2 (hidden-1 to hidden-2): weight matrix of shape $3 \times 4$, giving $3 \times 4 = 12$ entries.
Transition 3 (hidden-2 to output): weight matrix of shape $1 \times 3$, giving $1 \times 3 = 3$ entries.

Step 4: Sum the sizes of all weight matrices.
$$\text{Total} = 120 + 12 + 3 = 135$$
Since no bias vectors exist in this network, this sum of matrix entries is the entire parameter count, nothing else to add.

Step 5: Confirm nothing was double counted.
Each pair of consecutive layers contributes exactly one weight matrix, and there are exactly 3 consecutive layer pairs for 4 layers total (input, hidden-1, hidden-2, output), so all three products were needed and none overlap.
$$\boxed{135}$$