For a classification problem, Principal Component Analysis (PCA) has been used to reduce the dimensionality of a feature space from 100 to 10.
Which of the following options is true about the angle $\theta$ between the first and the tenth principal components?

Question

What is a loss function? Write the two categories of loss functions.

Answer 1

This question checks whether we remember what actually makes something a "principal component" in PCA, and the answer comes straight from a linear algebra fact rather than any specific dataset detail.

$\theta = 0^{\circ}$: this would mean the 1st and 10th components are the same direction repeated twice. PCA never produces two components in the same direction, each new component is chosen to capture variance that is not already explained by the earlier ones, so this is wrong.
$\theta = 90^{\circ}$: PCA finds the eigenvectors of the covariance matrix of the data, and covariance matrices are always symmetric. For a symmetric matrix, eigenvectors tied to different eigenvalues are always at right angles to each other, no exceptions. The 1st and 10th principal components use different eigenvalues (that is literally why one is ranked higher than the other), so they must be orthogonal. This fits.
$90^{\circ} < \theta \le 180^{\circ}$: this range describes an obtuse relationship, which is not what orthogonal eigenvectors give us, this is a distractor aimed at people who forget the exact right-angle property.
$0 < \theta < 90^{\circ}$: this describes a generic acute angle, again not matching the guaranteed perpendicularity of eigenvectors from a symmetric matrix.

Worth noting: this orthogonality holds regardless of how many dimensions we keep. Going from 100 down to 10 just means we throw away 90 of the eigenvectors (the ones with the smallest eigenvalues), it does not change the geometric relationship among the ones we keep. So whether we compare component 1 and component 2, or component 1 and component 10, the answer is the same right angle.

The correct choice is (B), $\theta = 90^{\circ}$.

\[ \boxed{\theta = 90^{\circ}} \]

Answer 2

This question checks whether we remember what actually makes something a "principal component" in PCA, and the answer comes straight from a linear algebra fact rather than any specific dataset detail.

$\theta = 0^{\circ}$: this would mean the 1st and 10th components are the same direction repeated twice. PCA never produces two components in the same direction, each new component is chosen to capture variance that is not already explained by the earlier ones, so this is wrong.
$\theta = 90^{\circ}$: PCA finds the eigenvectors of the covariance matrix of the data, and covariance matrices are always symmetric. For a symmetric matrix, eigenvectors tied to different eigenvalues are always at right angles to each other, no exceptions. The 1st and 10th principal components use different eigenvalues (that is literally why one is ranked higher than the other), so they must be orthogonal. This fits.
$90^{\circ} < \theta \le 180^{\circ}$: this range describes an obtuse relationship, which is not what orthogonal eigenvectors give us, this is a distractor aimed at people who forget the exact right-angle property.
$0 < \theta < 90^{\circ}$: this describes a generic acute angle, again not matching the guaranteed perpendicularity of eigenvectors from a symmetric matrix.

Worth noting: this orthogonality holds regardless of how many dimensions we keep. Going from 100 down to 10 just means we throw away 90 of the eigenvectors (the ones with the smallest eigenvalues), it does not change the geometric relationship among the ones we keep. So whether we compare component 1 and component 2, or component 1 and component 10, the answer is the same right angle.

The correct choice is (B), $\theta = 90^{\circ}$.

\[ \boxed{\theta = 90^{\circ}} \]

Column 1		Column 2
(p)	First In First Out	(i)	Stacks
(q)	Lookup Operation	(ii)	Queues
(r)	Last In First Out	(iii)	Hash Tables

For a classification problem, Principal Component Analysis (PCA) has been used to reduce the dimensionality of a feature space from 100 to 10.
Which of the following options is true about the angle \(\theta\) between the first and the tenth principal components?

Show Hint

The Correct Option is B

Solution and Explanation

Top Questions on Machine Learning

Questions Asked in GATE DA exam

For a classification problem, Principal Component Analysis (PCA) has been used to reduce the dimensionality of a feature space from 100 to 10. Which of the following options is true about the angle \(\theta\) between the first and the tenth principal components?