Step 1: Recall how adjoint entries work.
The entry in row $i$, column $j$ of $\text{adj }A$ equals the cofactor of the element in row $j$, column $i$ of $A$. So we read positions in a crossed way.
Step 2: Find $x$.
Here $x$ sits at row 1, column 2 of $\text{adj }A$, so it is the cofactor of $a_{21}$. Delete row 2 and column 1 of $A$ and attach the sign $(-1)^{2+1}$:
\[ x = -\begin{vmatrix} 0 & 2 \\ 2 & 1 \end{vmatrix} = -(0-4) = 4. \]
Step 3: Find $y$.
Now $y$ is at row 3, column 3, so it is the cofactor of $a_{33}$, with sign $(-1)^{3+3}=+1$:
\[ y = +\begin{vmatrix} 1 & 0 \\ -1 & 1 \end{vmatrix} = 1-0 = 1. \]
Step 4: Add them.
So $x+y = 4+1 = 5$.
\[ \boxed{x+y = 5} \]