Step 1: Find the inverse
$\det A=\frac14(1+3)=1$. For a $2\times2$ matrix with determinant $1$, the inverse swaps the diagonal entries and negates the off-diagonal ones.
So $A^{-1}=\frac12\begin{bmatrix}-1&\sqrt3\\-\sqrt3&-1\end{bmatrix}$.
Step 2: Square $A$
$A^2=\frac14\begin{bmatrix}1-3&2\sqrt3\\-2\sqrt3&-2\end{bmatrix}=\frac12\begin{bmatrix}-1&\sqrt3\\-\sqrt3&-1\end{bmatrix}$.
This equals $A^{-1}$, so $A^{-1}-A^2=O$. It is null, diagonal and scalar, but not a unit matrix. Option (B).
Final Answer:
Option (B).
\[ \boxed{\text{(B)}} \]