Step 1: Set up a recursion for the count of involutions.
Let $I(n)$ be the number of involutions (permutations with $f(f(n)) = n$) on a set of $n$ labeled elements. Look at where the last element, call it element $n$, is sent by $f$.
Step 2: Split on the image of element $n$.
Either $f(n) = n$, so element $n$ is fixed and the remaining $n-1$ elements form an involution on their own: this gives $I(n-1)$ possibilities.
Or $f(n) = k$ for some other element $k$. Since $f$ is an involution, $f(k)$ must equal $n$ as well, so $n$ and $k$ form a swapped pair. There are $n-1$ choices for $k$, and the remaining $n-2$ elements form an involution among themselves, giving $(n-1) \cdot I(n-2)$ possibilities.
So the recursion is $I(n) = I(n-1) + (n-1) I(n-2)$, with $I(0) = 1$ and $I(1) = 1$.
Step 3: Build up the values for $n = 0$ to $4$.
$I(0) = 1$
$I(1) = 1$
$I(2) = I(1) + 1 \cdot I(0) = 1 + 1 = 2$
$I(3) = I(2) + 2 \cdot I(1) = 2 + 2 = 4$
$I(4) = I(3) + 3 \cdot I(2) = 4 + 3 \cdot 2 = 4 + 6 = 10$
Final Answer:
The number of bijections on $S = \{1,2,3,4\}$ with $f(f(n)) = n$ for all $n$ is 10.
\[ \boxed{10} \]