Step 1: Understand diminishing roots.
Diminishing every root by $h$ means substituting $x=y+h$. The new constant term is just $f(h)$, the original polynomial evaluated at $h$.
Step 2: Set the constant term to zero.
For the constant term to vanish we need $f(h)=0$, so $h$ must itself be a root of $f$.
Step 3: Write the equation for $h$.
That is $h^5-3h^4-5h^3+27h^2-32h+12=0$.
Step 4: Find the roots by testing.
Try $h=1$: $1-3-5+27-32+12=0$, so $h=1$ works. Continuing the factoring gives roots $h=1,1,2,3,-2$ (with $1$ a double root).
Step 5: List the possible values of $h$.
The values of $h$ that make the constant term disappear are $1,1,2,3,-2$.
Step 6: Add their squares.
Sum of squares $=1^2+1^2+2^2+3^2+(-2)^2=1+1+4+9+4=19$. \[ \boxed{19} \]