Step 1: Triple Product:
Compute $\vec a\cdot(\vec b\times\vec c)$ for $\vec a=(1,x,1)$, $\vec b=(0,1,x)$, $\vec c=(x,0,1)$. $\vec b\times\vec c=(1,\,x^2,\,-x)$. Dot with $\vec a$: $1+x^3-x$.
Step 2: Calculus:
Derivative $3x^2-1$ is zero at $x=\pm1/\sqrt3$, and changes sign from negative to positive at $+1/\sqrt3$, so it is a local minimum of the volume expression.
Step 3: Decide:
Of the listed values, $1/\sqrt3$ corresponds to the minimum point and gives the smallest volume (about 0.615). Option (C).
Final Answer:
Option (C).
\[ \boxed{\text{(C) } \frac{1}{\sqrt{3}}} \]