Step 1: Compute $\vec b\times\vec c$.
With $\vec b=(2,-1,0)$ and $\vec c=(0,1,2)$: \[ \vec b\times\vec c=\begin{vmatrix}\hat i&\hat j&\hat k\\2&-1&0\\0&1&2\end{vmatrix}=(-2,-4,2). \]
Step 2: Its length.
$|\vec b\times\vec c|=\sqrt{4+16+4}=\sqrt{24}=2\sqrt6.$
Step 3: Use the given cross product size.
$|\vec a\times(\vec b\times\vec c)|=|\vec a|\,|\vec b\times\vec c|\sin\dfrac{\pi}{3}.$ So $3\sqrt2=|\vec a|\cdot2\sqrt6\cdot\dfrac{\sqrt3}{2}.$
Step 4: Solve for $|\vec a|$.
The right side is $|\vec a|\sqrt6\sqrt3=|\vec a|\sqrt{18}=3\sqrt2\,|\vec a|.$ So $3\sqrt2=3\sqrt2\,|\vec a|$, giving $|\vec a|=1.$
Step 5: Use $|\vec a+\vec b|=3$.
$|\vec b|=\sqrt{2^2+(-1)^2}=\sqrt5.$ Square the given: $|\vec a+\vec b|^2=|\vec a|^2+|\vec b|^2+2\vec a\cdot\vec b$, so $9=1+5+2\vec a\cdot\vec b.$
Step 6: Solve for the dot product.
$2\vec a\cdot\vec b=3$, so $\vec a\cdot\vec b=\dfrac32.$ \[ \boxed{\dfrac32} \]