Before emission the nucleus is stationary, so its momentum is zero. Momentum must stay zero for the whole system.
The photon carries momentum $p_\gamma = \dfrac{E}{c}$. To balance this, the nucleus gains momentum of the same size in the opposite direction: $p_N = \dfrac{E}{c}$.
A nucleus with momentum $p_N$ and mass $M$ has kinetic energy\[K = \frac{p_N^2}{2M} = \frac{E^2}{2Mc^2}.\]The stored internal (nuclear) energy that is released has to supply two things: the energy carried away by the photon, $E$, and the kinetic energy of the recoiling nucleus, $K$. Adding them,\[\Delta U = E + \frac{E^2}{2Mc^2}.\]So the drop in internal energy is a little larger than the photon energy alone, because of the small recoil.\[\boxed{\Delta U = E + \frac{E^2}{2Mc^2}}\]