Step 1: See it is homogeneous.
Every term in $(3xy + y^2)dx + (x^2 + xy)dy = 0$ has degree two, so put $y = vx$, which gives $\dfrac{dy}{dx} = v + x\dfrac{dv}{dx}$.
Step 2: Substitute and separate.
After substituting and tidying, $x\dfrac{dv}{dx} = \dfrac{-2v^2 - 4v}{1 + v}$, which separates to $\dfrac{(1+v)\,dv}{2v^2 + 4v} = -\dfrac{dx}{x}$.
Step 3: Integrate.
This gives $\tfrac14\log(v^2 + 2v) = -\log x + \log c$, so $(v^2 + 2v)x^4 = c$.
Step 4: Put $v = y/x$ back.
$$\left(\frac{y^2}{x^2} + \frac{2y}{x}\right)x^4 = c \Rightarrow x^2(y^2 + 2xy) = c$$
\[ \boxed{x^2(2xy + y^2) = c} \]