Step 1: Picture the three forms of carbon side by side.
Diamond has every carbon atom locked into a rigid three dimensional tetrahedral network, while graphite is made of flat hexagonal sheets stacked loosely on top of each other, and fullerene is a hollow cage of carbon atoms.
Step 2: Think in terms of energy content, not just structure.
A useful way to compare stability is to look at the heat released when equal amounts of each allotrope are burnt completely. Diamond releases slightly more heat on combustion than graphite does, which means diamond starts out at a higher energy level.
Step 3: Connect energy level to stability.
Whatever sits at the lowest energy level is the one that nature prefers to stay in, so the allotrope releasing the least energy on burning is the most settled, most stable one. That is why the standard enthalpy of formation of graphite is fixed at $0$, and every other form is measured relative to it.
Step 4: Match this to the everyday observation.
This is also why diamond, over extremely long timescales, tends to convert towards graphite and never the other way round, confirming graphite sits in the deeper energy well. \[ \boxed{\text{Graphite is the thermodynamically most stable allotrope of carbon}} \]