Step 1: Count the complement:
Count numbers that are not greater than 24000 and subtract from 120.
Step 2: Small numbers:
The only numbers below 24000 start with 23 (since digit 2 first, and second digit must be less than 4, so 3). Fill the last three places from 5, 7, 9 in $3!=6$ ways. There is no number equal to 24000 as 4 and 0 are not allowed.
Step 3: Subtract:
$120 - 6 = 114$.
Final Answer:
Subtract the 6 numbers starting with 23 from 120.
\[ \boxed{C} \]