To simplify the expression \( \cos^{-1} \left( \frac{12}{13} \cos x + \frac{5}{13} \sin x \right) \) for \( \frac{\pi}{2} \leq x \leq \frac{3\pi}{4} \), we proceed as follows:
- The expression inside the inverse cosine function can be rewritten using a trigonometric identity. Let \( a = \frac{12}{13} \) and \( b = \frac{5}{13} \).
- Since \( \sqrt{a^2 + b^2} = \sqrt{\left(\frac{12}{13}\right)^2 + \left(\frac{5}{13}\right)^2} = 1 \), the expression can be transformed into the form \( \cos(\theta) \cos x + \sin(\theta) \sin x = \cos(x - \theta) \).
- The angle \( \theta \) is defined by \( \theta = \tan^{-1}\left(\frac{b}{a}\right) = \tan^{-1}\left(\frac{5}{12}\right) \).
- Thus, the original expression becomes \( \cos^{-1}(\cos(x - \theta)) \).
- Given that \( \cos \) is non-increasing in the specified domain, \( \cos^{-1}(\cos(x - \theta)) = x - \theta \).
- Substituting the value of \( \theta \), we get \( \cos^{-1} \left( \frac{12}{13} \cos x + \frac{5}{13} \sin x \right) = x - \tan^{-1}\left(\frac{5}{12}\right) \).
This result corresponds to the provided option.
Hence, the correct answer is
Option: \( x - \tan^{-1} \left(\frac{5}{12}\right) \).