When talking about the speed of light are we referring to visible light? Wouldn't X-rays and gamma rays travel faster than Visible or UV light?
Nope, the speed of light is a constant for all forms of massless radiation. Higher energy radiation doesn't go faster, it has a shorter wavelength.
I think there are a few fringe theories on the nature of time that postulate that high-frequency gamma rays may arrive faster than lower-frequency ones, but those are far from as well substantiated as the one constant to rule them all.