Recent research tends to suggest that transexuals have had an incorrect amount of hormones in the womb at critical times of developement. For instance, a male foetus doesn't get the right level of hormones at the right time, so you end up with a male body and the (default) female brain. When they scan the brain of a male-female transexual, they show the same activity female brains, not male brains. The body is male, but the brain is female.
This is quite different from male homosexuals who have a male brain but just fancy other blokes. There's some tentative evidence to suggest it is a genetic disposition, but the jury is still out on that one.
The arguments based on genetics or development in the womb are of course difficult to separate out from all the various (and different) social and environmental things that happen to define a person's sexuality as they grow into adults, and ones tends to obscure the other. This is why even now there are still so many "nature versus nurture" arguments about this and many other subjects. I suspect that it's probably a bit of both.