Just so you know, drift velocity is proportional to mobility at a given voltage, so you can't have one stay the same and the other change.
MistaPi, the real reason that there's no real impact is that the signal speed - which is orders of magnitude faster than the electron velocity - is independent of temperature. If you're curious about the difference, consider that it takes a second or two to suck water up a straw, but a tenth of a millisecond for your mouth's sucking force to travel down the straw and show movement in the cup.
Erm, signal speed in CMOS has everything to do with electron mobility. A transistor doesn't switch the minute an electron is nudged on its gate. A sufficient number of electrons has to either be pushed onto the gate (discharge) or attracted away from the gate (charge).
When the charge accumulated on the gate is enough to create a potential at the threshold voltage of the source-channel junction, the transistor moves into conducting range (well linear region at first, but MOSFETs will quickly move into saturation).
How fast the previous transistor can charge up the gate of the transistor its driving as well as the wire connecting them is mainly a function of how much current that driving transistor can supply -- why do you think performance characteristics of process technologies are all measured in Ion (current while on) and Ioff (current while off)?
Current supplied is a function of temperature (which affects electron mobility), voltage, material (base mobility, dielectric constant), and the dimensions of the transistor. There are also secondary effects that are starting to make a big difference at the smaller (45nm, 28nm, etc.) geometries such as diffusion density, temperature inversion, etc.
Interestingly, starting at around 65nm, the carrier mobility is actually highest at around ~0C. Lower temperatures actually reduce electron mobility due to inhibiting drift.