Lots of factors, basically.
Hard drive:
Cache
RPM
Seek time
Ram:
Amount
speed
CD/DVD rom:
Speed
The Ram amount is all good and well but theres many varying speeds of ram, especially new DDR ram, everything from PC2500 through to PC4500? Them serial numbers are actually data rates in MB/sec.
Having more ram is good because you can load up the game engine to ram and then when you change map you only have to load the map data, generally having more ram is better but theres a very definate maximum for each game before performance doesnt increase. Ram speed is better here, get some nice fast ram and make sure you can run it at that speed (cpu compatability) initial load times will be better.
Hard drive speeds are important, make sure you get a 7200rpm hard drive, 5400 are to slow now a days, 8mb caches are starting to replace 2mb caches as the best. Seek times and other factors important in Hard drive times are usualy relative to how large the drive is, best to use a smaller faster primary drive for the OS (C
and have a backup data drive (larger and slower). For even faster hard drives you can consider raid and use striping, this requires a RAID motherboard or a RAID PCI card, and 2 Hard drives. Or Invest more money in SCSI, again can come as a PCI card, SCSI is very expensive though.
CD roms and DVD roms generally are so cheap now a days if you dont have a 52 speed CD rom then splash out a tenner and get one
Full game installs on the HDD help loading times, and IF you own a leagal copy of the game then maybe install a crack so that CD checks are removed, cracks from
www.gamecopyworld.com (perfectly legal NON-warez site may i add) For instance many people play BF1942 online, you need a unique key and therefore have had to bought the game, but you can remove CD checks when the game loads by using a crack. Some people find connecting to servers impossible otherwise, simply takes too long.
Your video card can also affect loading times, although im not exacture sure how, maybe different speed memory? some of the new FX cards use 2.2ns memory which is very fast, do the math and its ~910Mhz DDR speed, i expect that loads texture data very fast compared to other slower memory (when clocked that high obviously) Might want to overclock your video cards memory, again im not sure if this will help loading times specifically but its a good idea to do it anyways.
CPU speeds i doubt will help improve load times, it really depends on what the game engine is doing during load, engines like UT2003 have a prep time for effects like particles getting ready etc, i expect cpu speed would improve this.