sounds like a short or the contact switch (cheap peice of junk behind the power button) in the case isnt functioning properly anymore. You'll have to open the case to mess with it. One thing you can do to see if its the power button is switch the reset (RST) button connection to the motherboard 'power on' (PWR) of the Front Panel Switches and Indicators connections (FPIO) on the motherboard. Its the area on the motherboard where all the case stuff connects too, usually the bottom right hand side of the board. So in effect what you'd be doing is making your reset button becomes your power on. There should be labels in very small print near the FPIO telling you where everything is, or you'll need your motherboard manual with the layout.
one other thing you may want to do since it takes about 2 seconds is remove and replace (not with a new one) the motherboard battery, thought that usually wont cause a delayed response.
PSU might be going as well though thats the last thing i'd check.