If you're going to be downtown, I suggest checking out Yorkville/Bay & Bloor, or the St. Lawrence/Front Street area. The Financial District is pretty cool too, but if you're only going to be around for half a day, Yorkville/Bay & Bloor is probably your best bet. That's where all the super-trendy shopping, boutiques, cafes, celebrity hangouts, etc. is.