1. Would it have been viable for Sony to have Nvidia design a 256 bit BUS to the GDDR3 while keeping that 128 bit to the XDR? How expensive would it have been?
2. In your opinion would it have been wiser to have 512 mb pool of GDDR3 RAM for both the CELL and RSX, with the inclusion of a 256 bit BUS?
1 -- RSX's link to XDR isn't 128-bit, it never has been, and all cases where someone said it was 128-bit are people assuming that it's bandwidth inherently implies that it must be 128-bit because other buslinks in the same general bandwidth range are that wide. CELL's XDR bus is 64-bit, and the FlexIO link is also 64-bit (and that's 8 lanes, 8bits each, at 5 GB/sec each or so it would seem)... RSX gets 7 of those 8 lanes, making for a 56-bit bus (and that's both directions put together).
As for would it have been viable? Possible, yes. Cheap, no. Especially if you assume that RSX was meant to have that GDDR-3 on the same package with the GPU... because of the nature in which the components are manufactured, a wider GDDR-3 bus usually means more DRAM devices, which is invariably more expensive by far than RSX's core even if each DRAM is lower capacity.
2 -- No, never. Latency is the bane of all performance, and GDDR-3 is high latency and putting it behind the wall known as RSX makes it a thousand times worse, to say nothing of the fact that a GPU is a very bandwidth hungry device, so it's going to be monopolizing that bus pretty much all the time, which makes matters worse for the CPU, which makes it a trillion times worse... CPUs are far more sensitive to latency than GPUs since there's no guarantee of work filling in those latencies, and even if you did, there's always dependencies between contexts which, again, isn't the case with GPUs. Similarly, because of all those SPEs doing work that affects other work, you end up with Cell being all the more dependent on getting loads of small chunks of data "Right Now." So while there are disadvantages, split memory pools works out as the lesser of a thousand evils.
Though I wouldn't have minded having 512 MB just in the RSX pool, though even more so, I would have liked it on the CPU side. Software can never have too much RAM. There's no such thing.