The RSX die size I was able to google was 240 mm^2, Xenos figure was 182 mm^2 parent die and 80 mm^2 daughter die (262 mm^2 total) (all on 90 nm).
GTX580 is 520 mm^2, HD 6970 is 389 mm^2. And that's on 40nm.
So if they have roughly the same budgets as last time, you could probably fit a GTX 580 or a HD 6970 in those budgets, on 28nm (I'm assuming a rough halving of die size per node). Or, a mid range of the next gen on 22nm, so to speak.
On 22nm, on the same ~240 mm^2 budget, you could probably fit a high end Southern Islands, or Nvidia Kepler chip in there. Sort of, leaving out the EDRAM to some extent, or leaving it off to the side, or squeezing it in somehow or other, depending, and depending how much you want. Of course keeping in mind, Kepler and Southern Islands will be last gen chip by the time these consoles launch. (or again, instead of high end Kepler/SI, you could get a mid range HD9xx or Nvidia Kepler+1 for the same effect).
For EDRAM, well if you got 10MB on 90nm, 20 on 65m, 40 on 40, 80 on 28nm, 160 mb on 22nm? You could save some space there it seems, I dont think you need 160 MB am I right? So perhaps you could fit your EDRAM in a 40 mm^2 die this time, if you "only" want 80 MB of it. That is if EDRAM still makes sense next gen, I'm unclear.
I would say the one fly in all these calculations is, what if the console manufacturers next gen are willing to increase their die budget? Say up to 300 mm^2, or even higher, for the GPU?
But yeah, high end AMD HD7XXX series or mid range HD9XXX (I use 9XXX because 8XXX will just be the refresh card) doesnt sound bad at all. That's IF we get to 22nm for next gen consoles. Subtract a gen from the GPU class otherwise .
It also becomes questionable if HD 9XXX would actually be out in time. There was a little over a year between HD5870 and HD6970, both on 40nm. If Southern Islands hits in first quarter 2012, the HD8XXX would hit in 1Q 2013, and perhaps the HD9XXX in 1Q 2014 if all goes well. Again, you'd be straddling some things. But I guess for our purposes it doesnt matter to much if you saw a high end HD8XXX, or a mid range HD9XXX in the console. They should perform similarly.
So yeah actually, I think that's pretty reasonable. on 28nm you might be looking at a HD6970 performance-class chip (in presumably next box here since we know their AMD connection). On 22nm you would be looking at a high end Southern Islands performance-class chip.
Unless that's not how it works out at all, heh.