AMD: Zen 3 Speculation, Rumours and Discussion

[Kaotik]

I can have a lot more sata drives than I have pcie slots and connectors ? I have 8 sata connectors , 1 nvme m.2 and 1 sata m.2. Also motherboards are only so big , so it sounds like i'd be greatly limited by that

Yes, when your board is like that it can be an issue, meant more for future that you'd be better off looking for a board which isn't filled by SATA connectors and instead uses the PCIe lanes for something useful (SATA connectors except for the standard 4 eat PCIe lanes), than thinking of getting more SATA drives

One has to wonder how large obstacle the 4lane pcie4 bus between chipset and io-die is for zen2 based systems? If the data being read has to go from disk/network to cpu/gpu it looks like single fast nvme ssd could hog all the chipset to io-controller bandwidth and leave nothing for rest of the system.

Neglible if anything, at least some motherboard manufacturers even suggest using chipset connected slot first because it's further away from video card and thus less likely to overheat due surrounding components, heck only X570 even has that PCIe 4.0 x4 to chipset, everything else has 3.0 x4 (which of course means you can't use PCIe4 SSD in there, but since PCIe 3.0 drives actually might saturate the bus unlike 4.0 drives, and they're still fine, I don't see any real way you could have issues unless you have like several 10 Gbit network adapters or something blasting full steam ahead)

It feels odd how few pcie4 lanes gaming cpu's get. It's definitely possible to have much more as shown by threadripper. Maybe the move is to use only 8x pcie4 lanes for gpu and then connect 2 nvme ssd's to regular pcie slots. Leave chipset for networking/USB/legacy storage that can have variation on quality of service depending how many components in chipset are active and how much load there is in that single 4lane pcie bus. Network traffic etc. flows via chipset so having ssd's connected directly to cpu(not chipset) might make system performance more stable.

It would be interesting thing to benchmark motherboards once we have real fast pcie4 ssd's available.

It's because most people really don't need anymore than those 20 lanes from the CPU. Extra lanes cost extra money, they make the chip grow.
Just look at how much empty space the Matisse I/O-die already has simply because it needs to have room on the sides for all those PHYs

edit: source for the die shot

https://flic.kr/p/48319140097

edit2: it parses flickr automatically, removed embedded image then

 

[manux]

Neglible if anything, at least some motherboard manufacturers even suggest using chipset connected slot first because it's further away from video card and thus less likely to overheat due surrounding components, heck only X570 even has that PCIe 4.0 x4 to chipset, everything else has 3.0 x4 (which of course means you can't use PCIe4 SSD in there, but since PCIe 3.0 drives actually might saturate the bus unlike 4.0 drives, and they're still fine, I don't see any real way you could have issues unless you have like several 10 Gbit network adapters or something blasting full steam ahead)

It's because most people really don't need anymore than those 20 lanes from the CPU. Extra lanes cost extra money, they make the chip grow.
Just look at how much empty space the Matisse I/O-die already has simply because it needs to have room on the sides for all those PHYs

Neglible using existing ssd's because they are not capable of saturating single pcie4 4 lane bus. Towards end of this year one can buy nvme ssd's where singe ssd can saturate 4 lane pcie4 bus. At that point if you tried to use both fast nvme ssd and fast network/sata drive something would have to throttle as there isn't enough bandwidth available. Not sure what would happen if one attached 2 fast nvme ssd's to chipset. Would something like copying data between drives work at full speed(everything under chipset) or would it be half speed due to the slow link between chipset/die if cpu has to touch the data in question.

 

[manux]

Only few pcie4 lanes business is bad imho. Hopefully we get more by time zen4 ships. That would be roughly timeline when I want to upgrade my pc. I'm looking at something like 16core/32thread cpu, next gen optane for boot/os/applications, pcie4 nvme ssd for games and another slower large nvme ssd for storage/older games not benefiting from faster ssd. I really would like all of those 3 separate ssd's to work at full speed and not affect things like networking due to insufficient bandwidth being available.

 

[Kaotik]

Neglible using existing ssd's because they are not capable of saturating single pcie4 4 lane bus. Towards end of this year one can buy nvme ssd's where singe ssd can saturate 4 lane pcie4 bus. At that point if you tried to use both fast nvme ssd and fast network/sata drive something would have to throttle as there isn't enough bandwidth available. Not sure what would happen if one attached 2 fast nvme ssd's to chipset. Would something like copying data between drives work at full speed(everything under chipset) or would it be half speed due to the slow link between chipset/die if cpu has to touch the data in question.

It's neglible even on PCIe 3.0 drives with PCIe 3.0 link between chipset and cpu, and those might actually saturate the bus unlike PCIe4 currently.
Of course there are ways to make it a problem, but it's not something most could ever face in actual use and if they do, they most likely should have gone Threadripper or Epyc to begin with, rather than consumer Ryzen (for other reasons besides I/O-bandwidth, too)

Only few pcie4 lanes business is bad imho. Hopefully we get more by time zen4 ships. That would be roughly timeline when I want to upgrade my pc. I'm looking at something like 16core/32thread cpu, next gen optane for boot/os/applications, pcie4 nvme ssd for games and another slower large nvme ssd for storage/older games not benefiting from faster ssd. I really would like all of those 3 separate ssd's to work at full speed and not affect things like networking due to insufficient bandwidth being available.

Unless you plan to be copying files 24/7 from drive to drive while also downloading from some 10 Gbit ethernet link at full blast, you shouldn't face any issues.

 

[manux]

It's neglible even on PCIe 3.0 drives with PCIe 3.0 link between chipset and cpu, and those might actually saturate the bus unlike PCIe4 currently.
Of course there are ways to make it a problem, but it's not something most could ever face in actual use and if they do, they most likely should have gone Threadripper or Epyc to begin with, rather than consumer Ryzen (for other reasons besides I/O-bandwidth, too)


Unless you plan to be copying files 24/7 from drive to drive while also downloading from some 10 Gbit ethernet link at full blast, you shouldn't face any issues.

Key word here is Most. Most is not a highend gaming pc. Most doesn't apply to what I want/need.. Most is not yet directstorage compliant and most doesn't yet run unreal5 based games without compromise. At least for me the idea is to have apps/os and games in different disks while allowing full speed access to both. Not have to do something like forbid steam updates or file sharing in local network or close apps when playing game. Not hitting some obscure edge cases causing random stutters.

We go to crazy lengths and arguments when looking at 1% low frames in game/gpu reviews. If you apply same attitude towards ssd's interesting things start to pop up. Especially once we have those pcie4 4 lane saturating ssd's available later this year.

For me things will hopefully align around zen4. By that time unreal5, directstorage, faster nvme ssd's/optanes are out etc. I hope we have better io at that time. Current state for highend gaming is just sad. Maybe by time zen4 ships pcie5 comes out. Maybe same amount of lanes but double the bandwidth?

 

[Kaotik]

Key word here is Most. Most is not a highend gaming pc. Most doesn't apply to what I want/need.. Most is not yet directstorage compliant and most doesn't yet run unreal5 based games without compromise. At least for me the idea is to have apps/os and games in different disks while allowing full speed access to both. Not have to do something like forbid steam updates or file sharing in local network or close apps when playing game. Not hitting some obscure edge cases causing random stutters.

We go to crazy lengths and arguments when looking at 1% low frames in game/gpu reviews. If you apply same attitude towards ssd's interesting things start to pop up. Especially once we have those pcie4 4 lane saturating ssd's available later this year.

For me things will hopefully align around zen4. By that time unreal5, directstorage, faster nvme ssd's/optanes are out etc. I hope we have better io at that time. Current state for highend gaming is just sad. Maybe by time zen4 ships pcie5 comes out. Maybe same amount of lanes but double the bandwidth?

Maybe you should first wait for the DirectStorage to actually launch so we know what it needs. And we don't know whether next gen SSDs will saturate the PCIe4 bus or not.

 

[manux]

Maybe you should first wait for the DirectStorage to actually launch so we know what it needs. And we don't know whether next gen SSDs will saturate the PCIe4 bus or not.

That's exactly what I'm doing. I'm waiting until 2022 to update my rig. It's because I really want to see how/if consoles affect pc and I also want to see where unreal5/directstorage is going.

For ssd's this intel ssd is 7GB/s read: https://www.anandtech.com/show/15860/intel-announces-d7-series-pcie-40-enterprise-ssds
Samsung has something similarly specced coming out late this year. Those already are awfully close to the theoretical max of 8GB/s for 4 lane pcie bus.

Not sure why it's so difficult to accept claim current pcie lanes is not optimal for highend gaming rig? 4 or 8 lanes more would give substantial benefit be it adding those to cpu or making the chipset to io controller interface wider(8 or even 16 lanes). Of course if one updates pc every other year the issue will look very different than from my POV where I keep same machine for very long time(other than adding storage and switching gpu to newer model).

 

[Kaotik]

That's exactly what I'm doing. I'm waiting until 2022 to update my rig. It's because I really want to see how/if consoles affect pc and I also want to see where unreal5/directstorage is going.

For ssd's this intel ssd is 7GB/s read: https://www.anandtech.com/show/15860/intel-announces-d7-series-pcie-40-enterprise-ssds
Samsung has something similarly specced coming out late this year. Those already are awfully close to the theoretical max of 8GB/s for 4 lane pcie bus.

Not sure why it's so difficult to accept claim current pcie lanes is not optimal for highend gaming rig? 4 or 8 lanes more would give substantial benefit be it adding those to cpu or making the chipset to io controller interface wider(8 or even 16 lanes). Of course if one updates pc every other year the issue will look very different than from my POV where I keep same machine for very long time(other than adding storage and switching gpu to newer model).

So far the difference is either non-existent or neglible even with PCIe 3.0 x4 link which we know can get saturated much easier, that's why I'm not worrying.

If you believe 20 lanes isn't enough from CPU, you can always go Threadripper, then you'll have plenty more. Product designed for the masses won't be designed because some outlier cases where 20 lanes + more behind x4 link might not be enough in every imaginable scenario (but still would be in most).

And since you mentioned those .1% FPS etc, if you think SSD behind too slow link can cause spikes there, maybe you should do benchmarks to show it?

I'll also repeat that the current I/O die is already larger than it could be if it wasn't for the PHYs requiring a chip of that size, adding more would add just more and more dead space on it. On 7nm there would be even more dead space even with current configuration simply because PHYs won't scale like the logic does.

 

[manux]

And since you mentioned those .1% FPS etc, if you think SSD behind too slow link can cause spikes there, maybe you should do benchmarks to show it?
.

I think the proof will be in unreal5/directstorage pudding and how things age. For me, I'll wait and hope for better future by the time I'm upgrading.

I'm not in a moment, my thinking is much more long term. I bought my current pc in 2013. If I manage to keep it until 2022 that is 9 years of use out of same mobo+cpu. I did do midlife gpu and storage and usb port upgrades via pcie cards. If I buy let's say zen4 in 2022 and plan to upgrade somewhere around 2030 that is a long time for that machine to cover. If consoles/direct storage takes off the io will be super important during that time. Especially so if I do some multitasking while gaming(downloads/installs, video imports from camera/rendering, streaming, serve files to local network,...).

Threadripper is not option for me. It's too cumbersome and not great for gaming. If I was to build 2 separate machines I would build one with threadripper for productivity and another one with intel just for gaming. But I'm not going to build 2 machines.

 

[Kaotik]

Threadripper is not option for me. It's too cumbersome and not great for gaming. If I was to build 2 separate machines I would build one with threadripper for productivity and another one with intel just for gaming. But I'm not going to build 2 machines.

Well I'm sure we all have our "dream configurations", but as you're surely aware, no-one is going to serve those dreams.
The companies make few different types of CPUs to serve different needs, consumer products are fine with 20 lanes + 4 lanes to chipset because that's more than enough for most, if it's not for you, they offer Threadripper and Epyc to serve those needs.
Also, 3rd gen Threadripper fixed the "cumbersome issues" aka NUMA nodes and instead offers just UMA, it's gaming performance is fine and around what you'd expect at their clocks. Obviously the extra cores are pretty much wasted for gaming, but if you want loads of I/O, you really don't have other options than to go HEDT (or server hardware)

 

[manux]

Well I'm sure we all have our "dream configurations", but as you're surely aware, no-one is going to serve those dreams.
The companies make few different types of CPUs to serve different needs, consumer products are fine with 20 lanes + 4 lanes to chipset because that's more than enough for most, if it's not for you, they offer Threadripper and Epyc to serve those needs.
Also, 3rd gen Threadripper fixed the "cumbersome issues" aka NUMA nodes and instead offers just UMA, it's gaming performance is fine and around what you'd expect at their clocks. Obviously the extra cores are pretty much wasted for gaming, but if you want loads of I/O, you really don't have other options than to go HEDT (or server hardware)

My feeling is that 4 more lanes to chipset would just fix the situation for 99.99% of use cases. My major gripe is that if you put fast pcie4 nvme ssd and connect that to chipset it basically can use all available bandwidth. having 4 more lanes there would allow ssd+rest work without potentially interfering with each other. Obviously we would not yet see this issue as nobody outside samsung/intel/... has those ssd's available. Also we don't yet have games based on unreal5 engine that might stress streaming. I'm firmly on complain about current situation now, wait until 2022 to upgrade camp.

I suppose the way to go is to put the fastest disk into pcie4 slot connected directly to cpu. Make sure cooling is not issue. This drive can run full tilt without throttling anything else down. On chipset side it might make sense to buy pcie3 nvme ssd, use half the pcie4 4 lane bandwidth for ssd and save rest for other things connected to chipset. If one needs third drive then connect it via the USB ports that are connected directly to cpu(avoid overloading chipset)

If I have to go 2 drives route that probably kills my dream of optane as boot drive. In that scenario I would likely opt for much bigger and cheaper pcie4 nvme ssd for boot+apps+games. Secondary slower and even bigger nvme drive would be for older games and storage.

 

[Kaotik]

My feeling is that 4 more lanes to chipset would just fix the situation for 99.99% of use cases. My major gripe is that if you put fast pcie4 nvme ssd and connect that to chipset it basically can use all available bandwidth. having 4 more lanes there would allow ssd+rest work without potentially interfering with each other. Obviously we would not yet see this issue as nobody outside samsung/intel/... has those ssd's available. Also we don't yet have games based on unreal5 engine that might stress streaming. I'm firmly on complain about current situation now, wait until 2022 to upgrade camp.

I suppose the way to go is to put the fastest disk into pcie4 slot connected directly to cpu. Make sure cooling is not issue. This drive can run full tilt without throttling anything else down. On chipset side it might make sense to buy pcie3 nvme ssd, use half the pcie4 4 lane bandwidth for ssd and save rest for other things connected to chipset. If one needs third drive then connect it via the USB ports that are connected directly to cpu(avoid overloading chipset)

If I have to go 2 drives route that probably kills my dream of optane as boot drive. In that scenario I would likely opt for much bigger and cheaper pcie4 nvme ssd for boot+apps+games. Secondary slower and even bigger nvme drive would be for older games and storage.

Again, I can't stress this enough, we have PCIe 3.0 x4 link between every other AMD consumer CPU and chipset besides X570. We have devices which could saturate it. We don't have, at least as far as I know, any benchmarks or other studies showing any issues or performance loss from having only that PCIe 3.0 x4 link to the chipset.
Why on earth would it be any different when we get to saturate PCIe 4.0 x4 link?

 

[manux]

Again, I can't stress this enough, we have PCIe 3.0 x4 link between every other AMD consumer CPU and chipset besides X570. We have devices which could saturate it. We don't have, at least as far as I know, any benchmarks or other studies showing any issues or performance loss from having only that PCIe 3.0 x4 link to the chipset.
Why on earth would it be any different when we get to saturate PCIe 4.0 x4 link?

We don't yet have directstorage, unreal5 and next gen console inspired games available. World might or might not change a lot once next gen consoles are baseline for gaming. Hence my waiting stance as at this time so many things are moving that predicting future is going to be terribly difficult. Is ps5 nvme ssd still too slow and pc will leapfrog to provide real 4k and real source level assets? Does it turn out that in reality 2GB/s is enough and any old nvme ssd is more than enough? I don't know. I'm going to wait it out as I can keep suffering with my old pc for couple of more years.

 

[pjbliverpool]

I don't think bandwidth from the chipsets going to be an issue because you're rarely going to be using the full transfer speed of the drive and when you do it's going to be in relatively short bursts at times when nothing else hanging off the chipset will be doing anything.

I guess it may add some latency although I'm not sure if that's an issue in terms of the overall timescales in play.

 

[Kaotik]

We don't yet have directstorage, unreal5 and next gen console inspired games available. World might or might not change a lot once next gen consoles are baseline for gaming. Hence my waiting stance as at this time so many things are moving that predicting future is going to be terribly difficult. Is ps5 nvme ssd still too slow and pc will leapfrog to provide real 4k and real source level assets? Does it turn out that in reality 2GB/s is enough and any old nvme ssd is more than enough? I don't know. I'm going to wait it out as I can keep suffering with my old pc for couple of more years.

But again and again, if saturating PCIe 3.0 x4 isn't a problem, why would saturating PCIe 4.0 x4 be a problem?

 

[Lightman]

AMD already solved this problem and enabled 8x PCIe link between CPU and chipset.

I see no problem here, just spend more money for less bottlenecks and buy Threadripper 3000

 

[BRiT]

I wish NVME would be a bit more like SATA, in terms of expandability. Specifically, where you can connect multiple NVME devices to the interface point (NVME PCI-Express Card or motherboard) and if you use one device at a time then it gets the full bandwidth.

I'm less concerned about maximum throughput on ALL NVME when ALL are used, as long as 2 NVME devices can get max performance at the same time. This way I can have one OS NVME and start with one GAME NVME, then when that runs low on space I can add in another one. It seems a silly when you upgrade storage space with NVME you discard the old NVME entirely. At least in most situations where you have limited number of NVME slots.

 

[pjbliverpool]

AMD already solved this problem and enabled 8x PCIe link between CPU and chipset.

I thought AMD was using a 4x link? 24 lanes total, 4 for chipset, 16 for GPU and 4 for NVMe.

EDIT: manux already posted the diagram above confirming this.

 

[Kaotik]

I thought AMD was using a 4x link? 24 lanes total, 4 for chipset, 16 for GPU and 4 for NVMe.

EDIT: manux already posted the diagram above confirming this.

You forgot the "Threadripper"-part of Lightmans post.

 

[pjbliverpool]

You forgot the "Threadripper"-part of Lightmans post.

My bad, I saw them as two separate and disconnected statements, the first one referring to mainstream parts.

For TR the chipset shouldn't matter anyway as it already has more than enough CPU lanes.