AMD Bulldozer Core Patent Diagrams

fellix said:
Speaking of thread management, if someone here have access to a Bulldozer system, I would be grateful for a test run with this little console application in the post attachment. It measures the sync latency between all the CPUs and cores (physical/virtual/local/remote) that are present in the system.
Click to expand...

It's quite...slow. 30ns for same module, 116 for across module. At 3.6GHz that translates into about 108 cycles for same module and a whopping 417.6 cycles across modules. Now, I don't know how the test is structured (if there are any details please share), but if it's doing some message passing (Send->RSVP, for example), there'll be some overhead associated with OS messaging - still, it looks rather sub-mediocre. This is under Win 8, by the way.
 
AlexV said:
It's quite...slow. 30ns for same module, 116 for across module. At 3.6GHz that translates into about 108 cycles for same module and a whopping 417.6 cycles across modules. Now, I don't know how the test is structured (if there are any details please share), but if it's doing some message passing (Send->RSVP, for example), there'll be some overhead associated with OS messaging - still, it looks rather sub-mediocre. This is under Win 8, by the way.
Click to expand...
Hmm sync latency is higher than with Core2, which was only connected via FSB?
 
AlexV said:
It's quite...slow. 30ns for same module, 116 for across module. At 3.6GHz that translates into about 108 cycles for same module and a whopping 417.6 cycles across modules.
Click to expand...
Damn. The intermodule latency is actually worse than Phenom I numbers (TLB fixed), as far as I can remember. But the latency inside a module is puzzling. Is it possible that the rather small coalescing cache is wrecking something here (on top of other things)?
 
There were similar problems with Barcelona when it launched, though I think more in-depth testing showed that this was a worst-case result. This poor showing belied the stupid "native quad core" marketing AMD put out, since an inferior FSB setup had no problem keeping up with it.

It seems to point out that the uncore for Bulldozer has not moved very far from 10h. I had posted earlier that I expected some kind of improvement over 10h with regards to the L3 and uncore. It doesn't seem I was right.

I wish there were some occasion that AMD would surprise me by doing better than expected.
 
AMD doesn't actually need a super-fast L3, since their implementation is [mostly] exclusive to the higher levels and the L2 caches are the ones being truly burdened with the coherency traffic. But still, at least BD manages to bump read bandwidth from its L3 to more "modern" levels, probably thanks to the new bank-interleaved organisation. The access latency took a hit, though.
It's the L2 overall performance that's the troubling factor here -- at least they got the size right, as a compensation. The L3 is doing its job, and this time AMD managed to improve its SRAM density by a ~20% over the L2.
 
fellix said:
AMD doesn't actually need a super-fast L3, since their implementation is [mostly] exclusive to the higher levels and the L2 caches are the ones being truly burdened with the coherency traffic.
Click to expand...
Is that certain? The ability to prefetch directly to L1 and bypass the L2 means there is probably a snoop to the L2 and then to both L1 caches.
 
3dilettante said:
Is that certain? The ability to prefetch directly to L1 and bypass the L2 means there is probably a snoop to the L2 and then to both L1 caches.
Click to expand...
Don't forget that the L1d caches are mostly inclusive to the L2 with WT policy. Which on the other hand doesn't guarantee that the L2 would contain the most recent and valid data bits from the L1, but still it has to be updated also.
 
Some guy ran the timing test on FX4100@ default:
Code: 
CPU0<->CPU1:       33.8nS per ping-pong 
CPU0<->CPU2:      136.7nS per ping-pong 
CPU0<->CPU3:      135.9nS per ping-pong 
CPU1<->CPU2:      136.1nS per ping-pong 
CPU1<->CPU3:      136.4nS per ping-pong 
CPU2<->CPU3:       33.6nS per ping-pong
 
hoho said:
Some guy ran the timing test on FX4100@ default:
Code: 
CPU0<->CPU1:       33.8nS per ping-pong 
CPU0<->CPU2:      136.7nS per ping-pong 
CPU0<->CPU3:      135.9nS per ping-pong 
CPU1<->CPU2:      136.1nS per ping-pong 
CPU1<->CPU3:      136.4nS per ping-pong 
CPU2<->CPU3:       33.6nS per ping-pong
Click to expand...
Almost 100+ ns? How? That's about the same level as some MCM does.
 
I think it's going to be a problem that ALL AMD CPU share

Phenom II X6 1055T -- 2800Mhz

CPU0<->CPU1: 112.2nS per ping-pong
CPU0<->CPU2: 111.9nS per ping-pong
CPU0<->CPU3: 111.6nS per ping-pong
CPU0<->CPU4: 111.7nS per ping-pong
CPU0<->CPU5: 112.1nS per ping-pong
CPU1<->CPU2: 112.6nS per ping-pong
CPU1<->CPU3: 112.6nS per ping-pong
CPU1<->CPU4: 112.7nS per ping-pong
CPU1<->CPU5: 111.7nS per ping-pong
CPU2<->CPU3: 112.6nS per ping-pong
CPU2<->CPU4: 113.8nS per ping-pong
CPU2<->CPU5: 112.5nS per ping-pong
CPU3<->CPU4: 113.2nS per ping-pong
CPU3<->CPU5: 112.8nS per ping-pong
CPU4<->CPU5: 112.8nS per ping-pong
=============================================
Core 2 Duo E6700 -- 2670Mhz

CPU0<->CPU1: 33.9nS per ping-pong


This result is made by two of my friends
 
3dilettante said:
Is that certain? The ability to prefetch directly to L1 and bypass the L2 means there is probably a snoop to the L2 and then to both L1 caches.
Click to expand...

Maybe they do something stupid, like wait for store queues to drain before reading data from L2 when tags indicate a hit in a foreign L1 cache. You end up with something like this:

Miss own L1 cache
Miss own L2 cache
Broadcast read-request through uncore.
Tags in other modules are checked.
Hit in foreign L1
Wait for foreign L1 store queue to drain
Read data from foreign L2.

Cheers
 
The 30ns time period for the in-module transfer may be due to the draining of queues and traffic to the WCC, L2 and L1.
The weird part is when it goes across the interconnect, where the latency shoots up for all AMD chips.
Is it querying the memory controller, is it the SRQ?

Are there outputs for dual/quad/hex AMD cores for comparison?
 
3dilettante said:
Are there outputs for dual/quad/hex AMD cores for comparison?
Click to expand...
Unfortunately people in the other forum where I asked them to run the app haven't really done much testing. Only semi-interesting thing I have is from s939 x2:
CPU0<->CPU1: 120.9nS per ping-pong

Not sure what specific model or at what speed it was.
 
That big chunk of time spent in the uncore puzzles me.
Perhaps it is waiting on the SRQ to process through or buffers in the IMC to empty.
I wonder what Intel is doing differently, since it has had an integrated north bridge since Nehalem.
 
My PII 940 @ 3000Mhz
Code: 
CPU0<->CPU1:      113.6nS per ping-pong
CPU0<->CPU2:      113.5nS per ping-pong
CPU0<->CPU3:      113.4nS per ping-pong
CPU1<->CPU2:      114.2nS per ping-pong
CPU1<->CPU3:      112.8nS per ping-pong
CPU2<->CPU3:      113.0nS per ping-pong
 
AlexV said:
Now, I don't know how the test is structured (if there are any details please share), but if it's doing some message passing (Send->RSVP, for example), there'll be some overhead associated with OS messaging - still, it looks rather sub-mediocre. This is under Win 8, by the way.
Click to expand...
From this article: http://www.anandtech.com/show/1910/3
Michael S. started this extremely interesting thread at the Ace's hardware Technical forum. The result was a little program coded by Michael S. himself, which could measure the latency of cache-to-cache data transfer between two cores or CPUs. In his own words: "it is a tool for comparison of the relative merits of different dual-cores.

Cache2Cache measures the propagation time from a store by one processor to a load by the other processor. The results that we publish are approximately twice the propagation time. For those interested, the source code is available here.
Click to expand...
 
3dilettante said:
That big chunk of time spent in the uncore puzzles me.
Perhaps it is waiting on the SRQ to process through or buffers in the IMC to empty.
I wonder what Intel is doing differently, since it has had an integrated north bridge since Nehalem.
Click to expand...
Nehalem does not really intergrated "North Bridge"
PCI-E isn't a part of it untill Lynnfield released

fellix said:
AMD doesn't actually need a super-fast L3, since their implementation is [mostly] exclusive to the higher levels and the L2 caches are the ones being truly burdened with the coherency traffic. But still, at least BD manages to bump read bandwidth from its L3 to more "modern" levels, probably thanks to the new bank-interleaved organisation. The access latency took a hit, though.
It's the L2 overall performance that's the troubling factor here -- at least they got the size right, as a compensation. The L3 is doing its job, and this time AMD managed to improve its SRAM density by a ~20% over the L2.
Click to expand...
Isn't exclusive design require lower latency in order to sync more quickly?
 
I just traded my 5Ghz Phenom 2 x6 for Sandy Bridge, If I would of checked this a few days ago I could of tested the latency all varying clocks.

Guys when testing the latency do one at default and then run it again with an overclock on the HT and Northbridge.
 
More results, this time from stock FX8120:
fx-8120_cache2cache_latency.jpg
 
You must log in or register to reply here.
Back
Top