"What's with the 3Gb memory barrier"?

http://www.dansdata.com/askdan00015.htm

What BRiT said.
Everything in your computer must have a physical memory address for the CPU to "talk" to it. Your CPU can address 2^X bytes of memory where X=OS bits. You must subtract from that total all the memory addresses and what you've left over is available to the OS for use.

 

Short answer: Because when you're under 3Gigs of memory, the address range of memory and the address range of devices total up to less than 4 gigs total.

I hope the follow helps. I've been down this path as I build my 8Gig system.

The largest address that can be designated by 32-bits is 4Gig. The device address range consists of register space and i/o space mapped to the physical devices video cards, sound cards, network cards, and other hardware devices. Think of it as an equation for 32Bit WinOS:

4 Gig MAX Address Range >= (Address Range Memory) + (Address Range Hardware)

The Address Range of Hardware is fixed for the same combination of hardware. When you add a second video card for SLI/Crossfire, it needs another set of address ranges to use for register space and i/o space in order to use the device.

The upper bound limit of 32bit is fixed at 4Gig.

The only variable that can change in this equation is the Address Range of the Memory. This can safely be artificially lowered without causing any hardware devices from properly functioning.

MS could have enabled the Physical Address Extensions on processors that support it to be able to use more memory on a 32-bit CPU/OS. In order to do so, that would require each and every single device driver ever possibly used on any system to be rewritten to remove any assumptions made on their accessing of memory. To steal from my previous post in another topic::

When the system has less than 4Gigs total, the I/O and Registers are mapped below the 4Gig boundary. When the system has 4Gigs or more, most motherboards will remap the I/O and Registers for hardware to be above the 4Gig memory boundary. This is to provide the most amount of memory available for older OSes that are 32bit. In the 32Bit version of XP, MS does some tricks with PAE so that all device drivers see their memory region to be below this boundary. This eliminates the need for all device drivers to be updated.

This info can be dug up when reading up on Page Address Extensions under WinXP. Here's where I ran across this info before.

 

Let me/us know how you get on with 8GiB in your main day-to-day system, since I don't know many others with that amount and occasionally it trips me up in subtle ways.

 

Just to make sure, I'm about to upgrade from two gigabytes of ram to four gigabytes of ram (it's so cheap right now ), since I run Vista x64, I should be good right?

 

As long as all software and drivers you need can handle it. Which might take some time.

 

If you're already running 64Bit Vista then you should be fine from a hardware and software standpoint. I'm not seeing any memory holes and no issues have occured when moving from 2Gig to 4Gig.

The only piece of hardware I have that was not liking Vista 64bit was a TV Tuner card from 1997 that has not been supported even for WinXP. There are some opensource drivers for the BT878 but they are not signed. In order to get it to function in 64bit Vista, I had to run in "test" mode. I'm in process of shopping around for a decent alternative. Also, the tv tuner app K!TV kicks the desktop down to Aero Basic. That wont be an issue once this item gets replaced with a supported tv tuner.

Vista 64bit is having no issues with the following hardware and software mixes for me.

Hardware: Xeon 3220 (Core 2 Quad 6600), AMD X2 4400+, 2Gigs DDR, 4 Gigs DDR2 (will install the other 4Gigs this weekend), ATI X1900 XT, Nvidia 8800 GTX, Crative X-Fi, HD HomeRun (HD TV Tuner), HP MultiFunction Printer/Scanner/Fax, iPod, and Logitech MX Cordless Duo.

Software: 7Zip, mIRC, FireFox 2.x, VLC, foobar2000, dVArchive, Java 6, FoxIt PDF Reader, Diskeeper, nod32, Steam, Lost Planet, 3dMark 06.

I have yet to try out Anapod, Trillian, QuickTime Alternative, Real Alternative, Joost, FileZilla, Alt.Binz, uTorrent, QuickPar, Undelete, and a few others.

There was a slight issue with the newest Creative XFi drivers with 4Gigs memory, so I had to revert to the earlier version. That issue should be fixed in the next revision.

 

The max addressable memory is 4 Gig for 32bit OS's. That never changes, regardless of how much memory you have installed. Let me rework the equation to show you exactly how much memory addressable you will have in the situation of GTX SLI which uses 1.5Gig. It is very simple math.

Max Memory Addressable >= (Memory Installed and Useable) + (Device Memory)

Plugging in some numbers and solving for X, the max addressable memory is 2.5Gig.

4 Gig >= X Gig + 1.5 Gig
4 Gig - 1.5 Gig >= X Gig
2.5 Gig >= X Gig

If one only has 2 Gig installed, that last statement is still valid. In this case, instead of installing 3Gig, the user is better off installing only 2.5 Gig.

If however you're only running 1 GTX card, the most memory you can use is 3.25 Gig.

4 Gig >= X Gig + 0.75 Gig
4 Gig - 0.75 Gig >= X Gig
3.25 Gig >= X Gig

Did that help any?

 

assuming you don't have any other device than the GTX?
I'm pretty sure the last GB of addressing space is reserved anyway, hence the 3GB limit.

 

No. That is not the case. There is no 3 Gb limit. There is never a set amount of space reserved across the board. It depends on what's installed and what those devices need as well as how BIOS handles possible memory remapping for those devices.

On my work Laptop with 4 Gigs memory and no bios remap function inside Windows XP 32bit shows 3.25 Gig available. That extra 256Meg makes a world of difference -- between my system thrashing to death or running smooth when I'm doing heavy development work. Some might say the system would be fine with a 2Gig + 1 Gig DDR2 sticks, but I wouldn't. Unfortunately for me, the Information Protection Services and Support Services have not blessed any 64bit OS's.

Yes. I used only the GTX SLI requirements since that is what Silent_One had directly asked about.

 

That helped, thanks BRiT....
I would be grateful if someone explained how it looks from memory usage point:

Lets say we have 2 configurations, both with GTX SLI (1.5 Gig Device Memory mapped). First configuration has 2Gib physical RAM installed the other has 4Gib RAM.

If we run 2 processes and each procces allocates 2Gib Ram for its use - what would be swap file footprint in each configuration?

 

Th Phyisical memory limit is not per process, it's for the entire operating system. If the OS is only addressing 2.5 GB of your memory, each process will only be able to address that same 2.5 GB. It's a safety net for badly written drivers.

Answering your question, the swap file usage of the 4 GB machine will be only be 500MB less than the 2GB machine because only 500 MB of the extra 2 GB can be used.

Using a Windows 2003 Server version then all your memory could be used. But then you might run into driver compatibility problems. Not all drivers like PAE mode, hense the limit in the first place in Windows XP.

 

Yes i wrongly assumed the limit applies to virtual address of process. Since usually win32 processs can anly address lower half of 4G address space (higher half is owned by OS) it just didnt make sense.

Thanks for clarifing things Colouresss

 

If you're running XP and didn't change your boot parameters, it's safe to assume that all your apps together can at most access 2 GB. You can tweak that up to 3 GB, and there are apps (like Photoshop and SQL server) that can access even more if you enable PAE, but most everything else can't. And you need to patch or recompile most apps to be able to use more than 2 GB.

 

Hi Frank, are you sure that's the case? I'm not intimate with Windows memory menagement but my understanding was total memory avaliable to all applications would be physical limit we are talking about (2.xGiB - 3.xGiB depending on range of addresess mapped to devices). Memory manager should be able to divide left free memory to all processes during paging translation?. 3GB switch rather allows application to use part of virtual address range normally reserved for kernel....
Oh well i'm lost again....

 

With the original win32 extender (Windows 3.x), 4 GB was seen as a HUGE amount of memory, going from less than one megabyte. 4 MB was very much at that time. Remember Bill Gates stating that "640 kilobytes of memory is enough for everyone"? And, as they neededed kernel and I/O address ranges, at a time where 32 bit integers (2 GB max) were seen as the end-all fix-all data type, and the 64 kilobyte segment limit was still fully in charge, it didn't seem to pose any upper limit when they simply split that 4 GB in half and allocated a part to user and kernel space. "NOBODY will EVER have SO much memory!", was the idea at that time. They were only defining it to see what they had to do to support that huge virtual memory range.

Well, things changed. But the memory model didn't. So, now we're back to the extended and expanded memory days, only now they're called PAE or PSE. ANd instead of 640 kB of program-useable memory and 384 kB of system meomory, we have 2 GB of both.

But, that 2 GB of system memory isn't needed in most cases. It can be decreased to 1 GB, with the /3GB boot parameter. But, most programs and drivers are written and compiled while assuming they can never access more than 2 GB in any case. Which means, that they can use 32-bit integers for everything. Which is what most developers did and do.

Unfortunately, going beyond 2 GB requires not only a different way to compile your program, but also the use of unsigned 32 bit integers instead of regular ones (quite a difference) to address memory locations.

And, if you want to use PAE or PSE, you're back to the very start: instead of small, compact, medium, large and huge memory models, you now have integer, long, paging, extended and lineair models again. Which will be very recognizable to all the developers who ever used the first win32 model.


In short: we hit the limits again, and we came up with the same work arounds. But there is no silver bullet, and all the bits and pieces will use their own subset. This will go on until everyone and everything has migrated to a flat, 64 bit adress space.

And in about 10 years from now, the same thing will happen yet again.

 

If we start seeing 1Gb Video Cards soon, and you have 2 of those running SLi in a Vista 32bit system, does that mean you'll never be able to get 2Gb of RAM available?

 

If you want them to be part of the virtual memory pool, then yes.