cristic said:
Afaik 32bit windows, will normally only let you use 3GB of RAM for user processes, the rest is reserved for the OS.
It's a bit more complicated than that.
1. Memory is accessed in 4KB pages, giving you a largest total memory size that could be managed of 16TB, if your processor supports that. For example, an Opteron supports 48-bit virtual memory access.
2. The amount of memory that can be adressed within a single process is specified in segments. The maximum size of a segment is 4GB (32-bit). For ease of use, each process nowadays uses a set of code and data segments that map to the same location and are initialized to a size of 4GB. Although the physical starting point of those segments is often different for each process, they are all mapped at the start of their own 4GB segment(s).
3. For ease of use, Windows restricts the maximum amount of memory that can be managed. First to 2GB, so everything can be done with 32-bit integers (and all negative values are used for errors), with 2000/XP to 4GB. That way, the operating system can access all of it directly.
4. The OS reserves an amount of that adress space for it's own use. With Windows NT, this was half of it, with 2000/XP it is 1GB. So the largest amount of memory that can be used is still 4GB, but you can only use 3GB and the bit that is actually used by the OS. (Lunix/Unix do this differently).
5. If you use 32-bit integers to do adress calculations, you can only use half of the maximum amount. That leaves you with 2GB at max. And you only have to use a single integer by mistake on an obscure place to create really hard to track errors in all the memory above 2GB.
6. Due to all those limits, single structures (like files) should never exceed 2GB, even if theoretically possible.
7. Of course, if you use a 64-bit processor, you expand all that by orders of magnitude. But if you try to run a 32-bits program, it probably breaks down when you try to access a 64-bit value. That is the main problem with switching to 64 bits.
On the other hand, if you do a malloc(8589934592) on Linux64, it should just return you a pointer, even if you don't have more than 8GB of memory in your computer.