How I came Accross the Problem
Recently, I was building a windows service that aggregates a large amount of data from multiple data sources and then performs queries against that data using PLINQ. This process right or wrong required a lot of memory and unfortunately I began receiving out of memory errors.
While debugging, I noticed the error would occur when my process would take up about 1.6 GB of memory. To me this looked pretty close the limit for 32bit processes. My first thought was to check the Windows service project's build properties to see if the platform target was set to x86. It was set as I wanted, to Any CPU. However, there was an option I hadn't noticed before that caught my eye. It was an option named "Prefer 32-bit" and it was checked. After doing some research, it turns out that if this option is set, the process will run in x86 mode even when running on a x64 capable machine as long as that machine also supports x86. It appears this option is set by default for at least Console and Windows Service applications.
Unchecking the checkbox next to "Prefer-32 bit" allowed my Windows service to run without error. Please note that this option is disabled in web applications, or if the platform target is x86 or x64.
Platform Target ~Available Memory Cheat Table
All values are approximate as the exact numbers depend on many factors. Also, the 2800 MB limit is a .Net (Pre 4.5) imposed limit.
* There is a configuration option in .NET 4.5 and above that allows you to increase the available memory of a .Net x64 process passed the 2800 MB limit. See below.
Increase the Available Memory Limit for .Net 4.5+ Process
In the App.Config file, set the "gcAllowVeryLargeObjects" element's "enabled" attribute to "true" underneath the runtime element. There is a cost to doing this of course. The main thing to be aware of is the more objects that are used and deallocated the bigger the stress on the GC. So make sure you monitor it.