Making Exchange 2007 perform on ESX.

For a couple years we’ve maintained a full lab environment of our production hosted Microsoft Exchange 2007 CCR cluster. I have to be honest, and the Exchange administrator would agree, its never really performed that well. Finally the Exchange admin pretty much got ticked at the performance the other day and while he was out on vacation I thought I’d see what I could do. Its now performing about 100x the speed it ever did, and when he comes back Monday I hope he’s happy. More than anything we’ve just not had the time to really dig into the issue(s).

Needless to say over these couple years we’ve all learned quite a bit about how to eek more performance out of ESX, and in particular Exchange on ESX. I thought I’d share a bunch of the concepts and tidbits in one spot I used to arrive at better performance. The old tricks of throwing RAM and vCPU’s at the problem just didn’t cut it.

Its worth noting that people these days might not choose to use CCR on a virtualized environment (or never did), however I feel these concepts bleed over into stand-alone or maybe even FT/vLockStep implementations going forward.

First and foremost, I invite you to read this article on Exchange 2007’s memory managment strategy:

http://msexchangeteam.com/archive/2008/08/06/449484.aspx

Okay, now that you’ve read that lets continue. Suffice it to say Exchange literally grabs every piece of memory and page it can…. if you let it (which most people do for cache/performance reasons).

As most of you reading know, ESX has quite a few tricks up its sleeve in the memory management department itself and I invite you to read about those concepts in the vSphere/ESX manuals. Obviously the VM’s have access to physical RAM, shares RAM pages when possible, begs/borrows/steals from other VM’s (balloon) and when necessary swaps to disk as a last resort. After careful examination of the performance logs of the Exchange VM’s, it became very obvious ESX was swapping.

Now, we happen to have an entire lab cluster and nobody really cares about performance … well nobody except the persnickity Exchange admin anyway. After doing some research I came to the conclusion I didn’t want the Exchange servers to swap memory … period. I then set a reservation on the VM to the exact same size as the RAM I’d granted. In this case 3GB. This effectivly disables the vswap since the host has no choice but to ante up.  Doing  just this provided an incredible performance boost to the Exchange cluster, but I scratched a little deeper.

Within the VM itself I observed that it was paging quite a bit, and as the above referenced article shows… it always will. So to extract maximum paging performance I decided to create a couple LUN’s and mapped them raw (RDM) to the Exchange servers. I then did some research and came to the conclusion 4kb was the optimal block size for a raw paging volume and if anyone has differing opinions on that PLEASE post them. I then created page files equal to granted memory +20MB. After doing all of that and rebooting, I could tell we’re really cooking with gas now.

Going forward I would like to talk to the Exchange adminstrator about migrating his VMDK based message stores to RDM as well.

So, in closing … hopefully some of these ideas will help you come up with your own Exchange performance issue resolution. I’m sure there are more tricks I need to find, but right now I’m pretty happy with the results.

Till next time…