SAN FRANCISCO – Every once in a while, a major comparative review comes together at exactly the right time, where tests of actual shipping products reveal the impact of just-released technology in no uncertain terms. This review is one of those times. Blade servers from Dell Inc., Hewlett Packard Co. and IBM Corp.– all three sporting the latest Xeon 5600 (aka Westmere) CPU — arrived at our test facility at the University of Hawaii before Intel had even officially introduced the chip. As it turns out, our benchmarks reveal that Westmere carries blades to new heights.
We also managed to add a budget-friendly blade chassis from Super Micro Computer Inc. into the mix. Where a Dell, HP, or IBM chassis with four blades starts upward of US$40,000, the Supermicro solution costs a mere fraction of that. It is not in the same class as the big three blade systems, but it’s an interesting option for shops that may not need the latest features or maximum performance. Since it was available in the lab, we put it through the same paces.
As with our previous blades shoot-out, all the testing was conducted at the University of Hawaii ANCL (Advanced Network Computing Lab) facilities on Oahu, facilitated by lab director, networking guru, and fellow InfoWorld contributor Brian Chee.
Blade server triathlon
We opted for a suite of custom VMware tests and an array of real-world performance metrics. The VMware tests were composed of a single large-scale custom LAMP application — a load-balancer running Nginx, four Apache Web servers, and two MySQL servers — laid out with sufficient vCPU and RAM resources to oversubscribe the physical and logical CPUs in each blade.
The workload was designed to mimic a real-world Web app usage model, with a weighted mix of static and dynamic content and randomized database updates, inserts, and deletes. This load was generated at specific concurrency levels, starting at 50 concurrent connections and ramping up to 200. The sweet spot on all the blades was the 150 concurrent connections mark, so we used those numbers for the published results. These VMware tests were run first on one, then across two blades. The blades were running VMware ESX 4 and controlled by a dedicated vCenter instance.
The other real-world tests were an array of common single-threaded tasks run simultaneously at levels that met and eclipsed the logical CPU count on each blade, running all the way up to an 8x oversubscription of physical cores. These tests included LAME MP3 conversions of 155MB WAV files, MP4-to-FLV video conversions of 155MB video files, gzip and bzip2 compression tests, and MD5 sum tests — the same benchmarks we used last week to compare the new Intel Westmere-EP and prior generation Nehalem-EP CPUs. They represent a good cross-section of actual workloads and proved to be consistent markers of overall blade performance.
In addition to those tests, we rounded out the benchmarks by using Ixia’s IxChariot tools to gauge network throughput between blades in the same chassis. Those tests were based on a few common loads, including iSCSI 32K and 256K read tests, a packet blaster load, and a BitTorrent P2P load that gauged connection rates. We also used Ixia’s IxLoad testing tool to generate load on the LAMP application for preliminary testing, prior to using the benchmark’s built-in client.
Westmere vs. Westmere
Although Dell, HP, and IBM were free to choose whichever CPU they liked for the test, all arrived with Intel Westmere-EP X5670 2.93GHz CPUs and at least 24GB of RAM in each blade. The reason for the Westmeres is clear: Six cores beats four, and with the Westmeres also edging Nehalem in per-core performance, they’re the fastest thing going in the x64 world right now.
Dell also brought along one set of X5680 3.33GHz CPUs, which allowed us to see what the X5680 CPUs could do. For the sake of apples-to-apples comparisons, however, we evaluated the Dell blade’s performance based on the X5670 chips. Our low-budget Supermicro blades were outfitted with slower (and far cheaper) AMD Opteron 2378s, quad-core Shanghai CPUs running at 2.4GHz.
We also allowed each vendor to choose which type of storage to bring. Dell showed up with a Dell EqualLogic PS6010XV 10G iSCSI SAN array, HP brought an EVA 2124 Fibre Channel array, and IBM also went the Fibre Channel route with a System Storage DS5020. Due to time constraints and the relative disparity between the storage mediums, SAN array throughput tests were not conducted.
The blade performance test results were quite interesting in that they were basically identical. Running the Westmere X5670s, the Dell, HP, and IBM blade solutions performed within the margin of error across all tests.
The threading concurrency tests showed that by and large, the Dell, HP, and IBM blades ran neck and neck, with IBM taking the slightest of leads near the upper end of the accuracy margin. The VMware LAMP application tests also resulted in a statistical draw, with IBM topping out at 2,125, HP at 2,110, and Dell at 2,104 requests per second, giving IBM a “lead” of around 0.7 percent. Suffice it to say, there’s no significant performance difference among the three Westmere-equipped blade systems in the test.
Our low-budget Supermicro entry pulled in last, as expected. With four cores per CPU instead of six, a lower clock rate, only 8GB of RAM, and a chip that’s several generations behind, the Supermicro performed quite well for its cost, but was nowhere near comparable to the Intel Westmere blades — no surprises there.
Since Dell also brought a set of 3.33GHz Intel Westmere X5680 CPUs, we swapped those processors into one of the Dell blades and reran the threading concurrency tests. The results were around 12 per cent faster than the X5670-based tests, reflecting the 400MHz increase in clock speed per core.
Interestingly, the Dell, HP, and IBM blades all performed equally well in the Ixia IxChariot throughput tests too. This shows that at this point in time, the major vendors have stabilized on Intel Westmere, and the internal 10G switching fabrics have matured. The Supermicro entry did not have 10G, but performed up to spec with the 1G interfaces present on the test blades.
In conclusion
Measured on an apples-to-apples basis, the Dell, HP, and IBM blade systems will all hit their performance marks. There is no significant difference in blade performance in similarly equipped blades from any of these vendors. Thus, without meaningful differences in performance to go on, our evaluations of this batch of blades come down to features, options, price, and management tools.
Since our last look, Dell has certainly been busy streamlining its blades management interface, and the company has introduced some extremely compelling features such as the global BIOS and firmware update tools. HP’s management is also well rounded, but is inexplicably missing a few features that the other vendors offer, such as those BIOS and firmware updates and the ability to map local shares for virtual drive and ISO image mounting. IBM is no slouch, with a spartan yet navigable management GUI. Of course, all three vendors also offer external global management tools to take blade and server management to the next level, but these tools were beyond the scope of our review.
Given a middle-of-the-road use case, you’ll do well with any of these solutions. If you want a solid blade server platform on the lower end of the pricing scale and don’t need extras like storage blades, then the Dell M1000e is a good fit. If you need internal storage, such as for a branch office deployment, or any of a wide array of blade types, then the HP c7000 is right up your alley. If you don’t mind losing two blade slots per chassis but need some extra redundancy, then the IBM BladeCenter H might be just the ticket. If you’re worried about the bottom dollar and don’t mind adding elbow grease and sacrificing some performance and options, then the Supermicro SuperBlade is an extremely affordable way to add blades to your data center.
Whatever the choice, it’s obvious that the current crop of blade servers is more than ready to ride the virtualization wave and meet just about any other challenge. By choosing one of these blade systems over ordinary 1U servers, you can expect not only stellar performance, but also greater availability, comprehensive management, and lower power and cooling costs. The advanced power and cooling management in some of these solutions can enhance the overall cost savings as well.
