While I usually prefer to focus on technical solutions that can help readers solve specific business problems, after reading Microsoft’s latest press announcement about SQL Server 2008 #1 ranking in the TPC-E benchmark I felt compelled to discuss this somewhat misleading announcement. While SQL Server 2008 certainly did top the TPC-E benchmark, the fact is that no other vendors participate in the TPC-E benchmark. As one of my colleagues put it, they are number one in a field of one. Woo Hoo! Seriously, this is no accomplishment to be trumpeting to the world.
Don’t get me wrong. I’m one of SQL Server 2008’s best fans. And no one was more excited than me when SQL Server used to top the TPC-C benchmarks. However, that’s not what this is about. The TPC-C is where Microsoft and SQL Server used to compete. IBM and Oracle are still there. The TPC-E is a totally different benchmark. A couple of years back Microsoft quit issuing TPC-C benchmarks in favor of the new TPC-E benchmark. However, IBM and Oracle did not follow suit. Nor did anyone else.
Without multiple vendor participation TPC-E numbers are meaningless. They can’t be used to compare SQL Server 2008 against other database platforms (the true purpose behind the TPC in the first place) nor can they even be used to compare SQL Server 2008 against previous version of SQL Server. If no one besides Microsoft uses TPC-E then there can be no cross database vendor comparisons. It can’t even be used for comparing SQL Server 2008 and the earlier SQL Server 2005 release because they are run on different hardware platforms.
Industry standard benchmarks are about comparing the performance of multiple vendor solutions. At the best, announcing #1 TPC-E is silly and meaningless – unless Oracle and IBM begin producing TPC-E benchmarks as well. However, all of their latest OLTP tests have been in the TPC-C. At the worst, Microsoft TPC-E announcements are misleading as they imply that SQL Server is better than the other competing enterprise database platforms (isn’t that what it is to be #1?). But, in fact, that’s not demonstrated by the TPC-E. It makes you wonder if the problem is that SQL Server 2008 can’t post competitive scores in the TPC-C.
End of Article
Could it be because the benchmarks submitted are by the hardware vendors and they didn't want to pay (money and time) for completing both the TPC-C and TPC-E benchmarks?
Microsoft Learning Snack - Green IT Through Virtualization Many organizations face rising operating costs caused by excessive energy consumption. Virtualization and "Green IT" can help cut these costs. Get the information you need to bring Green IT savings to your business.
Order Your Fundamentals CD Today! Register today for your in-depth copy of one of three Fundamental CDs on the following topics – Exchange, SQL, and SharePoint.
Microsoft Learning Snack - Virtualization Basics With virtualization, computing components essentially become on-demand services, freeing each element of a system from the others. This short video explains the needs, benefits, and technologies behind virtualization.
Microsoft Learning Snack - Virtualization Basics With virtualization, computing components essentially become on-demand services, freeing each element of a system from the others. This short video explains the needs, benefits, and technologies behind virtualization.
Empower Your Processes with PowerShell 201 Paul Robichaux delves deep into PowerShell how-tos in 3 informative lessons, each followed by live Q&A—all on your own computer! Register today!
Microsoft Learning Snack - Green IT Through Virtualization Many organizations face rising operating costs caused by excessive energy consumption. Virtualization and "Green IT" can help cut these costs. Get the information you need to bring Green IT savings to your business.
New Release: Windows IT Pro Master CD 13 years of content archives, fast answers with advanced search tools, and full access to WindowsITPro.com—order today!
Register
JPlietz December 11, 2008 (Article Rating: