Windows Azure outstands amongst 5 large IaaS providers in an independent comparative analysis by Cloud Spectator

Recently, I found an analysis paper about cloud server performance conducted by an independent cloud performance metrics company, Cloud Spectator.

This post is to summarize the paper and I definitely encourage you to read the full report over here:….pdf

Objective of analysis study

The objective of the paper is determine the price-performance value of the cloud providers. Providing some valuable insight for customer when selecting their prefer cloud vendor.


Figure 1 – Principle of value proposition [figure from the paper]

Who are being compared

The study (done in June 2013) compared five large IaaS providers in the industry:



The tests were run for 3 times in 5 consecutive days: May 25, 2013 – May 29, 2013.

VM Size

The most common size for cloud server, Medium Size (or equivalent / similar setup) was chosen from the 5 cloud vendors:


Figure 2 – Medium VM Spec [figure from the paper]


The tests used Unixbench 5.1.3 as benchmarking the performance of Linux OPS running on virtualized infrastructure, producing rating out of 10 stars. Details of Unixbench can be found here:


Two important pieces of info are collected:

  • Performance: how well the provider scores on Unixbench, and how consistent the scores are.
  • Price-Performance: after performance scores are established, we factor in cost to understand how much performance a user can expect on return for every amount of money spent, i.e., the value.

The Results

Performance Only

The performance result shows that Windows Azure provides the best performance and notably 3 times higher than AWS EC2 on average!


Figure 3 – Performance Only Result [figure from the paper]


Figure 4 – Average Unixbench Score, derived from Figure 3 [figure from the paper]


Retail hourly price of the cloud providers are captured (pay-as-you-go) basis on date of experiment.


Figure 5 – Pay-per-hour price [figure from the paper]

By taking each score and dividing by the price, we can get a relative price-to-performance score for each provider. And here are the score (The higher the score, the better):


Figure 6 – Price-Performance Result [figure from the paper]

CloudSpecs Score

CloudSpecs score is a further normalized value from Figure 6, taking the highest value to 100. And here’re the scores:


With the cloudspecs score, the ratio of each of the providers are formed as following



While acknowledging that Unixbench is just one test, customers may always consider other factors when selecting their cloud vendor.

To conclude, Amazon EC2 and Windows Azure offers the lowest price at $0.12 per hour. However, Windows Azure performs much better than EC2 in this experiment (approximately 3 times). The experiment also shows that Rackspace scores worst in term of price-performance.

This entry was posted in Azure, Cloud. Bookmark the permalink.

2 Responses to Windows Azure outstands amongst 5 large IaaS providers in an independent comparative analysis by Cloud Spectator

  1. Pingback: Windows Azure Community News Roundup (Edition #72) - Windows Azure - Site Home - MSDN Blogs

  2. Pingback: Windows Azure 社区新闻综述(#72 版) - 微软云计算: Windows Azure 中文博客 - Site Home - MSDN Blogs

Leave a Reply

Your email address will not be published. Required fields are marked *