IaaS Cloud Performance Comparison: ProfitBricks VS. Amazon and Rackspace (UnixBench and Iperf)

FULL DISCLOSURE: ProfitBricks hired Cloud Spectator to run a continuous UnixBench and Iperf benchmark for 15 days (4 times a day) from our CloudSpecs Performance Tests on their beta cloud IaaS servers, and compare the results to Amazon EC2 and Rackspace Cloud. We did not alter our methodology in any manner, and data presented in this post are meant only to report as information. Cloud Spectator does not endorse ProfitBricks or ProfitBricks services.

Abstract


New cloud IaaS providers, such as ProfitBricks, CloudSigma, and Cloud Provider USA are emerging rapidly in the expanding cloud IaaS market as a response to increased performance demands. This post describes the performance results of ProfitBricks against Amazon and Rackspace.

Introduction


New competitors are emerging in the cloud IaaS industry, bringing newer technologies and more powerful, reliable clouds. ProfitBricks, a German-based cloud IaaS provider with an emerging US arm, is an example. Leveraging its partnership with Mellanox, ProfitBricks takes advantage of InfiniBand technology to optimize its network speeds within the data center. It also gives its customers private cores, a fairly new concept in the IaaS market; in traditional IaaS providers such as Amazon EC2, virtualized cores are shared.

But what does this all mean? ProfitBricks hired us to run our CloudSpecs Performance Tests to be able to benchmark the true performance of their cloud server environment, to either prove or disprove their technological advantages. As mentioned in the disclaimer above, we do not endorse ProfitBricks nor its services, and all the data below is for informational purposes from our findings.

Comparison


UnixBench Results: ProfitBricks Outperforms Amazon EC2 and Rackspace Cloud
ProfitBricks’s UnixBench scores 227% better than Amazon and 247% better than Rackspace on average over a continuous period of 15 days.

The UnixBench suite is a collection of performance tests that are used to collectively calculate an overall UnixBench score of a Unix-like system, which gives the user an idea of the overall performance of the server.

UnixBench description from Google Code:
Multi-CPU systems are handled. If your system has multiple CPUs, the default behaviour is to run the selected tests twice — once with one copy of each test program running at a time, and once with N copies, where N is the number of CPUs. This is designed to allow you to assess:

  • the performance of your system when running a single task
  • the performance of your system when running multiple tasks
  • the gain from your system’s implementation of parallel processing
[box]

ProfitBricks Average UnixBench Score: 2,315

Amazon EC2 Average UnixBench Score: 1,016

Rackspace Average UnixBench Score: 936
[/box]




Iperf Results: ProfitBricks Outperforms Amazon EC2 and Rackspace Cloud
ProfitBricks outperforms Amazon EC2 (by 694%) and Rackspace Cloud (by 1,165%) on average over the course of 15 consecutive days by leveraging and optimizing the InfiniBand technology.

Iperf is an industry-standard and time-tested performance that is effective for measuring TCP bandwidth. TCP is the most common of two types of Internet Protocol (IP) traffic; HTTP, FTP, and SMTP are examples of TCP. The other, UDP, is mainly used to transfer time-sensitive data such as VOIP and DNS.

The duration of our Iperf is 120 seconds, and results are calculated by transferring as much data as possible from that time frame.
[box]

ProfitBricks Average Iperf Score: 4602 mibits/s

Amazon EC2 Average Iperf Score: 523 mibits/s

Rackspace Average Iperf Score: 394 mibits/s
[/box]




Methodology


The CloudSpecs Performance System, a software suite of open-source, industry-standard server performance tests monitors the performance of ProfitBricks US, Amazon EC2, and Rackspace Cloud 4 times a day over the period of August 15th, 2012 to the end of August 30th, 2012. Cloud Spectator anonymously creates a user for each of the services and installs the CloudSpecs system on to the provisioned cloud servers. From those findings, we average performance in a period of time. Individual points can be found by contacting us.

Server configuration is complicated because both Amazon and Rackspace only provide tiered structures, where you must select a pre-configured package (called “instances” on Amazon and “servers” on Rackspace) that contains a set amount of CPU, RAM, and disk space. ProfitBricks’ IaaS cloud is an independently scalable infrastructure, where resource amounts of CPU, RAM, and disk are all independently selected and adjusted.

    Server Configurations for UnixBench:

  • ProfitBricks: 16GB RAM, 4vCPUs, 50GB Disk
  • Amazon: 15GB RAM, 4vCPUs (8ECUs), 1,690GB Disk
  • Rackspace: 15GB RAM, 6vCPUs, 620GB Disk

Iperf was conducted by creating 2 servers within the same data center and with the same client account. By creating a VPN with those servers, Iperf measures the connection speed between those two servers. The Small server acts as an agent to transmit data to the Large server.

    Server Configurations for Iperf (SMALL):

  • ProfitBricks: 4GB RAM, 1vCPUs, 50GB Disk
  • Amazon: 3.75GB RAM, 1vCPUs (8ECUs), 410GB Disk
  • Rackspace: 4GB RAM, 2vCPUs, 160GB Disk
    Server Configurations for Iperf (LARGE):

  • ProfitBricks: 16GB RAM, 4vCPUs, 50GB Disk
  • Amazon: 15GB RAM, 4vCPUs (8ECUs), 1,690GB Disk
  • Rackspace: 15GB RAM, 6vCPUs, 620GB Disk

For more details into the methodology, please contact us.