Cloud Performance: How to Interpret Benchmarks (Part 1)

Posted by Kenny in Cloud Computing, IaaS, Performance Testing

Part 1 provides an introduction to the post and a look into relevant open source performance tests for Video and Database applications. Part 2 will look at other industries and applications; i.e., Storage, Gaming, and general eCommerce.

Abstract

Performance results from open-source benchmark tests can only reveal certain details about the server(s) the tests are performed on. To really take advantage of the results, you have to understand which tests are relevant to the application(s) you want to run. This post provides examples of 5 different application-types, and suggests a collection of tests that would be beneficial for user running that application on the cloud.

Introduction

What is the best car?

Give up? Well, there is no right answer; after all, what is the best car… for what purpose? Are you looking for speed? Heavy lifting? All-terrain? Good gas mileage? So many factors come into play, and your “best” car is best because it fulfills all of your specific car requirements.

The same idea applies in cloud. There is no “best” cloud provider (at least not from what we’ve seen). There certainly are some that are better than others in virtual core performance, or network speed, or maybe even the service-side. But there are tests out there that can accurately assess how a cloud server will perform well for your needs. By looking at all the necessary tests relevant to your application, you can have an accurate measurement of how a cloud server will handle in performance.

Below are 5 different application-types and examples of what would be required. We understand this isn’t a one-size-fits-all deal, but it is a fair starting point for anyone interested in looking into cloud performance for his or her applications. For more details and performance results, please view our CloudSpecs Performance Test System.

---

1. Video

Think services like YouTube or NetFlix. These services are just a couple of example in the plethora of SaaS video applications in the Internet. A fast disk I/O, low-latency network access, as well as good video and audio conversion performance is important in selecting the right cloud IaaS provider. Here are some tests to look at:

---

2. Database

The database lies at the center of Internet applications, continuously transmitting and receiving data to and from other servers. Without a fast and functional database, performance can be dragged down, regardless of how great an application would be otherwise. Performance in Disk speed, CPU, RAM, and even internal network all factor into database performance; the truth is, databases can be so custom that the suggestions in this post is not a catch-all, but rather a basic rudimentary look into some benchmark measurements that can pinpoint key metrics in predicting database performance.