Simon Krenger

In the last weeks, I have been considering to replace my HostEurope vServer (Linux L 4.0) with an Amazon EC2 Micro Instance. I already did some testing in another post and then stumbled upon a post mentioning CPU stealing. So I did some testing myself…

Introduction

The post of Greg Wilson mentions the following symptoms when setting up a webserver on the EC2 Micro Instance:

During the first 5 to 10 seconds of the bot crawling my site, everything was super fast, but then things would come to a near halt.

To test this behaviour, I wanted to apply a constant load over a few seconds to simulate a bot crawling a website. Since I also looked into the possibility of running this blog on an EC2 Micro Instance, I figured it would be best to write a script in PHP. As a webserver, I am using lighttpd with mod_fastcgi, just as I am on my current server.

Testing method

The testing was done with the following PHP script that simply hashes a string 100000 times and records the time it takes to complete. This is then repeated 300 times and all recorded times are echoed to the website:

 <?
for($outerloop=0;$outerloop<300;$outerloop++)
{
        $time_start = microtime(true);

        $randomvalue=microtime(true);
        for($innerloop=0;$innerloop<100000;$innerloop++)
                $randomvalue=md5($randomvalue);

        $time = microtime(true) - $time_start;
        echo "$time<br/>";
} ?>

To get a meaningful result, I ran the script five times on each server and recorded the output in an Excel sheet. I then calculated the average times for each server. These values are then plotted in a graph.

Results

So I ran the tests and put my results in a spreadsheet (Download the spreadsheet). When plotting the graph for both servers, things look very good for quite a while:

So what we see here is that the EC2 instance is actually faster than my current HostEurope server. It takes the EC2 instance around 50ms to calculate 100’000 hashes, while my HostEurope vServer has an average of about 65ms. So for the first 150 calculations, the EC2 instance actually excels at every level. But lets see what happens when we apply this load over a longer time…

Wow, ouch. So after about 180 calculations (around 10 seconds), the EC2 instance is suddenly a lot slower than my HostEurope vServer. It takes up to 2 seconds (!) to calculate 100’000 MD5 hashes. That is actually a 40-fold increase in calculation time! Lets see what top has to say while these tests are running on the EC2 machine:

So from here we learn what “st” means:

st – Steal Time. The amount of CPU ’stolen’ from this virtual machine by the hypervisor for other tasks (such as running another virtual machine) – a fairly recent addition to the top command, introduced with the increased virtualization focus in modern operating systems

Conclusion

So what can we take from that test? An EC2 Micro Instance is great for burst-type loads (which a web server definitely is), but as soon as you need to apply sustained load (say, for a crawler or any type of compression on your server), an EC2 Micro Instance is not what you want. When you apply this kind of load, the EC2 hypervisor will quickly cut you back and slow your machine down. Note that this is only on EC2 Micro Instances and does not apply for all other instance types.

So I have the decision for myself: Even though EC2 has great scalability, a good overall performance and a reasonable price, I’ll stay with HostEurope since I do a fair bit of backup compression and therefore need to be able to run a sustained load over a longer period.