Blog Tags: 

Reducing the load on a small VPS by 80% in 5 minutes

A few days ago I noticed the average load on a newly setup VPS was too high relative to its workload. The server was noticeably sluggish. Load was high and a page the wiki it was running would occasionally take 10 seconds to load.

I didn't have too much time to kill but I decided I would at least try picking off the lowest hanging fruit before upgrading to a beefier, more expensive plan which would cost a few hundred dollars extra a year.

When I noticed swap usage was relatively high, I suspected the server was constantly moving memory in and out of swap.

One of the first things you should try when you're tuning the performance of a server is to reduce memory usage so that it doesn't use as much swap and can use more of your memory for disk buffering. Accessing buffered blocks from memory is a gazillion times faster than accessing them from disk.

So how do you reduce memory usage without reducing functionality? By tuning the trade off between memory and CPU time.

One example of this trade off in practice is pre-forking, which is a common technique used by many systems that can improve user-perceived performance. The principle is that by starting the process in advance of when it's needed then that bit of overhead is not experienced by the user who would otherwise need to wait for the process to start up before it can serve him.

The catch is that keeping spare processes around requires memory.

When you have a limited amount of real memory, then any performance gains from pre-forking can quickly evaporate due to reduced IO performance.

On this server, we didn't really have that much activity for things like mail or web, but many subsystems were configured (by default) to pre-fork at least 5 worker processes in advance.

This is a reasonable configuration for a dedicated server with lots of memory and hundreds of users, but not ideal for that cheap VPS you just setup for your workgroup.

I reduced memory consumption by over 200MB by simply decreasing the number of preforked worker processes for apache (20-25MBs per process), spamassassin (30MBs per process), and saslauthd. Average load dropped immediately by about 80%.

Total time spent picking this low hanging fruit: 5 minutes.


MikeC's picture

I substantially reduced the memory usage on my own VPS a few weeks back by moving from the Apache Prefork mpm to the Worker mpm, and by disabling FastCGI so I didn't have idle php processes sat around using up memory.

Liraz Siri's picture

If I recall that's the main reason prefork is so widely used. Worker is known to be more memory efficient but not everything is thread-safe. In particular I remember this created a problem with PHP, so you had to run it outside of Apache in its own little pre-forking application server (e.g., with FastCGI).

Personally I prefer lighttpd over Apache. There's a bit of a learning curve involved but it's not only super-fast and efficient, it also has a very powerful configuration system which is in my opinion superior to Apache. These days the only reason I would use Apache is backwards compatibility on a server where memory resources are not an issue.

I've also played with nginx, and it's nice too. Slightly faster than lighttpd in many cases. But on the other hand, lighttpd is far more flexible and configurable.

MikeC's picture

I'd expect the php issue to only exist if you're using mod_php, but I'm running php as a CGI for security reasons. I did have to remove some mod_perl handlers when I moved away from prefork though.

I'd like to move to lighttpd to save resources, but I'm running quite a few sites, and some of them use very complicated rewrite rules so for the moment, I'm just going to stick with what I know.

Liraz Siri's picture

We had similar issues with porting rewrite rules to lighttpd from Apache and found the solution in lighttpd's mod_magnet, which allows you to script these things in Lua. Sure there's a learning curve involved but you can pretty much get anything done. I like solving these sorts of challenges so for me it was fun.

Also, lighttpd rewrite rules are getting much more powerful so in the future I bet you'll only need dive into Lua for the really complicated stuff.

Miguel's picture

Due to copy-on-write and the way virtual memory works, that does come as a surprise to me. Pre-forking shouldn't significantly increase the consumption of RAM and thrash your server that much, at least not for as little as 5 idle-just-forked processes.

I'd say it was good that it seems to have fixed your problem, but I don't believe pre-forking to be the real issue here.

Liraz Siri's picture

So with copy-on-write semantics pre-forking shouldn't necessarily be an issue but I think that depends a lot on where you fork and what the child process does afterwards. Also, I'm guessing with dynamic languages like Perl in which spamassassin is implemented the performance hit from running multiple instances of the Perl runtime could be significantly greater.
Jason B.'s picture

I used your tip about tuning the trade off between memory and CPU time and it helped speed up our server.  Pre-forking wored wonders.  We have lots of workers that all run automotive crm software and we all noticed the difference after I altered the server.

Liraz Siri's picture

Thanks for the positive feedback. It's delightful to hear that people are still benefiting from the posts on our little tips and tricks.
Muhammad Sohail's picture

Liked your article dude. I tried the trade off technique using prefork, reduced Apache conf to bare minimum and removed all unwanted modules but did not get the expected results out of it. Had to move to worker model. Although both prefork and worker caused the load average to shoot up to 20, the worker model was able to withstand 250 concurrent users against 15 concurrent users from prefork.

It would be extremely helpful if you can share tips on how to reduce the load average which is caused by Apache coz at this stage I would like to stick with Apache instead of switching over to Lighttpd or nginx although both are better than Apache any time.

managed vps's picture

Nice Post!!!!!!!!

Thaks for sharing.....

Virtual private servers's picture

Exceptionally great post!!!!!!

Looking forward for more post like these!!!!!!!!!!

Thanks for sharing!!!!!!

Andrew's picture

I'm using Amazon EC2 so I'd like to give this a try. Could you give me a clue where to start please?

Andrew's picture

For anyone wondering where to look to tweak Apache Prefork conditions, you'll find the settings in /etc/apache2/apache2.conf as follows:

<IfModule mpm_prefork_module>

    StartServers          3
    MinSpareServers       3
    MaxSpareServers      3
    MaxClients          50
    MaxRequestsPerChild   1000

These are my settings for now (using Amazon EC2 Micro Instance). Once you've made your changes, you'll need to restart Apache.

Mr. Louis's picture

Hi Andrew, 

I'm trying to use amazon ec2 for my MT4 usage. I'm a rookie in this kind of thing. I've seen your suggestion to maximise the performance so would like to seek your help to set something to improve the load.

I don't understand all these things you mentioned. Would you please email me the procedures to set it? Thanks.

you wrote:

For anyone wondering where to look to tweak Apache Prefork conditions, you'll find the settings in /etc/apache2/apache2.conf as follows:

<IfModule mpm_prefork_module>

    StartServers          3
    MinSpareServers       3
    MaxSpareServers      3
    MaxClients          50
    MaxRequestsPerChild   1000

These are my settings for now (using Amazon EC2 Micro Instance). Once you've made your changes, you'll need to restart Apache.

Warmest regards,

Mr. Louis

Jeremy Davis's picture

But IMO unless you have really low traffic A micro server just isn't going to cut it! In my experience if you want half decent reliability then a Small is the starting point. Once you start starving that of resources then it might be time to consider tuning... My 2c anyway...

As others have said elsewhere, perhaps it might be time to try one of the other lighter weight servers such as Nginx or LigHTTPd.

baruz's picture

Hi Liraz Siri, could you please write an easy Apache tuning and optimization guide for 24GB ram installed server?

my entertainment site runing on 6 x AMD Opteron(tm) Processor 4174 HE, Speed 2300.000 MHz, Cache 512 KB

and 24GB Ram installed but my server getting heavy load, hope you write an easy server optimization guide. 

Thanks advance

Liraz Siri's picture

Wow, that's one beefy machine. Use lighttpd or nginx. You may be surprised how well that works, once you set it up. :)

baruz's picture

Thank you so much, i'm gonna try and update here how is working.

Bertans's picture

Hi Liraz,

since I'm a REAL LAMP's rookie, can you provide me the optimal settings you tried (as Andrew did)? I've a "resoruce limited LAMP" with 384Mb RAM who runs our company's website... :P

Thank you for your past (and future, I hope) help!

Liraz Siri's picture

Wow, 384MB is very little for a LAMP server. I'd reduce the number of children as much as possible to conserve memory but you're not going to get very far with that kind of firepower.

If resources are constrained I suggest using an asynchronous web server such as Lighty or Nginx. Good luck!

Andrew's picture

I'm using Nginx now and it's lightweight and super-fast. Tons better than Apache

Akhund's picture

I have a VPS with 2 server. I want to konw about server load 

when 150 visitors  online on my site then server load going up to 2.5 it is increased my vps limits and when 200 visitors online the server load at 1.6, does the server depand upon the visitors or any other activities of website 


Please tell me about this Thanks


Ben Brown's picture

Every few days my VPS server with my host provider runs out of memory and I cannot access anything. I have to try to reboot it from my end. If this does not work I have to contact the host team, and then sometimes trying to access the server causes a Firewall to Block me out from my machine. Can I ask a couple of Questions regarding VPS Servers

My VPS Server along with CPanel Licences costs me about €36 per month.

I have 5 websites and 5 Cpanels set up for those sites.

My VPS server has 1GB of memory I am told. But every 3-4 days I have to reboot it to keep it visible.

Would you have any suggestions why this happens. My hosting company keep saying I need to buy more memory, if that is the case everytime I ad a new account to my hosting plan I will need to buy new memory and if you are hosting many websites then your bill to a hosting company will be astronomical?

All Help Greatly Appreciated.

This is the largest website I have on the server. - OH and BTW this will not be accessible until Monday 14th October as they have suspended me over the weekend.

Kevin Kinsey's picture

Thanks for the post.  My FreeBSD VPS (512MB RAM, dual-core) was swapping like crazy (almost all of the 512M on 1st virtual HD and some of the 2G swap on 2nd virtual HD).

Mysqld was running in its default state (w/o a config file), probably due to a recent fiasco with ports after updating from 8.2 > 9.2.  Copied /usr/local/share/mysql/my-small.cnf to /usr/local/etc/my.cnf.  Copied the "mpm_prefork_module" section of /usr/local/etc/apache22/extra/httpd-mpm.conf into the httpd.conf file.  Set "$max_servers = 2;" in /usr/local/etc/amavisd.conf and rebooted.

*NO* swapping!

Now, don't criticize ... sloppy management, yes.  But I just keep this box in memory of my former business ;-)

Your post was helpful in getting me fixed up without spending *hours* trying stuff :-)

Nick's picture

One of the easiest and best option to reduce the load on a cpanel server would be to install cpanel pluggins like "Apachebooster".

A lot of server admins have install "Apachebooster" and reduced the server load by a large extend.

This also enhance the website load time and load the site much faster for the end user.

Fagbemi Fisayo's picture

Thanks for this nice post. 

I have a similar issue at the moment and really need help. My apache server has 66GB RAM but only users 15GB, but the load average is over 90. 

I prefer the server uses 45GB RAM and load drops to 20, because this is killing my response time, users' experience is horrible. 

I will appreciate a quick help from you. 


See my apache settings on the 16 processors, (8 cores) server:

<IfModule prefork.c>
StartServers       32
MinSpareServers    32
MaxSpareServers   64
ServerLimit      400
MaxClients       400
MaxRequestsPerChild  10000


J's picture

Hey, just grabbed a VPS from a decent provider.
I left my VPS running for a couple days just to configure it and what not(not restarting). I found that when I logged in to check my RAM usage, it was at 48%. I'm running the Turnkey Wordpress appliance. Ideally after getting WordPress configured, I also want to run a Supybot with minimal usage, around 20 users.

The VPS stats are as follows:
2 x Intel E5 CPUs 2.3GHz
1024 mb RAM

After getting the VPS configured, would running a Supybot(for IRC) be obtainable?
Any suggestions about WordPress would be appreciated.

Jeremy Davis's picture

As per this article (and many of the comments) you can try tuning Apache to trade off CPU vs RAM. In the TurnKey appliances Apache is using it's (very general) default Debian config. I'm no expert but from what I hear there are lots of things that you can do to tune it to get maximum performance.

Another option is to use an alternative webserver such as LigHTTPd or Nginx. They are both significantly lower resource usage but much less user friendly (IMO) to configure. TurnKey has both a LigHTTPd or Nginx webstack appliances (with PHP & MySQL) but you'll need to install WordPress yourself.


Add new comment