The Hardware

    I found an e-bay special, an off lease IBM server.  It's an IBM x335 1U rack mount server.  It's powered by dual 2.4 ghz Xeon processors, 2 gigs of Ram, and dual 36.7 gb SCSI drives in hardware Raid 1.  The server is definitely not top of the line, but it definitely will do the job.  One thing to be careful about with these servers, is that they do not come with monitor, keyboard, or mouse ports.  To access them, you need a special "breakout" cable.  This is because these servers have a built in KVM switch (Which is of no matter to me, I don't plan on really needing a monitor 99% of the time)... 

The Operating System

    For this server, I chose CentOS 5.  I wanted to stick to Linux for this server, and being that I have been using Red Hat Enterprise for the past 2 years.  For those of you who aren't familiar with CentOS, it's a free version of Red Hat Enterprise.  The best part, is that it's 99.99% binary comparable with Red Hat Enterprise (good for me, since IBM's drivers are available for Red Hat Enterprise).  Since I didn't plan on running Apache, I installed a minimal configuration.  Now, with a bootable system, it was time to start on the services. 

The Core 

    Now, it was time to install MySQL and php.  For MySQL, I chose to download the source, and compile it myself.  Since this is pretty straight forward (and well documented), I'll skip the details of installation.  But, I did do some special things in my.cnf.  Here's most of mine:

[mysqld]
skip-locking
skip-innodb
query_cache_limit=8M
query_cache_size=256M
query_cache_type=1
max_connections=500
max_user_connections=10
interactive_timeout=20
wait_timeout=20
connect_timeout=6
thread_cache_size=128
key_buffer=16M
join_buffer=1M
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
max_connect_errors=10
thread_concurrency=4
myisam_sort_buffer_size=64M
log-slow-queries

[safe_mysqld]
open_files_limit=8192

[mysqldump]
quick

[isamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M

[myisamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M 

    With a working MySQL installation, I installed PHP.  The only real things I did with PHP that could be considered custom, were configuring it with '--enable-fastcgi' '--enable-discard-path' '--enable-force-redirect' '--with-apc' '--enable-mbstring'.  APC is not being used as you may think.  I have disabled caching of files in php.ini.  The reason I have it installed, is for use with my Page Cache component.  Remember, we are limited on memory here.

Lighttpd - The Web Server

    As you probably have gathered by now, I'm a BIG fan of lighttpd.  I found a new tool to use to make the fastest web server even faster.  Quite simply, it's FAM.  What FAM does, is it caches stat() calls, and watches directories for changes.  This cuts down on disk access (especially when you take into count that static files are cached in free memory on Linux), and greatly increases speed.  Another few changes were the server.event-handler and server.network-backend were changed to the optimized handlers for the 2.6 kernel.  These changes alone increased the performance, while reducing the load dramatically.  

Monitoring It All

     If you have ever run a high load website, you know that one of the hardest things to do is monitor the server.  Well, in my searches for a good solution, and after trying a few monitors, I found SICM.  SICM stands for Simple Infrastructure Capacity Monitor, and what can I say about it, other than it finished my search.  SICM is a RRD (round robin database) based SNMP monitor (similar to MRTG).  There really isn't much difference between them in terms of functionality, but I like that you can add custom (non SNMP) Perl modules to record whatever data you want (I wrote a module to pull the number of concurrent connections to Lighttpd).  You can record data at intervals independent to one another (for instance, you can pull the number of connections every 60 seconds, but access log size (if you wanted, for example) every 10 minutes.  The other benefit to SICM, is that it natively supports multiple devices, with different measurements on each device.  So if you run 6 different servers, one installation of SICM can monitor all 6 of them, and give you a nice display.

 The Colocation

    I ran into the best deal I have found in collocation on the internet (and still close enough to drive to).  Located a meare 30 minutes away, I found a plan for $50 a month that includes 1,500 GB of transfer per month over a 100mbit link.  Including multi-homed providers, battery backup, 24/7 access, free reboots and a 100% uptime guarantee (for the network), it was the best deal in my area.  With a 100mbit pipe, I should be able to top out the server before running out of bandwidth (heck, 1000 requests per second on static files is only about 20mbit). 

Conclusions

    My current VPS is costing me $100 per month (worth every penny, but I've out grown it).  So at $50 per month, the savings alone pay for the server in a little over a year.  That's the main reason I decided to buy my own server.  I could have purchased a high end dedicated server, but one with similar specs to the one I purchased would have cost AT LEAST $200 per month (without SCSI drives OR Raid).  Now, when you add in the bandwidth I am receiving, that cost would be closer to $300 per month.  So I've built a high performance server, that will pay for itself in a year or so, and is more reliable than other dedicated servers (that I could find) at more than 6 times the monthly price.  I couldn't be happier.  

TrackBack URI for this entry

Comments (2)

In Newark
written by ircmaxell, July 08, 2007

It's in Newark NJ... The company (cihost) has multiple datacenters in the US

...
written by rdt, July 08, 2007

You said "Located ... 30 minutes away". Where might that be?

Name

Email

Website

Title

Comment

Write the displayed characters

Add Comment