Kiwiblog Server upgrade
The Kiwiblog Server is being upgraded this weekend, which means Kiwiblog will be down between 10 am and 4 pm on Sunday 20 April.
Inspire Net, who are the very kind hosts, currently have me on a Hyperthreaded P4 3.0Ghz, with 4 Gbytes of ram, and a hardware raid 5 disk array.
Normally this would be more than enough for a blog. But due to the sheer volume of posts and comments on Kiwiblog, the new plugins that work well on a small site, struggle due to not being written to handle the sheer mass of data that Kiwiblog has accumulated over the years.
This has meant the blog performs very slowly during high load, due to a combination of sheer lack of horsepower in the current platform, and a memory leak in one of the new plugins that is being investigated at present.
So what are we moving to?
The new server is a Quad Core Xeon, 8Gbytes of Ram, and hardware Raid 10 disk array, and the server is capable of expanding out to 2 * Quad Core processors and 32 Gbytes of ram if the site continues to grow.
Once again many thanks to Inspire Net for their work on our behalf. And due to the upgrade you all have an excuse to get outdoors on Sunday!
April 18th, 2008 at 11:50 am
Nice kit. Should have more than 4 times the horsepower. I bet that makes a difference.
April 18th, 2008 at 11:51 am
I think I just wet my pants with delight.
April 18th, 2008 at 12:12 pm
But what will everyone do during that time? My life will be meaningless.
April 18th, 2008 at 12:12 pm
“the new plugins that work well on a small site” — this is a blight on the current world of IT. In the ‘old’ days of mainframes, the software would have been stress tested against a known benchmark so sufficient hardware was available. In today’s world you simply through it into production and if it can’t handle the work load then tough luck for the users.
April 18th, 2008 at 12:29 pm
New server funded by the Labour party by any chance?…they seem quite keen on supporting political blogs
April 18th, 2008 at 12:37 pm
David, you should have told Computerworld.
It is interesting that such is the success of your blog, you need fancy new equipment to operate/continue it.
Just think, a few years from now, your set up might become as sophistcicated as that of Trade Me or a government department.
April 18th, 2008 at 12:47 pm
Wow, I think you really need to get your plugins rewritten! Any chance of seeing any site stats? The number of comments doesn’t seem that massive to me, you’re getting around 40 per average post (emperical guess) but I’ve seen plenty of blogs with thousands. I’d be quite interested in average hits per day. I would’ve suggested cacheing over upgrading the hardware, I’m surprised at the current specs of this server let alone the new one!
Good news though is that your site is that well subscribed that it requires an upgrade. Congratulations!
April 18th, 2008 at 12:53 pm
The problem is the wordpress plugins. They are pretty shonky in many areas, but they are freeware/open source, so not too much to complain about. He is limited by the capabilities of the product – it isn’t like there are a team of people writing dedicated kiwiblog code here.
April 18th, 2008 at 1:04 pm
Yeah around 500 comments a day which isn’t huge huge. The plugins as Paul L says are what is available. If anyone wants to offer to write some dedicated plugins, I’d love that – esp a proper karma one.
April 18th, 2008 at 1:20 pm
I hope it’s an IBM David…
April 18th, 2008 at 1:22 pm
‘But what will everyone do during that time? My life will be meaningless.’
Not so,
hoolian go to MONKEYS WITH TYPEWRITERS end this kiwiblog madness for ever!
http://monkeyswithtypewriter.blogspot.com/
the revolution starts here!!!. Use this hiatus. Enrich your lives now before it is too late!
Or something…
April 18th, 2008 at 1:23 pm
oh yes. Congratulations, David.
April 18th, 2008 at 1:42 pm
HP > IBM
April 18th, 2008 at 1:57 pm
As PaulL says, those 53XX Clovertown chips are around 4 times as fast as the 3GHz hyper-threaded P4s (or perhaps even more). I’ve deployed onto a Dell that has two of these – 8 cores in total. The cost was (comparatively speaking) ridiculously cheap for the performance. Most of the cost was in the memory (it has 32GB – those 4GB ECC DIMMS were over the top for my needs). Total cost was almost $20K but would have been under $10K if it hadn’t have been for the memory – 16GB would have been plenty.
April 18th, 2008 at 2:01 pm
At the bottom of this pages source I had “Dynamic Page Served (once) in 1.282 seconds”. That seems like a hell of a long time to generate a page! I hope you try rebuilding your indexes(especially the karma and comments table) before you upgrade the server, just to get an apple to apple comparison. Your new server will automatically have it’s indexes rebuilt when you install it making the server seem alot faster than it possibly is. This is assuming you don’t rebuild your indexes all the time anyway.
April 18th, 2008 at 2:03 pm
HP > IBM? yeah, for printers, PCs and other low-margin gear… which is why we don’t make them anymore.
April 18th, 2008 at 2:40 pm
HP > IBM in price only.
April 18th, 2008 at 2:56 pm
David, stupid question, but do you have WordPress Caching enabled? If you don’t, that will probably help a lot – depending, of course, where your bottleneck is.
Obviously throwing more hardware at the problem will help, but WordPress is notorious for being unable to handle heavy loads – especially when they come in burst. This is a problem with dynamic pages in general, every hit results in sometimes dozens of DB queries, but I’m sure you know this.
Static Page generation is a huge plus of the Movable Type platform, that you incidentally moved away from… Also, will you be upgrading to WordPress 2.5? I hear that has many improvements as well
April 18th, 2008 at 3:11 pm
Bevan: Only because they underspec
April 18th, 2008 at 3:39 pm
Now that is a geek war. Get some real iron, you need a zSeries. Serve whatever you want off one of those…..
April 18th, 2008 at 4:42 pm
I suspect there are a few things you could do to reduce the load. For instance I’m a looking at this story and I can’t see the pools, logged in users etc but the scripts to create them are still being run. You should disabled these properly on pages where they are not being used.
April 18th, 2008 at 4:49 pm
Hmmm. Good idea Simon.
April 18th, 2008 at 5:38 pm
Oops, on the wrong thread. This is obviously the thread for balding single men who still live with their mothers.
I must go straight to a sports site to cleanse myself of this nerdy nonsense.
Goodbye.
April 18th, 2008 at 5:45 pm
The karma php code creates 1 entry per comment and hits the database once for each comment. 90 comments = 90+ hits of the database per page load! That’s the problem with modular architecture, simplicity often beats efficiency. You’d need to rewrite the karma code to cache the karma results because if you try and change the db structure for efficiency, you can’t upgrade to new versions of wordpress. Would love to know if the indexes are being rebuilt every couple of hours too.
April 18th, 2008 at 6:02 pm
nice! very nice! 8 gigs of ram! oh yeah!! this is like porn.. go to 32 gig now! ya know youre gonna need it
April 18th, 2008 at 6:13 pm
103PapPap says:
Don’t be dissin’ Bill Gates like that, PapPap. It’s not his fault he’s got filthy rich on the back of software so fundamentally unfit-for-purpose that if there were any justice in the world he’ll one day be hooked up to some sort of life support system, only to discover it’s running the latest version of Windows.
April 18th, 2008 at 6:30 pm
Rex: very funny.
labrator – interesting, doesn’t sound that hard to change. I’m presuming the karma plugin gets called once for each comment, so it isn’t possible to rewrite it as a single query? So you need some way for persistent storage within the page, allowing the karma plugin to retrieve the whole page up front, and just return from cache for all the remaining comments on the page? Shouldn’t be too hard to write.
April 18th, 2008 at 7:34 pm
Quad Core Xeon, 8Gbytes of Ram, and hardware Raid 10 disk array, just to run a blog? Boy, is it wordpress or is it that people who can optimize things have left the country?
April 18th, 2008 at 9:26 pm
Berend , KB is probably the highest load of any ‘kiwi blog’. So there wouldnt be any one here to optimise to start with, ie been there done that.
Now if it was crystal oscillators it would be a different story
April 18th, 2008 at 10:56 pm
The blog does has WP Cache installed and this has improved performance. The plugins are pretty much take them or leave them. As I said, if anyone wants to write a new plugin just wave your hand!
April 19th, 2008 at 4:10 pm
Much faster. Very good work.