So, this just got weirder!
It turns out it has nothing to do with publishing a blog post - that works perfectly fine.
The 500 internal server error is triggered by posting a link to the blog post on Twitter! We have tried it twice now, and it happens everytime - post a blog post, load it over and over again, the site is perfectly fine! Then take the link to the post, and post it on twitter.
The site will then get hit by twitterbots, yahoo, etc just after this and in no time, will start spitting out 500 internal server error because the web server runs out of memory.
This is so weird, I have no idea what to look for or do now - anyone have any ideas?