34SP.com System Status History

Description

Update at 18:38:25(BST): Holly is back-up and running and all looks good.

---

We're unfortunately having to perform a clean reboot of the Personal hosting server Holly.

An exploited website has caused a number of issues that has resulted in the website being suspended. To fully clear up the mess, we are sadly forced to reboot the machine.

This will be happening shortly and should take no longer than 15 minutes.

Start Time

28th March 2011, at 6:17PM

Description

Update at 18:11:45(BST): Update at 11:29:53(BST): We are still monitoring the server. Mail should be arriving as normal now. Thank you for your patience.

Update at 18:13:11(BST): We are still waiting on the e-mail queue on this server to drop to sensible levels.

Please accept our apologies for the obvious inconvenience that this is causing.

---

We have become aware of a large influx of spam. This is causing delays in mail delivery. We are working hard to bring this down as soon as possible. Apologies for any inconvenience caused.

Start Time

28th March 2011, at 3:06PM

Description

Update at 18:12:13(BST): This outage is now marked as resolved.

---

Update at 08:41:01(GMT): Server is now back online, we are monitoring this at this time.

---

This server is currently offline, we are looking into this.

Start Time

23rd March 2011, at 8:20AM

Description

Update at 03:54:42(GMT): VPS' appear to be running normally -- status marked as resolved.

---

Update at 03:01:44(GMT): Reboot and relocation of the power successful: we also got a few key upgraded packages installed, which provides a nice extra benefit to the exercise.

All VPS' should be up and running shortly.

---

As a reminder to customers with VPS' on VPS14, we will be moving this machine to a separate power port shortly.

All virtual machines will be shut down cleanly before rebooting.

Start Time

23rd March 2011, at 2:22AM

Description

Update at 00:49:30(GMT): Down and up very quickly -- xenserve14 should be running on all four cylinders now.

Server14/Xenserve14 will be going down for a reboot as per the e-mails sent to customers last week.

The reboot is to correct a simple BIOS mis-configuration from the factory and should result in more performance for users.

We do not estimate that this work will take much longer than 15 minutes at most.

Start Time

23rd March 2011, at 12:04AM

Description

Update at 02:13:40(GMT): All VPS appear to be up and running without much issue. Status update marked as resolved.

---

Update at 02:04:56(GMT): VPS' are booting-up sequentially, one by one. All VPS' should be back up within 10-20 minutes.

At this point I/we apologise profusely for over-running the original downtime window.

We will definitely be bringing up the methods of BIOS upgrading via Linux with Dell technical support (again.)

---

Update at 01:56:49(GMT): Success! Rebooting in a moment, VPS' will come up shortly thereafter.

---

Update at 01:45:29(GMT): Make that a lot longer to load than expected. We may well abort this work if it doesn't load soon.

---

Update at 00:51:17(GMT): We're having to resort to the 'slower' method of updating the BIOS thanks again to the outdated BIOS version that lacks the UEFI features for updating itself.

This just involves PXE-booting a Dell live CD image, which takes a short while.

---

As a reminder to all customers, VPS accounts hosted on VPS19 will be going down shortly so that we may upgrade the BIOS on the host.

This is to provide proper support for the processor shipped with the machine, as this was initially missing from the factory.

Without this update, should the machine ever be restarted, it would currently not reboot by itself without human interaction, leading to an extended period of downtime even under such basic circumstances.

We hope to have this work completed without the hour.

Start Time

23rd March 2011, at 12:02AM

Description

Update at 09:02:08(GMT): The issue has now been resolved with this server.This issue has now been resolved.

We are currently investigating an issue regarding this server, it is currently offline.

A engineer is on the way to the datacenter to look into this in more detail.

Start Time

22nd March 2011, at 8:30AM

Description

Update at 23:29:39(GMT): Some malicious processes seem to be the 'cause' of the problem, in the sense that they attracted the traffic rather than were the target of it.

We're currently clearing away this malicious code and shall be contacting any responsible owners in due course.

---

Update at 23:19:56(GMT): The issues appear to have subsided; the target being a shared hosting service.

The source of the attack was highly distributed, but we are working to try and identify it in some fashion.

---

We are currently experiencing heightened network traffic on all routes to a target somewhere in London that is causing problems for customers in that data centre.

We are currently working with our providers to resolve this.

Start Time

19th March 2011, at 10:45PM

Description

Update at 23:58:59(GMT): Status post marked as resolved.

Update at 15:01:00(GMT): K9 is back up (and has been for a short while) and we're currently checking all databases as per routine.

We'll continue to monitor this machine for a short while.

The Personal Hosting server, K9, is currently down.

We're currently working on restoring this server ASAP.

Start Time

17th March 2011, at 2:15PM

Description

Update at 14:11:21(GMT): This outage is now marked as resolved.

Update at 11:48:37(GMT): The machine is now back up and running. We'll be making some routine checks and monitoring the server for a short while hereafter.

One of the Personal hosting servers, Twiki, has gone down. We're working on bringing the machine back up to working order as soon as possible.

Start Time

14th March 2011, at 11:40AM