34SP.com System Status History

Reminder: VPS19 BIOS Upgrade (vps19.34sp.com)

resolved

Description

Update at 02:13:40(GMT): All VPS appear to be up and running without much issue. Status update marked as resolved.



Update at 02:04:56(GMT): VPS' are booting-up sequentially, one by one. All VPS' should be back up within 10-20 minutes.

At this point I/we apologise profusely for over-running the original downtime window.

We will definitely be bringing up the methods of BIOS upgrading via Linux with Dell technical support (again.)


Update at 01:56:49(GMT): Success! Rebooting in a moment, VPS' will come up shortly thereafter.


Update at 01:45:29(GMT): Make that a lot longer to load than expected. We may well abort this work if it doesn't load soon.


Update at 00:51:17(GMT): We're having to resort to the 'slower' method of updating the BIOS thanks again to the outdated BIOS version that lacks the UEFI features for updating itself.

This just involves PXE-booting a Dell live CD image, which takes a short while.


As a reminder to all customers, VPS accounts hosted on VPS19 will be going down shortly so that we may upgrade the BIOS on the host.

This is to provide proper support for the processor shipped with the machine, as this was initially missing from the factory.

Without this update, should the machine ever be restarted, it would currently not reboot by itself without human interaction, leading to an extended period of downtime even under such basic circumstances.

We hope to have this work completed without the hour.

Start Time

23rd March 2011, at 12:02AM

Resolution Time

23rd March 2011, at 2:13AM

vps11.34sp.com (vps11.34sp.com)

resolved

Description

Update at 09:02:08(GMT): The issue has now been resolved with this server.This issue has now been resolved.

We are currently investigating an issue regarding this server, it is currently offline.

A engineer is on the way to the datacenter to look into this in more detail.

Start Time

22nd March 2011, at 8:30AM

Resolution Time

22nd March 2011, at 9:02AM

Network Congestion (N/A)

resolved

Description

Update at 23:29:39(GMT): Some malicious processes seem to be the 'cause' of the problem, in the sense that they attracted the traffic rather than were the target of it.

We're currently clearing away this malicious code and shall be contacting any responsible owners in due course.



Update at 23:19:56(GMT): The issues appear to have subsided; the target being a shared hosting service.

The source of the attack was highly distributed, but we are working to try and identify it in some fashion.


We are currently experiencing heightened network traffic on all routes to a target somewhere in London that is causing problems for customers in that data centre.

We are currently working with our providers to resolve this.

Start Time

19th March 2011, at 10:45PM

Resolution Time

19th March 2011, at 11:29PM

K9 Down (k9.34sp.com)

resolved

Description

Update at 23:58:59(GMT): Status post marked as resolved.

Update at 15:01:00(GMT): K9 is back up (and has been for a short while) and we're currently checking all databases as per routine.

We'll continue to monitor this machine for a short while.

The Personal Hosting server, K9, is currently down.

We're currently working on restoring this server ASAP.

Start Time

17th March 2011, at 2:15PM

Resolution Time

22nd March 2011, at 11:58PM

Twiki Down (twiki.34sp.com)

resolved

Description

Update at 14:11:21(GMT): This outage is now marked as resolved.

Update at 11:48:37(GMT): The machine is now back up and running. We'll be making some routine checks and monitoring the server for a short while hereafter.

One of the Personal hosting servers, Twiki, has gone down. We're working on bringing the machine back up to working order as soon as possible.

Start Time

14th March 2011, at 11:40AM

Resolution Time

14th March 2011, at 2:11PM

Network Congestion (N/A)

resolved

Description

Update at 16:38:17(GMT): We have not seen a recurrence of this issue, so we're marking this outage as resolved.



Update at 17:18:46(GMT): We have isolated this as a DDoS attack targeting one of our Business hosting servers (or a site there-on).

We are in contact with our providers to help mitigate this. For now, the traffic appears to have subsided.


We're aware of a problem affecting our network, resulting in slow connections/time outs.

We're trying to resolve this as quickly as possible.

Start Time

13th March 2011, at 5:08PM

Resolution Time

14th March 2011, at 4:38PM

Xenserve4 Issues (server4.xenserve.com)

resolved

Description

Update at 17:19:09(GMT): This outage is now marked as resolved.

Update at 21:10:26(GMT): A outdated installation of osCommerce appears to be the cause of the grief. We've made a quick fix and will be contacting the owner of the domain to arrange for a more permanent resolution.

Server4 has been unfortunately disconnected from the network whilst we work on preventing an exploit in someone's website.

The exploit has been used maliciously to flood our network with packets, which we are working to put a stop to as soon as we possibly can.

Start Time

12th March 2011, at 7:20PM

Resolution Time

13th March 2011, at 5:19PM

Business4 Apache Issues (business4.34sp.com)

resolved

Description

Update at 12:11:45(GMT): A single domain was found to be utilising a disproportionate amount of processing power and has now been disabled. We have contacted the owner to speak to them regarding this matter.

Business4 is currently having problems serving web pages. We're looking into the fault and hope to have this resolved shortly.

Start Time

11th March 2011, at 11:34AM

Resolution Time

11th March 2011, at 12:11PM

Phone Support Unavailable (N/A)

resolved

Description

Update at 09:49:23(GMT): The issue has now been resolved with this server.Staffing has now returned to levels permitting telephone support.

Unfortunately, due to sickness, we're currently unable to provide telephone-based support this morning.

Please note that all enquiries can be dealt with by e-mailing to support@34sp.com, with as much information as possible.

We apologise for any inconvenience that this may cause.

Start Time

11th March 2011, at 8:49AM

Resolution Time

11th March 2011, at 9:49AM

Prohost6 Essential Updates (prohost6.34sp.com)

resolved

Description

Update at 22:46:26(GMT): The issue has now been resolved with this server.Prohost6 back up and running again after a successful update.



Similar to last night's maintenance to prohost10 & 11, it has become apparent that Prohost6 also requires some updates that unfortunately require a reboot to take effect.

We hope to have this completed within 15-20 minutes -- e-mail will not be affected.

Start Time

10th March 2011, at 10:29PM

Resolution Time

10th March 2011, at 10:46PM