34SP.com System Status History

Server9 (Server9.xenserve.com)

resolved

Description

Update at 22:21:54(BST): The issue has now been resolved with this server.We are currently investigating an error on this system.

Start Time

24th August 2010, at 10:02PM

Resolution Time

24th August 2010, at 9:21PM

Xenserve13 Unresponsive (server13.xenserve.com)

resolved

Description

Update at 22:22:21(BST): Update at 22:16:00(BST): Slight misjudgement - it does appear to be multiple bots, but only one domain. A work-around as been found to circumvent the issue and the customer has been notified via a support ticket. There have been no load spikes for some time now but we will continue to monitor the machine for now.



Update at 21:14:59(BST): It seems that a single bot has been visiting a single domain and causing havoc. We've blocked the bot (an IP of Chinese origin) at our router's edge and will continue to monitor.


Update at 20:41:22(BST): It seems we were a little too quick to resolve this outage. The load on the server has shot up again since and we're still trying to get to the bottom of the cause.


Update at 19:22:58(BST): The issue has now been resolved with this server.The server fault has now been corrected and we are looking at what caused this issue.

Update at 18:22:45(BST): Remote restoration attempts have proved unsuccessful so an engineer is now being dispatched to the datacenter to manually inspect and attend to the system.

Update at 17:55:56(BST): The server is now being rebooted, in an attempt to restore services.

Update at 16:55:37(BST): Xenserve13 (server13.xenserve.com) has become unresponsive. Engineers are currently working to restore service as soon as possible.

Start Time

23rd August 2010, at 4:55PM

Resolution Time

24th August 2010, at 9:22PM

Prohost1 Mail Queue (prohost1.34sp.com)

resolved

Description

Update at 15:10:19(BST): The issue has now been resolved with this server.
The mail queue on Prohost1 was finally emptied of all malicious content this morning. We apologise for the severe delay in receipt of any e-mails submitted via sendmail.



The mail queue has seen some quite substantial abuse from a user's poor coding/validation. The root issue has now been solved and the owner will be notified.

However there is a queue of spam on the server waiting to go out that needs to be removed (to the tune of 300,000+ e-mails) therefore it will take some time to clear these messages out.

We ask all customers to please be patient - e-mails sent via sendmail (usually via PHP's mail() function) will be delayed significantly. Please ensure that all important site-related e-mails are sent via the provided Professional hosting SMTP service, to maintain reliability.

Start Time

22nd August 2010, at 5:52PM

Resolution Time

23rd August 2010, at 2:10PM

Queeg Down (queeg.34sp.com)

resolved

Description

Update at 18:22:39(BST): Update at 18:22:19(BST): The server was rebooted and has been returned to service. All databases have been checked and are working fine. If you continue to have issues with Queeg, please contact support via support@34sp.com



This server has failed and we are presently investigating the fault. All hosted services affected.

Start Time

19th August 2010, at 5:50PM

Resolution Time

19th August 2010, at 5:22PM

Server8.and server9 (Server8.xenserve.com and Server9.xenserve.com)

resolved

Description

Update at 16:21:10(BST): The issue has now been resolved with this server.We are presently looking into faults on both of these systems.

Start Time

19th August 2010, at 5:34AM

Resolution Time

19th August 2010, at 3:21PM

Support Availability - Wednesday 18th August.

resolved

Description

Update at 05:33:46(BST): The issue has now been resolved with this server.Our support department will be closing at 5.30pm on Wednesday 18th August, to allow for system maintenance. Please ensure you contact us before this time for any urgent cases. All other cases will be dealt with first thing Thursday morning.
Apologies for any inconvenience this may cause.

Start Time

16th August 2010, at 10:54AM

Resolution Time

19th August 2010, at 4:33AM

Atmail E-mail System (webmail.34sp.com)

resolved

Description

Update at 12:25:29(BST): The issue has now been resolved with this server.



We're investigating an issue with one of the IMAP/POP3/Webmail servers utilised by the Atmail (@mail) system.

This is affecting multiple customers utilising our Spam Filtering services, characterised by slow or failed IMAP/POP3 logins.

We are working to rectify this fault as soon as possible.

Start Time

13th August 2010, at 12:15PM

Resolution Time

13th August 2010, at 11:25AM

ASP Business Server fault (business3.34sp.com)

resolved

Description

Update at 05:33:52(BST): The issue has now been resolved with this server.Update at 10:27:05(BST): All email and files that can be restored have now been restored. We are currently working to try and retrieve any databases that we can from the failed server, but advise customers to restore these from their own backups where available.

--

Update at 01:11:28(BST): Most sites should now be restored and active. Email should be live, and www data (that we could recover) restored.

Databases have not been restored. A full update email will go out to all affected clients first thing in the morning on 13/08/10.

--

Update at 23:00:16(BST): Throughout the evening we have been moving the recovered data from business3 to new accounts on business7. All users now have accounts recreated on this server, plus any data we could recover has been moved too.

The final stages are now underway which requires some DNS updates.

--

Update at 16:12:47(BST) :We have now restored all the email that could be retrieved from the failed server to the mailboxes on our atmail system. Work to restore sites is still under way.

--

Update at 12:09:56(BST) :An email has been sent this morning to all account holders on this server. Email has been reactivated on our atmail system and should work with your existing details. We are working to import mail held on the failed server to this.

Web sites are being transferred to a different server and we will attempt to restore these as fully as we can as soon as we can.

--

Update at 23:29:02(BST) :Attempts to restore the server have failed. We are now in the process of recovering as much data from the server disks as possible. Due to the amount of data and the nature of the failure, this is expected to take a significant amount of time, at least into 12/08/10.

Once we have recovered as much data as possible we will be beginning further restoration efforts. We will also be emailing all affected clients with a detailed update during Thursday.

--

Update at 21:49:16(BST) :Work has been continuing throughout the evening in an attempt to restore the system and save as much data as possible. Updates to follow as soon as further progress is made.

--

Update at 17:44:49(BST) :Server3 has experienced a severe fault, which we believe to be disk related at this time. We are now in the process of trying to recover as much data from the server as possible. We presently have no ETA on when services will be restored.

--

Update at 17:05:19(BST): As an emergency measure we have had to reboot the ASP Business Server.

Apologies for any inconvenience caused.

Start Time

11th August 2010, at 5:04PM

Resolution Time

19th August 2010, at 4:33AM

Server7.xenserve.com (Server7.xenserve.com)

resolved

Description

Update at 18:09:35(BST) :The issue has now been resolved with this server.We are presently looking at issues affecting this server, primarily www problems.

Start Time

9th August 2010, at 5:44PM

Resolution Time

9th August 2010, at 5:09PM

server17.xenserve.com (server17.xenserve.com)

resolved

Description

Update at 21:56:24(BST) :The issue has now been resolved with this server.We are presently investigating a fault affecting services on this server.

Start Time

5th August 2010, at 9:43PM

Resolution Time

5th August 2010, at 8:56PM