34SP.com System Status History

Phone Support

resolved

Description

Update at 08:41:20(BST): The issue has now been resolved with this server.Phone support is now available, 34SP apologies for any inconvenience caused.

Phone support it currently unavailable. It will be available again by 9am.

Start Time

11th October 2012, at 8:22AM

Resolution Time

11th October 2012, at 7:41AM

VPS26 Unavailable (VPS26)

resolved

Description

Update at 17:06:44(GMT): Update at 12:05:26(BST): Server maintenance was completed by 01.00, however, the server is still reporting that it is not using all available memory modules. As CPUs and the motherboard have all been replaced now, we are exploring options with Dell as to the next step. The server will continue to be closely monitored in the meantime.

Update at 22:50:52(BST): Maintenance will begin in a little over an hour. The server will go offline, have a CPU replaced and come back online. Services should be back around 01.00 for most users.

Update at 14:19:13(BST): The maintenance window has been brought forward 24 hours and the faulty CPU will be replaced at midnight tonight. This should result in 20-40 minutes downtime as all VPSs are started up following the replacement. We will do our best to ensure the downtime is as short as possible.



Update at 02:30:57(BST): The server is coming back up now. All VPS'es should be up within 30 minutes.

We were unable to 100% correct the hardware fault at this time. The machine will run at a slightly reduced resource capacity (40GB memory, rather than 48GB) for the until midnight on Saturday when we plan to complete another maintenance session.

In the meantime we do not expect any faults from the server, or any significantly reduced performance.


Update at 01:42:28(BST): To update everyone, two engineers are still on site reviewing the hardware updates. Some final tweaks are taking place, but we are confident the hardware issue is now accurately identified. The server should be back online within 60-90 mins.


Update at 00:31:21(BST): VPS26 is going offline one last time for a final motherboard replacement. This should be the last work required on the machine.


Update at 22:02:57(BST): While the server is up, and has been since our last update, the replacement motherboard was a stopgap measure and is partially broken, offering only 40 of the 48GB available memory on the server.

We are awaiting delivery of a new motherboard and hope to install that this evening when it arrives. While this will cause a little more brief downtime, it will allow us to restore the server to 100% capacity.


Update at 20:31:54(BST): Users may start to see their VPS slowly come back online over the next 30 minutes time. Please note while services may be restored, we are not closing this case, as underlying work on the hardware setup continues.


Update at 18:44:32(BST): As a more detailed update on the progress so far. Dell have a set a repair process that must be followed and is presently underway.

Initially the motherboard and memory of the server were replaced but the server is still failing to function correctly. A second engineer is about to arrive on site to perform further steps, the first of which is to update the BIOS of the new motherboard.


Please note we do have contracts in place for Dell to work on this 24/7 until the hardware is corrected.


Update at 16:24:19(BST): The Dell engineer is still identifying the fault. So far, various replacement parts haven't helped in getting VPS26 back online. They're continuing to investigate.


Update at 14:40:24(BST): The Dell engineer is now on site and investigating the issues with the server.


Update at 12:22:04(BST): BUSINESS VPS CLIENTS PLEASE NOTE: As business account SMTP validation is carried out on VPS26, using smtp.yourdomain.com will probably not work at present. You can either update your mail client to use smtp.34sp.com OR instead log in using your web browser temporarily at webmail.yourdomain.com (where yourdomain.com is your actual domain name).


Update at 10:45:50(BST): Following diagnosis with their support department, an engineer from Dell is on the way to our datacentre and is bringing spare parts for the server. They state that they will be on site within 4 hours.


Update at 10:37:55(BST): We have confirmed that the server is suffering from a hardware failure and are working to resolve this with our suppliers as quickly as possible.


Update at 10:06:48(BST): This server has rebooted itself again. We will be investigating this urgently as soon as the server is available again.


Update at 09:41:40(BST): The issue has now been resolved with this server.


Engineers are currently investigating an availability issue with VPS26.

Start Time

11th October 2012, at 8:15AM

Resolution Time

31st October 2012, at 5:06PM

Prohost8 (prohost8.34sp.com)

resolved

Description

Update at 12:23:47(BST): Changes made prior to the reboot have taken effect and the server appears to be behaving as expected.


Update at 11:40:03(BST): Changes have been made and a system reboot was successful. Engineers are continuing to monitor the situation.


We are experiencing intermittent issues with the Prohost8 server. Engineers are issuing a reboot and looking into this further.

Start Time

8th October 2012, at 11:21AM

Resolution Time

8th October 2012, at 11:23AM

Mail issues (Multiple)

resolved

Description

Update at 11:20:03(BST): Update at 09:15:00(BST): Engineers have resolved the issue and are monitoring the point of failure.

Update at 09:12:51(BST): We appear to have an issue with sending emails from some of our accounts. We're currently investigating this and have engineers on site to get this resolved as soon as possible. We apologise for the inconvenience caused.

Start Time

8th October 2012, at 9:12AM

Resolution Time

8th October 2012, at 10:20AM

Prohost14 (Prohost14.34sp.com)

resolved

Description

Update at 19:16:25(BST): Update at 19:16:16(BST): With apologies for the lack of updates throughout the day, we are pleased to report that this server has been restored to service.



Our engineers are currently performing an emergency restart as part of investigation in to the servers recent performance issues. The server and services will take longer than normal to come back up and running due to additional scripts running to try and isolate problems.



34SP apologies for any inconvenience caused during by this work.

Start Time

7th October 2012, at 10:11AM

Resolution Time

7th October 2012, at 6:16PM

34SP.com database (34sp database)

resolved

Description

Update at 10:05:22(BST): Our engineers have now resolved the database issue and have performed maintenance across it. During this time you may have encountered issues with SMTP. If you have, please restart your mail client and this should work fine.

34SP apologies for any inconvenience this may have caused you.

The 34SP.com database server that runs our site and several authentication services is presently offline.

We are presently looking into this.

Start Time

7th October 2012, at 3:36AM

Resolution Time

7th October 2012, at 9:05AM

Mail delays on Ash (Personal Hosting) (ash.34sp.com)

resolved

Description

Update at 12:21:47(BST): The issue has now been resolved with this server.

Engineers are currently working on the Ash server mail queue. There have been some delays caused by spamming. We are working to clear this as quickly as possible. Apologies for any inconvenience caused.

Start Time

4th October 2012, at 9:26AM

Resolution Time

4th October 2012, at 11:21AM

businesshost2 down (businesshost2.34sp.com)

resolved

Description

Update at 01:40:43(BST): The issue has now been resolved with this server.The server has been restarted and the fault corrected. All services should now be restored

This server is presently down and we are investigating the fault.

Start Time

4th October 2012, at 1:13AM

Resolution Time

4th October 2012, at 12:40AM

Atmail MailStorage (ATMAIL CLUSTER)

resolved

Description

Update at 11:30:48(BST):
Update at 00:08:39(BST): This was back online after around 10minutes - do let us know if you do find any issues due to the changes made.



Over the next 30 minutes we will be bringing more storage online for the mail servers, this will involve approximatly 20minutes of downtime where email will not be accessible.

This is necessary work and we hope to have the service back up quicker than this.

If you do have any issues after this has taken place please do get in contact with us.

Thanks

Start Time

2nd October 2012, at 11:00PM

Resolution Time

3rd October 2012, at 10:30AM

Phone Support (n/a)

resolved

Description

Update at 18:33:50(BST): The issue has now been resolved with this server.Phone Support is temporarily unavailable. This should be returned to normal operation soon - in the meantime we apologise for any inconvenience caused.

Start Time

1st October 2012, at 5:37PM

Resolution Time

1st October 2012, at 5:33PM