34SP.com System Status History

Description

Update at 08:48:46(BST): The issue has now been resolved with this server.Update at 08:48:42(BST): The engineers have now resolved the issue affecting the MySQL service so all aspects should be back up and running.

We have received reports of MySQL issues on the server which will be affecting Emails and any sites with database driven content.

Our engineers are working on the issue now and normal service should resume shortly.

Start Time

26th May 2011, at 8:28AM

Description

Update at 17:42:55(BST): All now seems well. This outage is now marked as resolved.

---

Update at 17:19:43(BST): There is quite a high load on the MySQL datbase at present (which is in itself, rising and falling). We're investigating the cause of this, which is most-likely to be as a result of rogue/un-optimised queries.

---

Update at 16:47:48(BST): The machine rebooted cleanly and we are working on ascertaining the cause of the fault.

We will be checking the consistency of all databases and monitoring from this point onwards.

---

Business8 appears to have suffered a kernel panic, we've restarted the machine via the power controllers and will be monitoring the machine as it reboots.

Start Time

20th May 2011, at 4:45PM

Description

Update at 15:46:31(BST): This outage is now marked as resolved.

---

Update at 09:47:58(BST): The reboot has succeeded and the machine is back-up and running smoothly again.

We'll be checking all databases as a matter of course and shall continue to monitor the situation.

---

Update at 09:39:41(BST): Having no luck logging-in remotely, we have been forced to hard-reboot this machine. It is likely that the machine exhausted its swap memory during the surge.

The machine is coming back up now.

---

At present xenserve9 is unresponsive, causing issues with services. This is related to a surge in traffic that has caused a very high load.

We are working as quickly as possible to return the machine to a working state.

Start Time

18th May 2011, at 9:20AM

Description

Update at 17:29:23(BST): This outage is now marked as resolved.

---

Update at 17:10:52(BST): Both machines should now be functioning correctly once again.

---

At present xenserve9 is experiencing issues related to the MySQL process that is causing a number of services/websites to fail.

Coincidentally, xenserve17 is also experiencing a high level of traffic towards one or two websites, causing a high load that is making website loading particularly sluggish.

We hope to have both matters under control as soon as possible.

Start Time

17th May 2011, at 4:54PM

Description

Update at 09:28:27(BST): And as soon as we've posted, it appears that the developers have now resolved the issue.

---

We're aware of an issue affecting the speed of our main website, www.34sp.com and are working to resolve this.

Customers needing to manage their account are advised to visit https://account.34sp.com directly.

Thank you for your patience during this time.

Start Time

16th May 2011, at 9:26AM

Description

Update at 03:39:28(BST): Update at 18:09:43(BST): We continue to monitor the status of this machine but services are currently available.

Update at 17:41:55(BST): We are currently rebooting this server to clear the current issues..

We thank you for your patience and hope to have service restored as soon as possible.

---

We're currently investigating an issue that is affecting Business4. At present the machine is up and this looks more network related than anything.

More updates as they come.

Start Time

13th May 2011, at 5:19PM

Description

Update at 12:14:55(BST): Both these servers are back in service so this issue is now resolved

Update at 11:55:47(BST): Server14 and Server3 seem to have been the two servers the worst-affected and lingering load issues are unfortunately still continuing.

We're working to get this resolved ASAP.

---

We've been alerted to a number of servers with high loads. Upon further inspection this appears to have been an automated attempt to find insecure PHPMyAdmin installations on our servers.

Whilst the level of network traffic was not noticeably high, the requests to the web servers were driving-up the load placed on these servers, causing a loss of service for certain customers.

We have now blocked this IP at our network edge and will be contacting the appropriate abuse department.

Start Time

12th May 2011, at 11:34AM

Description

Update at 09:38:00(BST): This outage is now marked as resolved.

---

Update at 08:44:03(BST): Looking at the surge of incoming traffic, this was apparently a spate of DDoS traffic hitting a machine on the same switch as server12 & server14.

This has quickly subsided and we're working on identifying the target application.

---

We've noticed an issue that is affecting one of our switches and this currently appears to be resulting in no access to services on both xenserve12 and xenserve14.

We'll update as and when more information is available.

Start Time

6th May 2011, at 8:35AM

Description

Update at 10:46:52(BST): Update at 10:45:20(BST): The issue has now been resolved with this server.Fire alarm cleared and full service has now resumed. Apologies for any inconvenience caused. 10:03:15(BST): Support and Livechat are currently offline due to a firealarm and building evacuation.

Start Time

28th April 2011, at 10:20AM

Description

Update at 23:07:51(BST): The issue has now been resolved with this server.Update at 23:00:42(BST): Business8 has now been power cycled.

This server is presently offline. At this time we are investigating this fault.

Start Time

27th April 2011, at 10:31PM