Update at 18:09:44(GMT): We have also isolated the cause of the excessive load and are in the progress of now removing this vulnerability. We do not expect a repeat.
Update at 18:06:07(GMT): We have managed to lower the incoming connection load and increase performance for users. SMTP should now be working for all users again.
Update at 17:34:56(GMT): Update at 17:33:47(GMT): The issue was tracked to excessive load on our SMTP database array. This is a series of servers in a high availability cluster. The number of incoming authentication requests rose exponentially above typical usage levels.
As a result we believe this could be an external attack against the SMTP array and have been working to restore availability to clients. Some clients may presently be able to send, some may not. As we lower load and increase capacity, and work to stymie the attack, connectivity will improve.
Update at 16:16:05(GMT): Some of our customers may be experiencing issues when sending emails. Our engineers are currently dealing with this issue. We apologise for the inconvenience caused.
There will be no need for you to change your email account details.
This issue is also affecting some customers accessing their control panel or experiencing slowness. We hope to have the issue resolved shortly.
31st January 2013, at 3:46PM
31st January 2013, at 6:09PM
Update at 14:04:26(GMT): There have been no further reports of issues since last night and this report is therefore considered resolved.
Update at 20:31:57(GMT): During the evening we've seen several reports of poor connectivity for customers connecting to our network via BT.
To confirm 34SP.com has no issues at this time and the issue does appear to be with BT. Some initial investigation appears to suggest a fault between Bt and Abovenet.
We would advise our BT clients to contact their relevant customer support channels to confirm.
24th January 2013, at 8:30PM
25th January 2013, at 2:04PM
Update at 15:21:12(GMT): The issue has now been resolved with this server.The server is now back on line. Apologies for any inconvenience caused.
We are performing an emergency reboot due to kernel issues.
Apologies for any inconvenience caused.
23rd January 2013, at 3:09PM
23rd January 2013, at 3:21PM
Update at 07:26:49(GMT): The issue has now been resolved with this server.We are presently investigating an issue affected network load causing service instability for some users.
22nd January 2013, at 3:59AM
22nd January 2013, at 7:26AM
Update at 20:25:45(GMT): Update at 03:59:10(GMT): Update at 19:45:30(GMT): We are presently seeing some unexpected high load on this server. This is causing infrequent access issues on the system.
As the load is distributed across many sites on the server, there is no one single cause.
We are beginning a move of several sites from this server ASAP to reduce load.
In the meantime please email us if you wish to have your site migrated off of this machine.
18th January 2013, at 7:43PM
24th January 2013, at 8:25PM
Update at 23:05:45(GMT): The issue has now been resolved with this server.A network issue on the VPS host has been found and resolved now. vps25 should once more be stable and running well. Apologies for any inconvenience this may of caused.
Server VPS25 has become unresponsive. Our engineers are currently looking into this issue and hope to have things back shortly.
17th January 2013, at 10:21PM
17th January 2013, at 11:05PM
Update at 06:42:25(GMT): The issue has now been resolved with this server.We are presently looking at a fault affecting www on this server.
11th January 2013, at 12:22AM
11th January 2013, at 6:42AM
Update at 06:42:32(GMT): Update at 18:10:37(GMT): After investigation by our engineers it appears there was an exploited VPS container on the VPS that was causing the excessive traffic. Our engineers have taken against this now which has brought the server back up and running.
We are monitoring the server in the meantime to ensure that this does not reoccur.
Our flood protection has been activated on this server to prevent the rest of our network being affected. Our engineers are looking into the flood and the network connection to VPS27 will be back online as soon as this is resolved. We apologise for the inconvenience caused.
9th January 2013, at 5:14PM
11th January 2013, at 6:42AM
Update at 11:12:17(GMT): The issue has now been resolved with this server.The server has had a disk check and a full reboot onto a different kernel. This has resolved the issue and we apologise for the inconvenience caused.
Update at 10:58:49(GMT): An engineer is currently at the data centre attempting to resolve the issue as soon as possible.
Server19 is currently undergoing a reboot and disk check. We apologise for the inconvenience caused and it should be back online shortly.
9th January 2013, at 10:06AM
9th January 2013, at 11:12AM
Update at 09:36:29(GMT): Update at 09:17:57(GMT): Our engineers have rebooted the server to bring it back online and are looking into the issue that occurred.
server2.xenserve.com has become unresponsive and is currently undergoing a full reboot. Should any further actions be necessary once this has completed, we will post accordingly.
2nd January 2013, at 8:52AM
2nd January 2013, at 9:36AM