34SP.com System Status History

VPS Server issue (VPS18)

resolved

Description

Update at 11:43:21(GMT): The issue has now been resolved with this server.The engineers have now resolved the issue that was causing the outage and all is not running back as it should.

We are currently experiencing issues with our vps18 server. Our engineers have been notified to the issue and are investigating. We hope to have resolved shortly.

Please accept our apologies for any inconvenience caused.

Start Time

18th February 2012, at 10:17AM

Resolution Time

18th February 2012, at 11:43AM

Phone support evening of 13/02/12 (Phone support)

resolved

Description

Update at 08:48:41(GMT): The issue has now been resolved with this server.Our phone support system is presently offline. It will be accessible again when this status message is resolved.

Start Time

13th February 2012, at 6:12PM

Resolution Time

14th February 2012, at 8:48AM

Siteadmin logins (Multiple servers)

resolved

Description

Update at 20:02:06(GMT): Update at 19:09:54(GMT): Update at 23:18:38(GMT): Due to a recently discovered Plesk vulnerability, several servers have had their siteadmin login details reset.

Affected servers include:

business: business4
reseller: reseller6,7,8,12
professional: prohost12
personal: k9,robbie,ted

If you have been affected, we will have emailed your registered email address with full details, and how to proceed.

Start Time

12th February 2012, at 11:18PM

Resolution Time

27th February 2012, at 8:02PM

Slowed network (Multiple servers)

resolved

Description

Update at 18:11:34(GMT): Update at 22:52:13(GMT): Services should now be running much smoother. However users may see one off minor glitches as we work to patch systems affected by the weeks attacks.

We are presently seeing a spike in traffic across our network which is causing slowed access to several systems (notably reseller servers 2, 6, 7, 8).

Most servers and systems should be accessible albeit with reduced speed at this time.

Start Time

12th February 2012, at 7:26PM

Resolution Time

13th February 2012, at 6:11PM

Xenserve 4 unreachable (xenserve4)

resolved

Description

Update at 19:09:49(GMT): Update at 19:15:15(GMT): Back online - investigations continuing as to why this is happening.




Update at 18:54:30(GMT): Server is currently offline again, we are continuing to investigate this.




Update at 17:44:06(GMT): The server has had a full disk check and is now back online, a database consistency check is currently running.

We will continue to monitor the server over the next 24hrs.




An automated service restart was initiated on server4 at 16.47. This failed to restart cleanly, and engineers were notified. A full reboot command was issued at 17.00 - this has so far not brought services back online. Engineers will continue to attempt to bring services back online as soon as possible, and we will update the status as the situation develops.

Start Time

12th February 2012, at 5:21PM

Resolution Time

20th February 2012, at 7:09PM

Phone Support 12th Feb (n/a)

resolved

Description

Update at 22:52:24(GMT): Update at 14:14:15(GMT): Due to unforseen circumstances, phone support for today, Sunday 12th February, will be via email only. Phone support will resume tomorrow, Monday 13th. Apologies for any inconvenience caused.

Start Time

12th February 2012, at 9:35AM

Resolution Time

12th February 2012, at 10:52PM

Network slowness (Multiple services)

resolved

Description

Update at 19:25:14(GMT): Update at 14:52:26(GMT): Whilst we continue to monitor the situation, we believe at this time that the problem has been resolved and that all services should be performing normally.

Update at 09:07:35(GMT): Engineers have identified a number of servers that are being maliciously flooded with requests. The increase in traffic has had a knock on effect to most servers in our network. Engineers are currently working on blocking IPs that initiated this attack, and searching out and replacing code on affected sites.

Update at 08:07:27(GMT): We have seen a few shorter and less serious repetitions of the same problem, and continue to solve them each time. We are looking to implement a more permanent fix as soon as possible.

Update at 22:48:21(GMT): Update at 22:39:42(GMT): Most services should now be back online and accessible at normal speeds.

Update at 22:19:35(GMT): We have started to isolate the cause of the network load and work has begun to mitigate the excess network traffic. Some users will now see services back to full operation while others may find access intermittent while work continues.

Update at 21:09:08(GMT): We now have an engineer onsite and are working on correcting the fault.

Update at 20:36:19(GMT): We presently have an engineer on site at one of our datacenters investigating this issue.

We are presently investigating slowed access to our network. All hosted services may experience access issues at this time.

Start Time

7th February 2012, at 8:27PM

Resolution Time

12th February 2012, at 7:25PM

Reseller accounts emergency maintenance (Various)

resolved

Description

Update at 20:37:28(GMT): Update at 10:13:20(GMT): We have completed this maintenance and are monitoring the situation on these servers to see that it has been improved.


Reseller accounts hosted on the following servers will be unavailable for a short period beginning very shortly.

server1.xenserve.com
server3.xenserve.com
server5.xenserve.com
server9.xenserve.com

Last night during some maintenance work on server1.xenserve.com we discovered a problem affecting the above 4 servers. It is imperative that we correct this as soon as possible as it is affecting both performance and stability of the servers. To do so will require a short period of downtime for all of these servers and we intend to do this very shortly. We apologise for the inconvenience that this will cause.

Start Time

7th February 2012, at 9:31AM

Resolution Time

7th February 2012, at 8:37PM

Prohost3 emergency DB work (prohost3.34sp.com)

resolved

Description

Update at 00:44:57(GMT): At midnight, databases on the PH3 server will be unavailable for approximately 10 minutes time, while emergency work takes place.

Start Time

6th February 2012, at 11:40PM

Resolution Time

7th February 2012, at 12:44AM

Slow access to reseller accounts on server1.xenserve.com (server1.xenserve.com)

resolved

Description

Update at 09:44:40(GMT): The issue has now been resolved with this server.server1.xenserve.com is now available again and the fault has been corrected.


Update at 09:29:31(GMT): The problem with this server has revealed a larger fault. To rectify this we are having to reboot the server now. This will cause a short period of downtime in the next few minutes. When the server comes back to service, performance will be greatly enhanced and stability improved.


Update at 21:52:37(GMT): We are now performing a full disk check on this server. Unfortunately while this is in progress, all services will be unavailable.


Our engineers are currently investigating the cause of slow performance on server1.
We will update this status as soon as we know more.

We thank you for your patience. Apologies for any inconvenience caused.

Start Time

6th February 2012, at 8:32PM

Resolution Time

7th February 2012, at 9:44AM