34SP.com System Status History

Description

Update at 13:54:14(GMT):

---

Update at 23:38:39(GMT): The upgrade to the storage has now been completed, we will monitor this for a short while before resolving this status.

If anyone is experiencing any issues with their email due to this upgrade please contact us with full details.

---

This is a reminder that the atmail cluster is about to go offline, this is to introduce some more storage into the system.

We expect the downtime to be less than 1hr, this status will be updated if this schedule is changed.

Start Time

31st October 2011, at 11:05PM

Description

Update at 10:24:27(BST): The changes made to the mail setup yesterday seem to have vastly improved the situation with the issues that people were having with their email.

There are some more changes still to come which should improve this further still that we are working on.

If anyone is still having issues with their email please do contact us with as much detail as possible as to the issue you are experiencing.

Update at 12:38:14(BST): The changes mentioned below are being brought online in a controlled manner to confirm that it is not causing any further issues, these changes rely on some dns updates taking place so its impact may not be immediate.

---

Update at 11:32:15(BST): We are currently making changes to our systems which will move those that use webmail to a seperate system, we will be bringing these changes online within the next hour slowly to ensure there are no issues.

Once this is done it should result in less connections going to the servers that server the pop3/imap connections where some users are experiencing issues with connectivity at this time.

---

Update at 10:50:09(BST): The issue with mail is continuing today. We are looking into ways to solve this as a matter of urgency.

---

Update at 19:56:56(BST): Email services should be largely working for all users at this time. If you do find you are still having lingering issues, please try restarting your email program to ensure you are not having caching issues.

---

Update at 16:07:57(BST): We are continuing to work on this problem and it should be improving for all customers affected. We have not finished yet and hope to bring additional server capacity online soon to restore the mail service to the standard you expect.

---

Update at 11:03:28(BST): Server load on the mail storage servers for @Mail is causing difficulties for some customers when trying to access their mailboxes via webmail, POP or IMAP. We are aware of this issue, and are progressively altering mail storage to better spread the load across the array now that a new server is in place. Users may experience connection issues during this time as messages are replicated between servers - if you do experience any problems of this nature, please try connecting to your mail account again after a short while (1/2 an hour should be fine). We apologise for any inconvenience caused.

Start Time

26th October 2011, at 2:11PM

Description

Update at 14:19:10(BST): Resolving.

Update at 02:28:25(BST): All containers are running on VPS18 quite happily now. most were up within 15 minutes of the host booting, but some took a little longer than others.

This concludes our maintenance window. If you have any lingering issues and/or side effects from this point onwards, please contact support ASAP.

---

Update at 02:00:14(BST): VPS18 is back up, one VPS would not shut down cleanly. This container had locked processes and was leaking memory.

Containers are starting now and should not take much longer. Sadly this is over the allotted 30 minutes and for that, we recognise the inconvenience and apologise for any untoward issues that this may raise.

---

Update at 01:27:58(BST): VPS18 will be going down for a reboot in a few minutes time.

---

Update at 01:27:34(BST): VPS15 back up and all containers started. If there are any issues, please let support know.

---

Update at 01:12:49(BST): VPS15 will be going down for a reboot in the next few minutes.

If we complete this by 01:30, we're back on schedule.

---

Update at 01:05:34(BST): VPS14 now back up, with all containers started and/or starting. If you find anything amiss, please notify support.

---

Update at 00:43:45(BST): VPS14 going down for a reboot very shortly.

---

Update at 00:43:20(BST): VPS11 is now running with all containers started. Please be sure to raise a ticket if you feel there is any issue with your VPS post-reboot.

---

Update at 00:22:28(BST): The @Mail server upgrade is complete. This should now be coming back up. We apologise for the delay to this maintenance.

VPS11 will be shutting-down now for re-location.

---

Update at 23:59:50(BST): Xenserve13 back up, finally.

We're now moving on to the RAM upgrade of the @Mail server. This will not take too long, but may push back the reboot of VPS11 by 10-15 minutes.

---

Update at 23:45:55(BST): Full upgrade of all CentOS packages happening on Xenserve13 (from a chrooted environment) at present. We appreciate that this needs to be resolved as quickly as possible and are continuing to work diligently.

---

Update at 23:31:16(BST): Unfortunately we've encountered a problem with xenserve13 that is preventing a complete, successful reboot. We're working as fast as we can to resolve this.

---

Update at 23:00:36(BST): Now it's xenserve13's turn to be moved. Shutting-down now.

---

Update at 22:46:51(BST): Lothos is now back-up and running.

---

Update at 22:29:16(BST): Lothos going down for moving in just under minute.

---

Update at 22:12:16(BST): Max now back up.

---

Update at 22:01:42(BST): 22:00, Max shutting-down for moving.

---

As a reminder to customers, certain machines are being relocated to newer cabinets this evening, as per e-mails that have been sent out recently.

Start times are intended to be as follows:

(Tues 25th)
22:00 - Lothos
22:30 - Max
23:00 - Xenserve13

23:30 - Emergency upgrade to one machine in the @Mail (Atmail) webmail system.

(Weds 26th)
00:01 - VPS11
00:30 - VPS14
01:00 - VPS15
01:30 - VPS18

Each move has a period of 30 minutes to complete and we should hopefully have most machines moved and rebooted in less than time-frame.

Updates will be posted to this page, so please check back.

Start Time

25th October 2011, at 9:38PM

Description

Update at 10:26:20(BST): This issue has been resolved.

---

We are presently seeing delays on outgoing emails across a number of our services.

Due to spam levels, certain virus scanning services failed, and mail queues began to build.

The issue has now been corrected and email is going out albeit with a slight delay.

We apologise for any inconvenience caused.

Start Time

24th October 2011, at 6:35PM

Description

Update at 15:36:20(BST): Update at 15:35:59(BST): The issue has now been resolved with this server.There were some corrupt databases on the server in which the engineers have now repaired. Everything back up and running as normal. \\



Update at 14:46:21(BST): vps16.34sp.com is suffering some speed issues at the moment. We are taking action to rectify this and can only apologise for the inconvenience caused.

Start Time

21st October 2011, at 2:46PM

Description

Update at 18:41:42(BST): The issue has now been resolved with this server.Telephone support is currently unavailable. We would like to apologise for this and will be offering support by phone again as soon as possible.

Start Time

18th October 2011, at 5:39PM

Description

Update at 10:58:58(BST):

---

Update at 20:32:47(BST): The migration has now been completed, please do let us know straight away if you find any issues with your domain that has been migrated from the pris.34sp.com server.

---

Update at 19:04:52(BST): DNS records are now being updated to the new server, please allow an hour for this to propagate.

---

Update at 17:37:02(BST): All services are offline on pris.34sp.com while we do a final sync of data.

---

Update at 16:04:52(BST): This migration is taking longer than expected for the initial data import, it is still ongoing and will be completed today.

---

This server is currently undergoing a migration to a different server, please try to avoid making any changes where possible to avoid any loss during this period.

Start Time

18th October 2011, at 8:07AM

Description

Update at 10:58:12(BST): The issue has now been resolved with this server.The server is now back on line. Apologies for any inconvenience caused.

Emergency Maintenance:

We have had to reboot this server in order to apply performance improvements. We thank you for your patience and hope to have the service back on line as soon as possible.

Start Time

17th October 2011, at 10:30AM

Description

Update at 15:57:04(BST): Update at 08:36:18(BST): The services on this Server are now coming back online.

---

We are currently investigating an issue with the holly.34sp.com server

Start Time

8th October 2011, at 8:32AM

Description

Update at 16:11:56(BST): This issue was resolved yesterday, an hour or two after it began.

---

Update at 20:54:45(BST): We are currently experiencing network issues affecting multiple servers. Engineers have been notified and are now investigating the matter at hand.

As soon as we have more accurate information we will update this status.

Please accept my apologies for any inconvenience caused.

Start Time

6th October 2011, at 8:42PM