00:00 The new server is now online and most services (www, mail, db) are now running.
99% of www and db data was successfully recovered from the failed system
Email data fared less well and we are looking at some small loss of stored email. Email sent during the outage will be resent in due time by the sender.
----
The server has failed (hardware) during the migration process.
We are now migrating the data that was retrieved before the failure occured.
All server4 clients will have been mailed regarding the issues affecting the system and steps being taken.
16th July 2007, at 8:30AM
27th July 2007, at 2:06PM
07:40 We presently have an engineer on site looking at the issues affecting server4.
11:50 The server is now available again.
14th July 2007, at 7:41AM
14th July 2007, at 9:48AM
19.18 - This issue has now been corrected and all services restored.
19.15 - There is an issue with this server that we are presently looking into.
7th July 2007, at 7:16PM
7th July 2007, at 5:19PM
04:05 the issue was traced and corrected. All services restored.
We are presently looking into an issue with this server.
6th July 2007, at 10:16PM
7th July 2007, at 2:08AM
Update at 13:00:
--------------
The BGP session to Level(3) ha sbeen restored following a hardware replacement by them. All connectivity is currently functinoal. We are monitoring this connection and will be contacting affected users today.
Update at 01:55:
--------------
IP updates have now been completed on business2 accounts also.
Please note that if you host your own DNS you will need to make IP address changes. This will be for any users not using ns and ns2.34sp.com for their domain.
If this is the case please contact us ASAP for your new IP details.
Update at 01:45:
--------------
IP updates have now been completed on spam2 accounts now.
Update at 01:00:
--------------
As issues could not be quickly resolved with Level3 work began on migrating affected accounts to new address space at around 23.40.
Business1 users should now find their sites routing over the Abovenet connection. Business2 and spam2 accounts are presently being migrated also.
Update at 18:15:
--------------
We are still waiting to hear from Level3 regarding the failure on their network.
At this time only users of our business servers and spam servers should be affected.
All other users are connected to our multi homed network and will be routed through our other connection during this period.
Update at 17:15:
--------------
A Level(3) engineer has gone to physically check their router.
Update at 16:50:
--------------
The issue has been traced to older IP addresses being routed by Level(3). New IP addresses in the 80.82.xx.xx range are not affected. Both 34sp.com engineers and Level(3) continue towards resolving this issue.
Network traffic is currently being impacted on, causing some sites to load slowly or be temporarily unavailable. We are currently investigating the cause and seeking a speedy resolution.
3rd July 2007, at 3:44PM
4th July 2007, at 12:39PM
Webmail is unavailable on the server Robbie at this time. We have an engineer investigating the causes and possible resolution of this issue at this time.This issue has now been resolved at 13:50
--------------
30th June 2007, at 11:35AM
30th June 2007, at 11:50AM
Update: Full connectivity has been restored. Our bandwidth providers are investigating the cause of the problems we have seen so that we can make any necessary changes.
You may presently be seeing some packet loss on your connection to 34SP.com
Both of our transit links are up, but one does presently seem to be experiencing issues.
We are currently looking into this matter.
28th June 2007, at 12:54AM
27th June 2007, at 11:43PM
9.05PM services restored
7.42PM server rebooted
22nd June 2007, at 7:42PM
22nd June 2007, at 7:12PM
11.37PM server back online and services restored
11.31PM server power cycled.
18th June 2007, at 11:24PM
18th June 2007, at 9:38PM
Server rebooted at 04:01
13th June 2007, at 4:01AM
13th June 2007, at 2:10AM