Plan B provided cover for a number of customers effected by last week’s flood and subsequent fire at BT’s aptly named ‘Burne House’ exchange in London.
The outage effected at least 437 local exchanges and over 37,000 data circuits, and took BT four days to fully recover from. If it hadn’t happened on the run up to a holiday weekend the consequences could have been even worse.
Plan B carried out two full customer invocations and was on standby for a further four customers. All invocations went perfectly and service was delivered to both customers within 30 minutes of them requesting their invocations.
Abdi Hersi commented that invoking the Plan B Service turned the incident into “the most enjoyable disaster ever”!
The incident highlighted a number of things worthy of note:
Firstly, and most obviously it illustrated that even with resilient facilities that are well protected, disasters do happen and the knock on effect can be significant. So it can happen to you!
Specifically, the incident highlights that the Burne House Exchange is a massive single point of failure for lots of services around the country, and even if you have multiple Internet connections from multiple suppliers (as one of our customers had) they can still end up terminating in the same BT exchange - or going along the same bit of BT fibre in the same BT duct - and you wont know about it until all your supposedly independent data lines get knocked out by a single BT failure. The lesson here is if you are buying independent data connections you will have to check exactly who your suppliers get their connections from and where exactly they go before you can be sure you have true independence for availability reasons.
We learned that for our customers, Email was the key application they wanted working in the event of a data connection outage and that they decided to invoke as soon as they knew the outage would be longer than 24 hours.
The incident also highlighted that when the length of the outage is uncertain, only a solution like Plan B’s that can be invoked with no effort within minutes as a fully working service is really fast enough and flexible enough to deliver value quickly enough to warrant taking the decision to invoke. A ‘build from back-up’ or ship-to-site solution would simply never have been able to provide an answer in these circumstances.
Make sure you have a Plan B.

