![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
When redundancy isn't
Sep. 20th, 2018 12:33 amI'm reminded of a joking phrase used in WAN operations: backhoe fade
That's when you lose your connection because some idiot dug up the cable with a backhoe.
A rather infamous incident shut down the Internet in New England back in the late 70s.
Seems that while the customer had specified separate routing for the pair of T-1(?) lines that carried the Internet, the provider had routed the connections via separate cables... in the *same* trench.
Needless to say the customer had words with the provider. And the provider revised their rules so the "separate routing" meant different cables *routed* differently so that one accident couldn't take both out...
A manufacturing place I used to work at got separate power feeds from two different power companies because they had processes that didn't take well to sudden power loss.
One power line came in from the north, one from the south. Only single point of failure was the company's substation that they both connected to.
And they had a *large* room full of batteries to enable shutting down those critical processes gracefully.
Sadly, we still lost power a few times in the dozen years I worked there.
That's when you lose your connection because some idiot dug up the cable with a backhoe.
A rather infamous incident shut down the Internet in New England back in the late 70s.
Seems that while the customer had specified separate routing for the pair of T-1(?) lines that carried the Internet, the provider had routed the connections via separate cables... in the *same* trench.
Needless to say the customer had words with the provider. And the provider revised their rules so the "separate routing" meant different cables *routed* differently so that one accident couldn't take both out...
A manufacturing place I used to work at got separate power feeds from two different power companies because they had processes that didn't take well to sudden power loss.
One power line came in from the north, one from the south. Only single point of failure was the company's substation that they both connected to.
And they had a *large* room full of batteries to enable shutting down those critical processes gracefully.
Sadly, we still lost power a few times in the dozen years I worked there.
