Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
JohnCF
Contributor - Level 1

Like many many many others, I woke this morning to a near total blackout of Verizon services.  After wasting 30 minutes trying to contact anyone to see if the issue was local or more widespread, I resorted to calling friends.  It was soon determined that except for a lucky few, all Verizon customers were cut off from the world.  

Fortunately, I still have a true landline, not a digital phone service routed through the Verizon Internet Service, so I was able to make calls.  My Verizon and Sprint cell phones were dead in the water.  With many thousands of people dropping their landlines altogether and relying on their cell for everything, it had to be a thought provoking morning about what they would do in a true emergency and stuck with unreliable equipment.  Verizon has already reduced the power they use for their cell towers to save money at the expense of service in customers homes making calls difficult unless you go outside like Oliver Douglas on Green Acres.  I live in a densely populated suburb of Washington DC and have to use a "Network Extender" to have reliable cellular service in my home, and I can see two cell towers when I climb up on my roof!  Now to have no cell service unless you drove 30 miles south of DC?  Wow!

A very few odd channels were working on my dvr/stb, most likey due to the fact that I have my DVR on a UPS battery so it won't get rebooted from a power glitch or failure.  Anyone who rebooted lost all channels of service, unable to check local broadcast channels, the weather channel, or any other important channel.  Funny enough, AMC, the network many services are dropping tomorrow because the AMC network suits thought they could charge ESPN rates for Mad Men and The Walking Dead and got slapped down, was working as was NBC Sports, MASN, and MLB.TV.  I did not see any scrolling messages about the outage, the efforts to remedy, the methods to contact someone for questions, or any emergency information.  All quiet from Verizon.

So, as a person who for a living supports multiple data networks and deals with redundant power and network services, backup support systems in a data center, and very familiar with terms like Business Continuity and Disaster Recovery, it was at first puzzling and then disappointing to face the issue we all had here in the National Capital Region of the United States of America.

Time to break up the all-in-one package I'm joined to with Verizon.  I give them thousands of dollars for services and expect a billion dollar corporation to do a better job than I do for my employer.  But this weekend is instead a wake up call that I've been wasting a lot of my money.  

Now, I will probably hear that Verizon is sorry about the brief interruption and is working hard to provide me with the very best service they can, but that's just the siren song of the Medusa.  Time to tie myself to the mast of the ship and sail on by and get some diversified services that can provide me with the ability to communicate and connect to my family when a real emergency occurs.

Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
dsfreybitz
Enthusiast - Level 1

Not ready yet to take up the pitchforks myself but thank you for the well-written post.

0 Likes
Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
HughB
Enthusiast - Level 2

Well said!.  It appears that "redundancy" and "backup power" are not part of Verizon's lexicon (along with "Customer Service").

0 Likes
Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
rubeel
Newbie

I didnt lose any service on friday night or saturday but come today, i just lost the TV, very interesting that Verizon will break more things before it comes back up.

Kinda **bleep** off, was looking forward to the Euro 2012 final, i guess i will be watching it on a computer. 

Thanx verizon. Truly a Network of the future.

0 Likes
Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
tgolds
Enthusiast - Level 2

I also had some outages but for the most part, Verizon did a great job getting the services back up and running. I was back up around 1pm, and my Internet never went down. Do you think Comcast or Dish did not have outages? We often always look at the negatice when it comes to service providers but I think I will not this time.

Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
JohnCF
Contributor - Level 1

I hear what you are saying and I agree that they seemed to work quickly considering the mess on their hands to get services back up.

Here's my main point:

For those who have gone the route of all cellular with no landlines or even digital home phone service, there was no way for them to call out to any help and there was no help from the Verizon services that were still working to help them figure out what to do other than join the tens of thousands of others on the road to find answers.

If you are going to become the default communication service for millions of people, you had better have the infrastructure and systems in place to ensure that those customers can contact emergency services during an event like Friday.

 I learned this weekend I cannot trust one company to ensure my ability to communicate and protect my family and property.

This is not a negative rant to bash a company. This is a lessons learned opportunity I was given and nobody was hurt this time.  I will learn from this and be better prepared for the next event.

0 Likes
Re: Massive Services Outage in Northern Virginia - Verizon Caught Unprepared
nicainnva
Enthusiast - Level 2

There is a later post inferring that "generators can only run for so long before they run out of fuel".  If I were in Verizon management I would have a serious talk with the business resiliency group leads about their future with the company.  While we can understand this was a powerful storm and that service interruptions can be expected closer to the client locations, as the "last mile" is more exposed, if indeed the network operation centers or TV head end, or key repeater facilities were allowed to run out of fuel and go completely offline, as some suggest, that would be totally unacceptable for one of the largest telecommunications companies in the US, and around the Nation's Capital.

To add insult to injury, the lack of transparency and communication to users of the network status is preposterous.  I was lucky to only have interruption in the TV services, which were restored toward mid-day Saturday.  Throughout the time my service did not work, the only assistance I was able to obtain was from my fellow Verizon customers.

Thanks to all of you for your mutual support and I hope everyone's service is restored very soon.