What can we do to actually fix Verizon's Netflix throttling?
bbinnard
Contributor - Level 3

Obviously there are lots of folks (including me, located in the Los Angeles area) who are furious about Verizon's unacceptably poor delivery of Netflix content.  There are either numerous pauses waiting for buffering or reduced resolution due to Netflix increasing compression ratios to reduce network traffic.  Neither of these are what we are paying for, and both are unacceptable.

This chart

http://ispspeedindex.netflix.com/results/usa/graph

clearly shows Verizon's steadily decreasing performance delivering Netflix content.  At this rate the delivery speed will drop to zero in a few more months.  Abd we are expected to continue paying for this?

So far, despite countless postings here about this issue, nothing has been done or appears to be planned to be done.  So what can we, the customers and consumers, do to cause some positive action?  

How about some sort of petition sent to the FCC ( but addressed to whom or what department?)  Or perhaps we flood the head of Verizon (who is that?) with emails or actual letters.  Or how about letters to the editors of major newspapers and/or wire services?  How about emails to all the popular PC & Internet oriented magazines & blogs?

My sense is that the only way to actually make something happen to fix this situation is to raise enough attention that Verizon management realizes it is in the company's best interest to stop stonewalling and instead actually do something.

To borrow a quote from Paddy Chayefsky: "I'm mad as hell and I'm not going to take it anymore!"

0 Likes
Re: What can we do to actually fix Verizon's Netflix throttling?
thetick1234
Specialist - Level 1

You can get your local cable company if they are higher on the Netflix buddies list. 

Re: What can we do to actually fix Verizon's Netflix throttling?
chomper87
Enthusiast - Level 3

I love how people tend to somehow avoid any blame on Netflix OR any other number of things that could be affecting performance. This is not a Net Neutrality, FCC, or a throttling issue. You haven't listed any of you're specs or you're FiOS package. How do we know that YOU aren't on the FiOS 3 Mb down / 1 Mb Up package? Then that would be YOUR problem. Do we know how many FiOS 3 / 1 subscribers Netflix includes in their stats? NO. How about how many that are on the FiOS 5 / 2 package. NO.

OTHER FACTORS besides BLAME VERIZON include:

Wired vs wireless

Cat5e vs MoCA

You're equipment including computer, OS, software, router.

Other traffic on you're equipment while attempting to stream Netflix

Squirrels. Yes the tiny animal that apparently likes to chew on fiber cables.

http://blog.level3.com/culture/what-we-learned-from-you-on-squirrels-and-fiber-cuts/

http://blog.level3.com/level-3-network/the-10-most-bizarre-and-annoying-causes-of-fiber-cuts/

Other than Squirrels, the other 9 items listed by Level 3.

The NETFLIX device you use - how many are there? A quick overview here: http://en.wikipedia.org/wiki/List_of_Netflix_compatible_devices

I sure bet they all perform exactly the same and perform at a super high performance. Oh whats that, no they don't perform the same. Some only do 720p. Some might not even do 720p. Some are only stereo instead of 5.1 audio. Some are HTML5, some are Silverlight, some have a native app.

What about if you have an XBOX. You have to pay you're ISP, Netflix and XBOX Live Gold. What a bunch of nonsense.

 

NETFLIX encoding and source material. How many ways can they encode a single video?

http://gigaom.com/2012/12/18/netflix-encoding/

I'm sure glad that sounds simple. It wouldn't have anything to do with the 900 fragmented Netflix devices out there, right?

 

Xboxes, iPads, connected TVs: Netflix streams to a lot of different devices. More than 900, to be precise. And many of them have different screen sizes, bitrate requirements and codec support. That’s why Netflix is doing a whole lot of encoding

What license to they have for to stream a video? 480p, 720p?

 

BTW there's 1000 Netflix Android variations alone.

To put device diversity in context, we see almost around 1000 different devices streaming Netflix on Android every day.

http://techblog.netflix.com/2012/03/testing-netflix-on-android.html

Netflix data exchange.

Netflix has Open Connect Peering and a Cache server (Open Connect Appliance). They pay an ISP ABC. They probably still pay for a another CDN XYZ. Super. Now what about that agreement between ISP ABC, CDN XYZ with Verizon.

 

Here's what their Open Connect letter looks like to Verizon: Dear Verizon, we offer for free, these services. Information on our Open Connect Peering and Open Connect Appliance.

Carbon Copy: ATT, CenturyLink

 

Verizon's responce: What. LOL? Huh? We aren't really in the BUSINESS of providing FREE services. You can pay us to put YOURE appliance in OUR datacenters, central offices, for colocation etc. You can also pay us for transit services or other data services.

 

Which ISPs benefit from Netflix Open Connect? Well certainly the smaller ones. Rather than Small ISP LLC pay for a 10 Gigabit connection to a Large ISP LIKE Verizon, they will end up saving money by using Netflix's Open Connect Appliance and potentially the Peering. So instead paying Verizon for that 10 Gigabit connection, they now only have to pay Verizon for a 5 Gigabit connection.

 

Perhaps you should send some messages to Netflix and point out some of Verizon's services that Netflix could PAY for

Verizon offers a number of readily available solutions for interconnecting providers who send significantly more traffic than they receive from Verizon’s networks. Solutions such as cloud, hosting, Partner Ports and others are designed specifically to provide a cost-effective means of delivering very large volumes of out-of-balance traffic. Other large streaming video providers (and/or network service providers carrying such one-way traffic) are already taking advantage of these solutions and seeing immediate benefits. These solutions are available today to Cogent, Netflix and any other content or network service provider with similar traffic profiles.

http://publicpolicy.verizon.com/blog/entry/unbalanced-peering-and-the-real-story-behind-the-verizon-...

http://www.terremark.com/services/it-infrastructure/

Having said all of that, it would be great for - us - the consumer of FiOS if Verizon used Open Connected, but as shown above, it's not going to happen because.

As for some of the news media on this topic really hit the fan when Netflix and Comcast had their agreement. Some of them are so full of nonsense. They have no idea who Verizon is outside of DSL - FiOS - Wireless, or who Level3 is.

Re: What can we do to actually fix Verizon's Netflix throttling?
bbinnard
Contributor - Level 3

thetick1234:

Yeah, I'm fortinate that in my area there is an alternative to FIOS - Cox Cable, which, surprisingly enough, is near the top of the Netflix speed chart.  However, switching to Cox does not do anything to address the basic problem and therefore seems like admitting defeat right out of the box.

chomper87:

I appreciate all the info you provided, but I think you are missing the point.  Granted, I should have said my FIOS plan is 35/35, more than enough bandwidth to handle anything Netflix throws out, and when watching Netflix I have only one device online - a hardwired PS3 which also has more than enough throughput for Netflix traffic.

The point I think you have missed is that although it is possible to list a whole host of reasons why Netflix performance at the TV set is poor, these are just excuses for providing poor service.  Netflix and FIOS promise full quality HD video with 5.1 sound.  That's what they sell, and that's what we pay for.  Providing less than that means we are not getting what we pay for, or what we expect.  

The Netflix problem is so widespread, and has been mentioned in so many places (other than FIOS), that it is clearly a ubiquitous one and must have some sort of common denominator.  Encoding is not the problem - the reason Netflix creates different encoders is to accommodate different screen sizes.  The encoding is done once, the results are stored on a Netflix server, and the stored file is transmitted to the end user when requested.  

Transmitted is the key word - the end user's experience is totally dependent on whether or not the transmitted data stream can keep up with the viewing device's data requirements.  Buffering delays happen when this is not possible, and this has nothing to do with how the source material is encoded.

OpenConnect is all about throughput, not anything else.  Granted, the problem of asymmetrical bandwidth traffic is not simple, but these problems are basically related to money and not technology.  Google's gigabit network shows what you can do if you really want to.

Verizon management simply has to decide whether or not it wants to provide the throughput it promised us.  My intent is to find a way to get them to do this.

Re: What can we do to actually fix Verizon's Netflix throttling?
VUser50
Contributor - Level 2

This may soon be a moot point. Verizon has indicated that they are talking with Netflix about a deal similar to Comcast's where Netflix will pay for a direct pipe to the network. (and while easily unrelated, I've personally seen some improvements.) Yay ISP victory!

While Verizon does have an incentive to charge Netflix high (what is high is debatable) peering/connection prices, at the end of the day unless they are super ridiculous Netflix can pay. It is more of an issue with new smaller players. In fact, Netflix may actually see a benefit to high prices since they set a barrier to entry for new streaming services.

If you still are concerned enough to write, you should write the chairmen whoose emails are on this page. Instead of writing that they should stop throttling, which is a kind of basic phrasing/ a simplistic reduction of the issue, you might want to say they should look into ways of regulating the peering and connection agreements set by the ISPs. Even people who may be sympathetic to Verizon in the Netflix dispute should see the fundamental issue with a monopoly (or often part of a duopoloy) being free to set the prices for streaming competitors. 

Re: What can we do to actually fix Verizon's Netflix throttling?
bbinnard
Contributor - Level 3

VUser50:

I saw that announcement about Verizon joining OpenConnect, but so far it is just an annouincement and the skeptic in me wonders if that's all it will ever be.  I guess time will tell.

Remember, Verizon was the company that got the Net Neutrality decision nullified.

In the meantime, here is an article that describes the problem quite nicely:

http://arstechnica.com/information-technology/2014/02/netflix-packets-being-dropped-every-day-becaus...

And here is what seems to be a simple/cheap/free way to work around the problem:

http://thevpn.guru/vpn-vs-smart-dns-proxy-unblock-location/

I think I might try the latter approach of specifying an alternate DNS server pair on my PS3.

But I wonder - what if there were a petition with 2,000 (or some other number) of names sent to the Verizon CEO, and what if that same petition were published in national newspapers and news websites?  Might that not create enough social pressure to force a change that actually worked?

0 Likes
Re: What can we do to actually fix Verizon's Netflix throttling?
SoNi67
Contributor - Level 1

Change the DNS setting in your router or in the computer/TV that you use for Netflix. Instead of Automatic (which brings Verizon DNS servers), use a service that "goes around" Verizon obvious blockage:

https://unlocator.com/

It is a free (for now, beta stage) service.

0 Likes
Re: What can we do to actually fix Verizon's Netflix throttling?
optivity
Contributor - Level 3

Nothing much other than to express our dissatisfaction with the only means Verizon cares about: cut service options or switch providers.

The Internet is rapidly devolving to an environment that is ruled by the political oligarchy that is bought and paid for by a few large corporations (e.g. Verizon, Comcast/Time Warner, etc.). With the demise of  "Net Neutrality," Verizon now dictates what services they will provide by determining throughput speeds across their network.

Verizon has intentionally harmed its subscriber base by:

1) Litigating and successfully having the courts to overturn the FCC's Net Neutrality rule.

2) Allowing their peer networks between CDNs (e.g. Cogent) to become so congested we can no longer receive HD video streams from Netflix.

Netflix provides a service that directly competes with Verizon's Red Box Instant and PPV offerings.

Verizon CEO Lowell McAdam pays its customers lip service by saying he will reach an agreement with Netflix "soon" (sic) however, given Verizon comprises about 6% of the CATV and 20% of the broadband markets, they are in no position to compel Netflix to adhere to Verizon's terms to restore HD video streaming w/FiOS.

Netflix has offered Verizon a "free" solution to mitigate the congestion between CDN's that Comcast recently agreed to implement while Verizon has pa$$ed on. What does this tell us regarding Verizon's desire to resolve the Netflix HD video streaming debacle?

"Netflix Open Connect Content Delivery Network

ISPs can directly connect their networks to Open Connect for free. ISPs can do this either by free peering with us at common Internet exchanges, or can save even more transit costs by putting our free storage appliances in or near their network."

Re: What can we do to actually fix Verizon's Netflix throttling?
SoNi67
Contributor - Level 1

@chomper87

We are not stupid, stop serving us the Verizon coolaide.

Easy test:

Every evening, my Netflix slowes down to 350-500kbps (I can measure that in IE using Silverlight). Quality is execrable, below SD.

Switching to a VPN provider (like unlocator.com, requires just a change in my DNS server address) brings the speed to 3000kbps INSTANTLY!

So "hiding" the endpoint from Verizon solves the "congestion". How is that not the Verizon's premeditated fault?

0 Likes
Re: What can we do to actually fix Verizon's Netflix throttling?
chomper87
Enthusiast - Level 3

No doubt, there's a widespread issue. As for the part about licensing and encoding - I was simply tring to state the obvious that if it's a 480p 1.25Mb encode for a Movie on a certain device, then there's realy no expectation that the streaming bitrate will be much higher. No, it won't impact buffering. But if I measure that network stream, I might only see a max of 2Mb.

Likewise if my device only supports 2 channel audio, well I'm not going to get 5.1 audio.

In you're setup you certainly should get the HD with 5.1.

Netflix and FiOS haven't promised anything together. You pay Verizon - they promise 35/35. You separately pay Netflix they promise HD with 5.1 (for you're PS3). There's no agreement between Verizon and Netflix. It might be unfair to you and the other consumers, but that's how it is.

My FiOS seems to be just fine to companies which actually have agreements with Verizon.

Including Level3:

http://i.imgur.com/H7x3jsi.png

Including CenturyLink:

http://i.imgur.com/iENOIjp.png

Once again, Verizon isn't being evil here or anything. Just like if I tried to download something from you're FiOS 35/35 connection. I will ONLY get a Max of 35 download even though I'm on the 75/35 package. Verizon can't control whether you want to pay for something more like the 150/65. That's up to you. Verizon isn't just going to give you a free upgrade so that I can download 75Mb from you're server. Verizon isn't going to give away free services for things which it normally charges for.

Google gigabit is not a large ISP. They are not a Network Service provider or a backbone provider or a Tier 1. Google is a Customer of companies like Level3 and Verizon.

traceroute to www.google.com (173.194.115.113), 30 hops max, 40 byte packets
 1  L100.TAMPFL-VFTTP-135.verizon-gni.net (96.254.188.1)  1.377 ms  1.118 ms  3.265 ms
 2  G0-9-2-4.TAMPFL-LCR-22.verizon-gni.net (130.81.140.62)  4.481 ms  4.901 ms  4.100 ms
 3  ae2-0.TPA01-BB-RTR2.verizon-gni.net (130.81.199.82)  7.699 ms  31.255 ms  72.756 ms
 4  0.ae4.XL2.MIA19.ALTER.NET (152.63.8.117)  10.548 ms  10.831 ms  9.965 ms
 5  0.xe-9-1-1.GW1.MIA19.ALTER.NET (152.63.84.218)  10.254 ms  10.413 ms  10.086 ms
 6  google-gw.customer.alter.net (63.65.188.38)  10.007 ms  11.720 ms *

 google-gw.customer.alter.net <-- Alter.net is Verizon.


So OpenConnect makes sense for Google. They can save money. CUT COSTS to Level3 and CUT COSTS to Verizon by doing the free OpenConnect Appliance and OpenConnect Peering.

There's no technical reason why Verizon isn't on the OpenConnect. Once again, have you also complained to Netflix?

As for DNS changs and VPN.

Using a VPN takes your traffic away from the shortest path by distance, but may be faster in cases when the path would otherwise be congested. Networks generally aren't intelligent enough to automatically route around congestion.

http://arstechnica.com/information-technology/2014/02/netflix-slow-on-verizon-or-comcast-a-vpn-might...

Okay, you do some manual rerouting. Doesn't necessarily prove or disprove anything other than this other route perfoms better right now.

Arstechnica has some info here:
http://arstechnica.com/information-technology/2013/07/why-youtube-buffers-the-secret-deals-that-make...

It's can be difficult to really pinpoint a reason.

0 Likes