Re: Funny Verizon blog post; lies about Netflix
PJL
Master - Level 3

@db909 wrote:

@PJL wrote:

So in the mean time and given today's reality of how the Internet works,  I have to determine the immediate cause of my poor Netflix streaming performance and in this case point the finger at Netflix, not Verizon.  Netflix was innovative in their service model to begin with, but then came up against the realities of how the Internet works from a business standpoint. 



You can "determine" whatever you wish.

I get  better Netflix on a vacation home that is 15/2 on a local cable company than I do on FIOS at 50/25.

Frankly after Comcast made their network changes last January, the only people really complaining about Netflix performance are FIOS customers, so call it a Netflix issue if it makes you feel better. 

This really is a FIOS issue and for whatever reason they seem incapable of "upgrading" their network in the same rapid manner that Comcast did.  Once FIOS does, people will stop complaining.

If you don't believe me check out this long thread on DSLR:

http://www.dslreports.com/forum/r28902007-Netflix-unable-to-stream-HD-in-Northeast-~start=300

The two most recent posts:

"The interesting thing (to me) in the graphic that Verizon published recently about the network utilization was just how high their peak usage rates were in their internal network. Those are pretty darned high numbers. Pretty close to the edge of failing.

If Verizon did fix their peering issues they may find that their internal network can't handle the Netflix traffic. Even without Netflix, natural bandwidth utilization increases will shortly overcome what they've got.

I hope they're upgrading their internal network like mad."

"Just tried my weekly random test-
Could barely pull 720p in the middle of the day.

Accessing via L3 in Atlanta."


You get "better" peformance at your vacation home because the peering points for that cable company apparently are robust enough to preclude saturation based on the peering agreements between your cable company and Netflix's content provider ISP, or your cable company has open connect.  (You did not indicate the cable company's name so I couldn't deduce which is the case.)

I am part of the thread on DLS Reports, so I am well aware of it.  By the way the peak-loading rates assertion in the DSL Reports (in the quote you include :natural bandwidth utilization increases will shortly overcome what they've got") is open debate.  The highest rate (65 percent) is not that bad, and Verizon likely monitors metrics like this and rapidly adjusts their network (routing changes to other paths, adding equipment) to preclude saturation over any one link just like any good tier-one ISP, or (from my experience) a well-run corporate internal network.

My determination (conclusion) is made based on testing and my IT networking experience, data observed, and Occam's razor.  Thanks for allowing that I can determine anything I wish.  Just curious, what (more specifically) are your assertions based on other than those included in your post, if any?

Re: Funny Verizon blog post; lies about Netflix
db909
Contributor - Level 3

@PJL wrote:

My determination (conclusion) is made based on testing and my IT networking experience, data observed, and Occam's razor.  Thanks for allowing that I can determine anything I wish.  Just curious, what (more specifically) are your assertions based on other than those included in your post, if any?


I work in IT networking and have for 20+ years.  {Keep your posts courteous} f you think you can diagnose the issues in the FIOS network.  You could only do so as an employee with access to it or are able to hack into it.


I base my position on two things.  Basic logic and articles like this:

"Verizon is deliberately constraining capacity from network providers like Level 3 who were chosen by Netflix to deliver video content requested by Verizon’s own paying broadband consumers."

"Above, I confirmed exactly the same thing for the Level 3 network. So in fact, we could fix this congestion in about five minutes simply by connecting up more 10Gbps ports on those routers. Simple. Something we’ve been asking Verizon to do for many, many months, and something other providers regularly do in similar circumstances. But Verizon has refused. So Verizon, not Level 3 or Netflix, causes the congestion"

http://blog.level3.com/global-connectivity/verizons-accidental-mea-culpa

Deja Vu, same thing happened on Comcast last year.  That too was all over DSLR.   You are in denial just like the "it's a Netflix problem" proponents were in the DLSR Comcast forum. They just vanished after Comcast fixed the issue on their end after getting Netflix to agree to pay them. 

And today's update for DSLR:

"I was in the caribbean recently with my family and we brought with us our Roku 3. Netflix streamed in beautiful HD the whole time using a 20 megabit cable connection down there.

Came back to the US using my 50 megabit FIOS connection and Netflix reverts back to streaming poorly "

0 Likes
Re: Funny Verizon blog post; lies about Netflix
RDJim
Contributor - Level 1
0 Likes
Re: Funny Verizon blog post; lies about Netflix
db909
Contributor - Level 3

@PJL wrote:

My determination (conclusion) is made based on testing and my IT networking experience, data observed, and Occam's razor.  Thanks for allowing that I can determine anything I wish.  Just curious, what (more specifically) are your assertions based on other than those included in your post, if any?


Another article:

"When Netflix accelerated the shift from third-party CDNs like Akamai to its own CDN in mid-2013, the links became saturated. Level 3 struck a deal with Comcast, the terms of which were undisclosed. Cogent refused to pay, and Comcast in turn refused to upgrade peering connections. Netflix videos began to stutter, and problems arose for other services that traveled over the same pipes."

Netflix is still angry that it had to pay Comcast (and Verizon), and the Federal Communications Commission is scrutinizing the deals. Comcast customers have good reason to be mad that Netflix performance suffered for months while the two companies argued over money, but at least performance has gotten better since the deal was struck.

Verizon customers aren’t so lucky. Even though Netflix agreed to pay Verizon for a direct path to its network, Verizon has been much slower than Comcast in rolling out the necessary infrastructure.

"The FCC is examining the paid peering deals Netflix signed with Comcast and Verizon, but for now the FCC’s net neutrality proposal would not regulate interconnection—a fact that pleases Comcast."

http://arstechnica.com/information-technology/2014/07/how-comcast-became-a-powerful-and-controversia...

0 Likes
Re: Funny Verizon blog post; lies about Netflix
monsterlab
Enthusiast - Level 2

What do the Verizon apologists have to say about this article?:

http://blog.level3.com/global-connectivity/verizons-accidental-mea-culpa/

0 Likes
Re: Funny Verizon blog post; lies about Netflix
PJL
Master - Level 3

@monsterlab wrote:

What do the Verizon apologists have to say about this article?:

http://blog.level3.com/global-connectivity/verizons-accidental-mea-culpa/


Although I'm not a "Verizon apologist" I see the L3 blog post as more PR finger pointing in Verizon's direction.  I'll explain this observation.

The post includes no hard data indicating the cause of the peering issue both parties acknowledge (although they might not use the term "peering" much in their PR material).   The blog only states that "So in fact, we could fix this congestion in about five minutes simply by connecting up more 10Gbps ports on those routers."  Well that may be true, but what about the peering arrrangement contract between L3 and Verizon?  That peering arrangement contract likely states traffic levels to and from Verizon and L3 that are within the terms of the contract, from which the number of port connections could be determined (although the actual number of connections at what rate might be in the contract).  And it's likely that the traffic from L3 to Verizon is greater than from Verizon to L3.  The levels dictating how many "10Gbps ports on those routers" is based on that traffic. 

Now if the peering agreement contract includes a traffic level to Verizon that can't be met by the existing port connections (either by inference from the contractually-specified traffic levels or by explicit numbers of port connections at specified speeds), then by all means, Verizon is at fault and should "connect the ports."  But if the L3 to Verizon traffic loads exceed the capacity in the peering agreement contract that is fulfilled by the existing number of port connections (either contractually specified or inferred by traffic levels), then Verizon has the legal right not to "connect the ports" to accept increased traffic. 

So which makes more sense?  (1) Verizon is violating a written contract by not adding the ports; or (2) L3 is complaining that Verizon isn't accepting traffic above the contractually-specified traffic levels supported by the contractually-required connecting infrastructure and wants Verizon to add more connections anyway?  Option 1 would allow L3 to file a lawsuit for violation of the peering agreement.  Option 2 allows L3 to issue PR statements like the blog entry to "shame" Verizon into doing something they're not required by contract to do without need for modifying a contract (which likely would cost L3 more since Verizon has no motivation to do it for free other than PR).

  • Option 1 is based on contractual law.  It would require proving in a court of law L3's assertions by providing actual evidence (data). It's also expensive and time consuming, and would require both parties to reveal data that is either proprietary or confidential per the peering agreement contract.  But if L3 is correct, then Verizon would be forced to connect the ports.  I don't recall reading about a lawsuit from L3, but I'm waiting if option 1 is the case.
  • Option 2 is based on PR spin with the hope that public pressure will get Verizon to add the port connections anyway even though they don't have to.  It's easy to implement.  It costs no money and is quick, although the results may not force Verizon to do anything but issue counter-PR statements.  (And I'm tired of reading the competing PR pronouncements from Verizon, Netflix, L3 and others.) This option also solves a future issue for L3:  that this 3rd party networking service might lose customers like Netflix because Netflix simply wants to solve their streaming issues as easily as possible and will bypasss companies like L3 and connect directly to Verizon, Comcast, etc. (It just occured to me that that's what Netflix is doing -- cutting out the middleman.)

Now don't get me wrong -- I just want this issue solved, even if option 1 was indeed the case.  But I think the probability that option 2 is the case is higher than that of option 1.  Option1 it  just makes more sense to me.  It's the simpler conclusion for me to reach.  My gut tells me that L3 is using option 2 because option 1 is the case and they're concerned about that "cutting out the middleman" thing. 

I could be wrong.  But I'm waiting for a lawsuit to be filed against Verizon by L3 if I am.  I just want this solved.  Oh, a solution is in work.  Netflix is already "cutting out the middleman" with Comcast and Verizon (in progress).  It's not moving at a quick enough pace to make me happy because my Los Angeles area streams are still not reliably SuperHD, although I get no buffering (the stream just goes down instead).  The East Cost issues apparently are more severe than ours and direct connections are already implemented for some East Coast areas, and spreading further as we speak. 

Re: Funny Verizon blog post; lies about Netflix
monsterlab
Enthusiast - Level 2

I am on the east coast, and Netflix is all but useless during peak times. Common sense dictates that every other method that I use (tethering LTE from my phone, borrowing a wifi connection from my neighbor's brighthouse connection, and VPN) all yield excellent results. I can stream video in glorious Super HD to my Roku and Xbox One in all of these scenarios. It is only when I use my 75/35 FiOS connection that I have problems. How can I, as a consumer, not blame Verizon for this issue?

0 Likes
Re: Funny Verizon blog post; lies about Netflix
PJL
Master - Level 3

@monsterlab wrote:

I am on the east coast, and Netflix is all but useless during peak times. Common sense dictates that every other method that I use (tethering LTE from my phone, borrowing a wifi connection from my neighbor's brighthouse connection, and VPN) all yield excellent results. I can stream video in glorious Super HD to my Roku and Xbox One in all of these scenarios. It is only when I use my 75/35 FiOS connection that I have problems. How can I, as a consumer, not blame Verizon for this issue?


As a consumer we can blame Verizon because we are Verizon's customers, but actual technical blame may be someone else.  Those of us with Verizon are experiencing the results of the wars between the large Tier 1 ISP and content providers and the content delivery ISPs they use as middlemen.  Comcast and AT&T customers are seeing the same thing.  Smaller ISPs (like brighthouse) probably are impacted as much in this war, or have jumped onto Netflix Open Connect.  So their customers don't see the same peripheral war damage.  Netflix is getting around the war by connecting directly to the large ISP (Comcast and Verizon in progress) and eliminating the middleman where the wars rage.

(Just curious:  is your LTE through T-Mobile or someone else?)

Re: Funny Verizon blog post; lies about Netflix
monsterlab
Enthusiast - Level 2

Believe it or not, my LTE is through Verizon wireless. The irony, right? Brighthouse isn't exactly small. They are basically a rebranding of Time-Warner. They serve all of Tampa and St. Petersburg, and are Verizon's biggest (and only) competitor here. If I had to guess, their customer base is pretty equal to Verizon's, maybe even larger.

0 Likes
Re: Funny Verizon blog post; lies about Netflix
PJL
Master - Level 3

@monsterlab wrote:

Believe it or not, my LTE is through Verizon wireless. The irony, right? Brighthouse isn't exactly small. They are basically a rebranding of Time-Warner. They serve all of Tampa and St. Petersburg, and are Verizon's biggest (and only) competitor here. If I had to guess, their customer base is pretty equal to Verizon's, maybe even larger.


Brighthouse surely isn't small, but relative to Comcast and probably Verizon it is.  My bad choice of words.  They appear to have about 2.1M customers.  I couldn't find current Verizon customer numbers, but I think it's much more. 

Maybe the reason you're seeing such good WiFI tethered through LTE is that the LTE Netflix traffic is usually to a phone (lower resolution, lower stream rate), not in the mode you're using?  So the peering points aren't saturating like the FiOS-connected points are (where everyone is trying to get the high streaming rates)?  I'm not sure, but it may be related.  People using T-Mobile tethered WiFi also have good Netflix performance. 

I couldn't find out if Brighthouse is on Netflix Open Connect.  If they're not, then their peering with the Netflix content provider networks is obviously more generous than Verizon if you're getting those good rates on your neighbor's Brighthouse service.