Accessibility Resource Center Skip to main content
Have a phone you love? Get up to $500 when you switch and bring your phone.

Packet loss and lag to AWS

Reply
test42
Copper Contributor
Copper Contributor
Posts: 8
Registered: ‎04-09-2015

Packet loss and lag to AWS

Message 1 of 16
(4,698 Views)
Ever since I signed up for 70/70 internet I've noticed slow connection speeds specifically to Amazon's cloud services. Speedtest.net shows that everything is fine, but I know that speedtest.net merely tests PART of your internet speed. Is verizon still throttling (aka refusing to pay for upgrades) to amazon web services? If so, why am I paying for 70/70 internet? My sprint phone on 3G has a better connection to my teamspeak server.
15 REPLIES 15
Smith6612
MVP MVP
MVP
Posts: 7,583
Registered: ‎12-15-2010

Re: Packet loss and lag to AWS

Message 2 of 16
(4,637 Views)

Many of us in the Northeast have been seeing issues with congestion to and from AWS. My DSLReports Line monitor will show this. It appears to be a peering problem between a transit provider used to connect Verizon and Amazon together, and Verizon. In my case it appears to be Qwest, but I need a reverse traceroute to fully determine that.

 

Here's my DSLReports line monitor as an exmaple.

 

Capture.PNG

 

The top graph shows a monitor from an East Coast EC2 instance. The bottom one is from a West Coast instance. Blue denotes packet loss - in my case this loss in the past day was due to my IP address changing (I was doing some work on my home network). The East coast instance consistently sees latency at night. West Coast EC2 sees less latency.

 

Just congestion. Verizon needs to look into their peering with others and fix it up. Has nothing to do with FiOS itself, as I'm on DSL.

mtgdam
Contributor
Contributor
Posts: 1
Registered: ‎04-12-2015

Re: Packet loss and lag to AWS

Message 3 of 16
(4,609 Views)

We have services hosted in AWS N. Virginia. Right now, anywhere from 10-25% packet loss and average 120ms rtt consistantly at multiple clients on FiOS Business Connections. On February 19th, the issue was so bad that services were unreachable from 9:30am to 2:15pm EST.

 

I notice the same where evenings tend to be worse than normal business hours. Nevertheless, completely unexceptable and quite the headache.

seitzbg
Nickel Contributor
Nickel Contributor
Posts: 37
Registered: ‎01-05-2015

Re: Packet loss and lag to AWS

Message 4 of 16
(4,601 Views)

I too have noticed this in the form of dropbox sucking.  100kb/sec download when trying to get my friend's video uploads.

I pay for 500/500 and Verizon's peering/routes to major players has started to *SUCK* in the past few months. I am considering dropping my $300/mo elsewhere.  Does Verizon care? Doesn't seem so.  

Namronorman
Bronze Contributor II
Bronze Contributor II
Posts: 136
Registered: ‎07-25-2011

Re: Packet loss and lag to AWS

Message 5 of 16
(4,555 Views)

Hey Smith,

 

I am Norm on DSLR. I see the same thing with my line monitor and actually have an AWS account with some credits on it from trying it out then immediately giving up because of what people are complaining about here. When I tried it out, it was qwest to and fro.

 

I've tried a lot of cloud providers recently for some projects I'm working on and I haven't found a single one that works reliably with Verizon for an extended period of time. Same thing with CDN providers as well.

 

- Norm

Rodder152
Contributor
Contributor
Posts: 1
Registered: ‎04-14-2015

Re: Packet loss and lag to AWS

Message 6 of 16
(4,522 Views)

Experienecing the same thing. I visit three Teamspeak3 Servers. The packet loss to two teamspeak servers that I visit is almost ALWAYS sitting at 10-20% packet loss. It's hosted by Typefrag with Amazon AWS. The third that I connect to is light-speed and doesn't use Amazon AWS. 

 

This is 100% a problem between something Fios and something Amazon related. Not really sure where the problem exists but there definatly a problem somewhere.

seitzbg
Nickel Contributor
Nickel Contributor
Posts: 37
Registered: ‎01-05-2015

Re: Packet loss and lag to AWS

Message 7 of 16
(4,492 Views)

Yeah this is getting silly.  How Verizon could have bad peering to *AWS* one of the largest cloud providers (or the) in the world is beyond words.  I wish we had a network engineering contact or someone other than script reading level1/2 support.  I want to know if Verizon will be fixing their peering and peering practices or if I'm switching to Comcast.  Can't pay for 500/500 and get 1Mbps.

CRobGauth
Platinum Contributor III Platinum Contributor III
Platinum Contributor III
Posts: 7,978
Registered: ‎11-04-2008

Re: Packet loss and lag to AWS

Message 8 of 16
(4,488 Views)

Verizon could be doing this for the same reason they played hardball with Netflix.

The content providers want the ISPs to pay to continuously upgrade their peering and internal backbone to provde better service to the content providers customers. But the ISP doesn't make any extra money.

So they can either get the content provider to help pay for the extra peering costs, raise rates to the ISP customers or let it sit.

I beleive Verizon (purely a guess based on previous actions) is doing the third.

And Verizon is not the only ISP to do this.
Comcast did the same when it came to Netflix.

 

Same game as between cable providers and TV content providers.


If a forum member gives an answer you like, give them the Kudos they deserve. If a member gives you the answer to your question, mark the answer as Accepted Solution so others can see the solution to the problem.
test42
Copper Contributor
Copper Contributor
Posts: 8
Registered: ‎04-09-2015

Re: Packet loss and lag to AWS

Message 9 of 16
(4,404 Views)

I'm fairly certain it is a peering issue. It was the one that was partly at the root of the netflix issue. Basically verizon is complaining that a particular peering connection is getting more data in one direction than another (verizon is receiving more than sending), and that L3/Cogent/whoever should pay for sending so much more data onto verizons network than they're getting back.

 

At least that was their argument the last I checked.

 

either way it makes no sense since the other network is clearly hosting content providers while verizon is hosting consumers. Of course verizon is going to be receiving more than its sending.

 

Hopefully the complaints to the FCC by Cogent/L3 pay off and verizon is finally forced to upgrade its own hardware.

CRobGauth
Platinum Contributor III Platinum Contributor III
Platinum Contributor III
Posts: 7,978
Registered: ‎11-04-2008

Re: Packet loss and lag to AWS

Message 10 of 16
(4,401 Views)

So when Verizon has to outlay more cash, they need to make it up.
That means our rates will go up.


If a forum member gives an answer you like, give them the Kudos they deserve. If a member gives you the answer to your question, mark the answer as Accepted Solution so others can see the solution to the problem.
How-To Videos
 
The following videos were produced by users like you!
   
Videos are subject to the Verizon Fios Community Terms of Service and User Guidelines and contains content that is not created by Verizon.
Have a spare Fios-G1100?Learn how to bridge it into your network
Get Started


Covid19

Browse Categories
Categories:
Posts

Verizon Troubleshooters
Unable to find your answer here? Try searching Verizon Troubleshooters for more options.
Modal Dialogue Title