Routing Issues
jpstone1022
Enthusiast - Level 1

Hello,

I live in northern VA. I connect to services via TCP to two servers: one in Ashburn, VA (literally a 5 minute drive to the server), and one in DC (abou ta 30 minute drive to the server). About 30% of the time, Verizon FiOS decides to route me to California in order to resolve one or both of the servers. This creates extremely high latency (80-120ms) and makes services unusable. Releasing/renewing DHCP on my home router does not help.

As an example, I took a Windows tracert to the server in DC that I'm currently pinging 81ms to. Here is the output (I highlighted the evidence that I'm going all the way to CA):

Tracing route to 50.97.59.34-static.reverse.softlayer.com [50.97.59.34]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 4 ms 4 ms 4 ms L100.WASHDC-VFTTP-153.verizon-gni.net [108.28.24
.1]
3 26 ms 10 ms 11 ms G0-3-4-1.WASHDC-LCR-21.verizon-gni.net [130.81.1
05.178]
4 16 ms 15 ms 24 ms so-12-1-0-0.RES-BB-RTR1.verizon-gni.net [130.81.
51.230]
6 99 ms 72 ms 71 ms 0.ae4.XL3.SJC7.ALTER.NET [140.222.226.203]
7 68 ms 73 ms 74 ms TenGigE0-6-1-0.GW3.SJC7.ALTER.NET [152.63.50.26]

8 88 ms 75 ms 87 ms be-209-pe03.11greatoaks.ca.ibone.comcast.net [23
.30.206.93]
9 70 ms 70 ms 71 ms 75.149.228.214
10 74 ms 75 ms 73 ms ae0.bbr02.cs01.den01.networklayer.com [173.192.1
8.149]
11 79 ms 76 ms 75 ms ae0.bbr02.eq01.chi01.networklayer.com [173.192.1
8.130]
12 74 ms 74 ms 73 ms ae0.bbr02.eq01.wdc02.networklayer.com [173.192.1
8.154]
13 75 ms 74 ms 74 ms ae1.dar01.sr01.wdc01.networklayer.com [173.192.1
8.193]
14 75 ms 73 ms 74 ms po1.fcr03.sr02.wdc01.networklayer.com [208.43.11
8.157]
15 75 ms 74 ms 73 ms 50.97.59.34-static.reverse.softlayer.com [50.97.
59.34]

Trace complete.

How do I get this fixed? Can Verizon create static routes for me, since I use these services all the time? I never got these routing issues when I was with Comcast. It only started happening when I switched to Verizon FiOS. Latency of this magnitude--due to poor routing decisions by Verizon's routers--is unacceptable and makes no sense.

Thanks,

Justin

0 Likes
Re: Routing Issues
eljefe2
Master - Level 1

Yours is a fairly common problem reported here on the forum.   There hasn't been an official solution offered by Verizon, but what you might try is selecting a different DNS server to see if that improves your routing any.

You can use a free utility, DNSBench, to identify the best performing server(s) at your location:

https://www.grc.com/dns/benchmark.htm

Some users have reported good luck using Google's DNS...8.8.8.8 and/or 8.8.4.4

Good luck.

0 Likes
Re: Routing Issues
jpstone1022
Enthusiast - Level 1

Hi eljefe,

Unless I'm missing something, DNS is irrelevant to this issue since I'm connecting directly to an IP address.

The issue is with Verizon's routers. Either a) they've configured their routing protocols poorly or b) they're doing some kind of load balancing that sends my packets off to another router dedicated to routing traffic as far away as possible before getting to its destination. 😄

All it would take to fix this is for Verizon to dedicate some resources to get their routing in order. It appears Comcast has done this, as I never experienced it with them.

In all seriousness, if this continues, I will cut my Verizon service and go back to Comcast. Routing me to CA to get to a server 5 miles away is unacceptable in any context. It's just extremely poor design.

0 Likes
Re: Routing Issues
eljefe2
Master - Level 1

What you say makes sense, jpstone.  I was thinking of connecting to a domain name.  However, I just ran a couple of experiements tracing the route to a specific IP address with different DNS servers in place.  I got a different routing with the different DNS servers.

That may just be a conincidence but it might be worth your giving it a try.

As for giving up on Verizon and going back to Comcast, I can't blame you if you get better service from the cable provider.  Here, for whatever reasons, my experience with FiOS has been much better than it was prviously with our local cable company.

Good luck.  I hope you find a solution.

0 Likes
Re: Routing Issues
jpstone1022
Enthusiast - Level 1

I tried different DNS servers just for giggles. Same issue.

0 Likes
Re: Routing Issues
eljefe2
Master - Level 1

Yeah, I was going to post an edit that I tried the IP address you need with different DNS servers and I also had the same results. Smiley Frustrated

Maybe a VPN would help?   Or maybe Comcast is indeed the answer.

Sorry.  I'll hope that another user here chimes in with a better solution that works.

0 Likes
Re: Routing Issues
jpstone1022
Enthusiast - Level 1

In my humble opinion, the answer would be for Verizon to fix these routing issues. Especially if this is a common problem. Why are they content to unintelligently route their users' traffic and create high latency? This should be a high priority for them. Especially if their competitor has it right.

0 Likes
Re: Routing Issues
CRobGauth
Community Leader
Community Leader

Your initial complaint is high latency.

What level of latency are you expecting?

And Verizon doesn't have direct control over routing for specific address within their network.

It is possible that a closer gateway was down or is congested.

Or that the server you are hitting has changed providers and they are advertising that network out of Ca to Verizon.

0 Likes
Re: Routing Issues
jpstone1022
Enthusiast - Level 1

@CRobGauth wrote:

Your initial complaint is high latency.


Yes, high latency is the problem. In theory, you could route me to CA to get to a server on the east coast that's 5 minutes away from where I live if you can somehow manage to keep the latency to below 20ms. But this is reality, and that is impossible. So the real issue is that Verizon is sending me to California to get to a server 20 minutes away in DC.


@CRobGauth wrote:

What level of latency are you expecting?


I said it in my original post: 80-120ms.


@CRobGauth wrote:

And Verizon doesn't have direct control over routing for specific address within their network.


This is absolutely false. For example, Verizon could institute static routes if they wanted. At the very least, they could identify what it is in their routing protocols that causes traffic destined for a server 10 miles away to be sent on a 6,000 mile journey. If this issue happened in a closed network at a fortune 500 company, it would be considered unacceptable and the routing would be fixed.


@CRobGauth wrote:

It is possible that a closer gateway was down or is congested.


Then they need to fix their topology. If it was down, they should have backups that keep my packet journey within a reasonable milage, rather than turning 10 miles into 6,000. If it's conjested, perhaps I'd get less latency going through the conjested router than traversing 6,000 miles?!


@CRobGauth wrote:

Or that the server you are hitting has changed providers and they are advertising that network out of Ca to Verizon.


As I mentioned in my original post, the issue is interminttent and happens only 30% of the time. If it was a changed provdier/advertising issue, then it would happen 100% of the time.

Also, they have had the same provider since 2011. Just for kicks, I went to someone's house that has Comcast and the latency was 17ms, and the traceroute was much more sane. After I returned home, the issue remained with Verizon. 

0 Likes
Re: Routing Issues
CRobGauth
Community Leader
Community Leader

Couple of comments:

I asked what you expected for latency.

You said you stated it in your post. But your post said you measured 80-120. Then said Comcast was 17. So you are really expecting in the 20 range I would assume.

Changing routing is not as easy as you report. Putting statics in a network the size of Verizon's is impossible. The number of nodes that would need to be updated would be huge. I work on a private corporate network. We would never correct a problem like this with statics. The problem would be found with the routing and corrected appropriately. Problem is, Verizon probably has little interest in the correcting a routing issue for just one network. Espeically for a residential user (unfortunately that is the reality. Those of us that are residential, get no SLA and minimal support when it comes to correcting this type of issue).

0 Likes