Accessibility Resource Center Skip to main content
Have a phone you love? Get up to $500 when you bring your phone.
end of navigation menu

DNS issues in SoCal

Reply
Porch
Contributor
Contributor
Posts: 7
Registered: ‎01-18-2012

Re: DNS issues in SoCal

Message 81 of 188
(2,236 Views)

I have clients all over the area that can't get to some sites. craigslist.org is often not working. Some can't get to Epson's FTP site for driver downloads. And I can't get to one of my email provider.  Most are on DSL, but some are on Fios. 

 

All work fine on my T-Mobile phone. 

 

 

 

 

mrballcb
Copper Contributor
Copper Contributor
Posts: 22
Registered: ‎01-27-2010

Re: DNS issues in SoCal

Message 82 of 188
(2,214 Views)

@JamesLaszko wrote:

To add insult to injury, I can now not access forums.verizon.com through my FIOS connection either:


Page times out

 

Oh, wait, it must be the fault of the web site that we're going to.


I was troubleshooting last week why I could not get to a fastly.net CDN node (the west coast node, the Dallas and Virginia nodes worked just fine if I hardcoded those IP's into my /etc/hosts file).  I was in IRC with the fastly.net admins and they could see my packets getting to the west coast node (both icmp and tcp), and they could see the return packets getting sent back to me, but those packets never got back to me.  It was the same issue in that I could not reach the
One opinion I got was that Verizon needs to go and start resetting some line cards, it sounds like they have some corrupted src+dst flow tables somewhere.  Another is a replacement box with a misconfiguration that's blocking certain packets.  Either way, it's been going on since about Mon Jan 8.
The only solution (not a solution at all, just a hack around the problem) is rebooting your fios gateway and trying to get a different IP.  You will likely need to call tech support and get them to invalidate your lease to ensure you get a new IP.
...Todd
frnkblk
Nickel Contributor
Nickel Contributor
Posts: 41
Registered: ‎01-13-2012

Re: DNS issues in SoCal

Message 83 of 188
(2,211 Views)

Perhaps the change in IP address results in a different hash, and so the return traffic goes over a different port in the LAG (or whatever load-balancing/port aggregation scheme they have in place.)

mrballcb
Copper Contributor
Copper Contributor
Posts: 22
Registered: ‎01-27-2010

Re: DNS issues in SoCal

Message 84 of 188
(2,209 Views)

@frnkblk wrote:

Perhaps the change in IP address results in a different hash, and so the return traffic goes over a different port in the LAG (or whatever load-balancing/port aggregation scheme they have in place.)


Exactly what we were thinking.

JamesLaszko
Nickel Contributor
Nickel Contributor
Posts: 26
Registered: ‎01-13-2012

Re: DNS issues in SoCal

Message 85 of 188
(2,133 Views)

The hard part of things is that VZ uses an internal MPLS network to distribute traffic to the "best" place for off-loading.  Even geographically close sites may end up routing through completely geographically distributed sites, based on least cost or best route available routing.  I'm at a loss that VZ NOC staff are not able to monitor issues like we have been seeing over the last week.....

 

We've been receiving a lot of support calls for one of our customers, Altura Credit Union.  They've had a ton of customers calling with connectivity issues - go figure, they are in the Inland Empire.  If there are any Altura customers watching this thread - Altura has been beating the crap out of VZ and us to make sure these issues go away - problem is that it's soley under the control of the VZ people.

 

Big pain....

 

 

James

frnkblk
Nickel Contributor
Nickel Contributor
Posts: 41
Registered: ‎01-13-2012

Re: DNS issues in SoCal

Message 86 of 188
(2,129 Views)

I'd hope that Verizon is working night and day on this issue, but based on the lack of communication and success, I'm skeptical.

loy
Contributor
Contributor
Posts: 1
Registered: ‎01-19-2012

Re: DNS issues in SoCal

Message 87 of 188
(2,106 Views)

after reading the messages, i decided to use chat live to check the status (i couldn't connect to steampowered and my wife couldn't connect to aol unless she goes through help.aol.com).  anyway, miguel, the tech guy, had me reset the router (i did this at least 3 times for the past week and a half) and after rebooting, i could now connect to these sites...so it might be worthwhile for those still having problems to use live chat...it worked for me...hopefully, this works for you guys too.

 

PS.  he mentioned something about a a break on the dhcp...but it solved my problem.....

mrballcb
Copper Contributor
Copper Contributor
Posts: 22
Registered: ‎01-27-2010

Re: DNS issues in SoCal

Message 88 of 188
(2,096 Views)

@loy wrote:

after reading the messages, i decided to use chat live to check the status (i couldn't connect to steampowered and my wife couldn't connect to aol unless she goes through help.aol.com).  anyway, miguel, the tech guy, had me reset the router (i did this at least 3 times for the past week and a half) and after rebooting, i could now connect to these sites...so it might be worthwhile for those still having problems to use live chat...it worked for me...hopefully, this works for you guys too.

 

PS.  he mentioned something about a a break on the dhcp...but it solved my problem.....


Yes, he manually "broke your lease", which is a way of saying he made your current dhcp IP assignment invalid so that when you rebooted your router, it ensured you got a new IP address.  We've been saying all along that it's a function of your IP assignment as to whether or not this issue appears.  But they're not fixing the problem when they have you do this.

 

This clearly indicates that, depending on what IP you have, if you have one of the bad ones your traffic gets routed across some distinct piece of equipment or combination of equipment or through some peer that has an incomplete path or is misconfigured.  Again, they're not fixing the problem when they give you a new IP.  The problem still exists within Verizon's infrastructure, but you're getting an IP that gets handled/routed differently and your new route doesn't experience the problem.  It's like your city's public works department telling you that the way to fix the potholes on your drive to work down Main Street is to take 2nd Street to get to work.

 

VZ's noc needs to be able to replicate it and have someone in real time be communicating with them so they can trace the packets as it traverses their network, until they find the point that it's being dropped (and I strongly suspect it will be the return path dropping, not the forward path).  My testing with the admins of the CDN node I was unable to reach confirmed that, at least in my case.

 

I'm in Murrieta, BTW.

mrballcb
Copper Contributor
Copper Contributor
Posts: 22
Registered: ‎01-27-2010

Re: DNS issues in SoCal

Message 89 of 188
(2,095 Views)

Oh, and I forgot to mention: in the original issue I had, which was last week, one of the sites I was unable to reach was forums.verizon.net, identical to one other poster who said he couldn't get to it at all.

RileyCat
Contributor
Contributor
Posts: 2
Registered: ‎07-28-2009

Re: DNS issues in SoCal

Message 90 of 188
(2,073 Views)

This has been going on in Temecula for over a week now. 

 

I have an open trouble ticket and can't get the issue solved.  I spoke with Rick at tech support again this AM and all that he could say was that it was an open issue and that it had been escallated.  He assured me that it would be resolved soon, yet when I asked what the issue was and how they planned to fix it he couldn't offer any information.

 

The orignaltechnicians I spoke with were clueless.  They asked me to clear the cache on all my browers and said that would clear it up. I didn't think it was funny. 

 

I can access craigslist.org from the starbucks down the street or from my work provided MiFi device and phone (Verizon) but not from any of my computers via Fios at home.

 

I can do a trace route to Craigslist..


C:\Users\Scott>tracert www.craigslist.org

Tracing route to www.craigslist.org [208.82.238.130]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  my.router [192.168.2.1]
  2     1 ms    <1 ms    <1 ms  myrouter.home [192.168.1.1]
  3     7 ms     7 ms     7 ms  L100.LSANCA-VFTTP-121.verizon-gni.net [173.58.73
.1]
  4    13 ms    12 ms    12 ms  G10-0-4-1921.LSANCA-LCR-09.verizon-gni.net [130.
81.138.84]
  5    15 ms    11 ms    13 ms  so-4-0-0-0.LAX01-BB-RTR1.verizon-gni.net [130.81
.28.72]
  6    16 ms    12 ms    12 ms  0.ae1.BR3.LAX15.ALTER.NET [152.63.2.129]
  7    12 ms    12 ms    12 ms  lap-brdr-03.inet.qwest.net [63.146.26.209]
  8    23 ms    21 ms    23 ms  svl-edge-23.inet.qwest.net [67.14.12.226]
  9    21 ms    22 ms    22 ms  65.113.32.14
 10    23 ms    21 ms    22 ms  vs-imgs-f.craigslist.org [208.82.238.130]

Trace complete.

 

 

Why can't Verizon resolve this?

How-To Videos
 
The following videos were produced by users like you!
   
Videos are subject to the Verizon Fios Community Terms of Service and User Guidelines and contains content that is not created by Verizon.
Have a spare Fios-G1100?Learn how to bridge it into your network
Get Started


Covid19

Browse Categories
Categories:
Posts

Verizon Troubleshooters
Unable to find your answer here? Try searching Verizon Troubleshooters for more options.
Modal Dialogue Title