Poor bandwidth despite compliant values in FiOS Speed Test

[Jb19761]

I am looking for assistance in convincing FiOS support that there is a problem within their network that affects bandwidth to various hosts. I have observed this to include Google products (G-Suite apps and search), Tumblr, and other sites.

I had a service ticket open for Internet outage that is rumored to have been a component between the neighborhood and CO; that was fixed before my scheduled tech visit. We discovered that when the tech and the neighbor had a chat about her service outage the previous day. What I did observe about my outage was the router received IP information but would not pass traffic from my devices.

Some weeks later I noted strange lag and connectivity issues with Google products needed for work from home effort. A call to Support determined there was some speed issue from my router to the network, as reported by the FiOS Speed Test app. A tech was scheduled to visit and "run more speed tests".

While I waited I looked at packet captures with WireShark and I think I observe recurring drops of random TCP packets, generating SACK retransmit requests. This is hard to pin down without observing services that maintain long-running TCP connections. Google products do (especially GMail), and it is observable with speed test streams. I believe the connectivity issue occurs due to the missing dropped packet and what I presume are dropped retransmit requests from the client. If the retransmit request can make it to the server, then suddenly things move along until the next drop. What this looks like is "jerky" loading for app frontends or content.

On the day of and just prior to the tech arrival, I set up a laptop near the router and ONT to see about conducting more testing. I replicated the WireShark observations over Ethernet, through my G1100 router. I then moved the laptop to the ONT and was able to observe the same behavior. So this isn't a router issue.

The tech was on site for a very long time. He was able to observe problems with the G1100 router. A remote tech was engaged and he, too, observed problems. Difficulty ranged from bad FiOS Speed Test values to total loss of connectivity. He swapped out the ONT to no effect. He tested and confirmed light levels on the fiber. He swapped my G1100 router with a new one from the truck. Nothing.

At times he had the remote tech on speaker and I heard conversation about cross connects for testing, etc. Sometimes this made for super slow connections or none at all.

Then he brought in a new G3100 router. It took more time, but eventually they got that "working". FiOS Speed Test showed Gigabit values from the router to the network.

But this wasn't what he was here to fix. I described my problem and observations. I was able to show him that Google email and search had serious lag. I showed him the behavior is consistent on a wireless cell phone using Google apps. At his request I paired my laptop to my cell phone, disabled WiFi on the cell, and made everything work fine over the cell network. I also showed him WireShark in operation, duplicate ACK messages scrolling by.

He agreed something was odd, but it had to be with something other than FiOS. Apparently since the FiOS Speed Test shows "Gigabit" values, that is all he needs to demonstrate to successfully resolve the ticket.

He was also unable to explain why he wasn't able to get a G1100 router working and had to switch to a G3100. He left quickly after three hours.

And now instead of being in customer-owned router status, I'm paying a $15 router rental fee. The bandwidth issue is still present.

I am suspicious that the bandwidth issue impacts the FiOS Speed Test, such that missing ACKs come in just toward the end of the test stream. If the operation were to compute the total time duration of the stream and the bytes transferred, a valid value could come back. But the customer experience waiting on the stream to complete before seeing the product would still be terrible. I think there are instantaneous time calculations made to help with "animating" the numbers (or the pointer on the gauge, if you use another service), but these aren't pertinent to the task of a "speed test".

Apologies for the long post. I am aware I have two issues now: Poor bandwidth and an unnecessary modem rental.

Is the bandwidth issue similar to anyone? 

[Cang_Household]

That G3100 is probably unnecessary. G1100 is capable of gigabit NAT throughput.

Otherwise, can you post some WireShark images showing TCP retransmission? Please redact your IP address for privacy.

If the Fios speed test shows gigabit, then it is gigabit between the Verizon network and yours. Fios speed test does not rule out layer 3 issues between the Verizon network and Google's.