Is there any downside to having the Orbi setup as a router as opposed to an AP? Could it be detrimental to the WiFi speed? I've had mine set up as a router for about 6 months, and wasn't even aware that it was supposed to be setup as an AP until I read this discussion thread.
Yes there are quite a few issues. You should never have two DHCP servers. It creates a double NAT. When you have two routers on the network you might have issues with port forwarding, VPNs, gaming, and even going to certain websites.
If you have FiOS TV you will most likely have issues with the FiOS TV app and remote DVR viewing.
The double NAT shouldn’t really cause wifi speed issues. Whether you have a mesh network or individual wireless access point you should only have one device handing out IP addresses.