After the new router is booted up, do I need to reset(reprogram?) the settop boxes? Do I unplug each one & then plug it back in again?
Nothing needs to be done to the set-top boxes. Simply unplug the old router, then plug in the new router.
The only thing you may want to do is change the WiFi name and passphrase if you don't like the defaults. You will need to reconfigure your WiFi devices to use the new network and passphrase. Note that if you set the name and passphrase to the same as your old router, some devices may still need to have the WiFi connection removed and re-added to work.