Log into the router's web interface, which is similar to when you first set it up, then navigate to the section where you can update the firmware. This is normally in the Administration, System, Setup ...

AirPods firmware updates automatically when you meet Apple's prerequisites. If updates aren't automatic, force an update by ...