Just keep your router firmware up to date, restrict external access to the control panel, and only port forward that port unless you do port knocking, however I would still be hesitant.
If your router supports DD-WRT, then any dynamic IP changes can be kept in sync with free dns services that...