Sorry for what you and your husband are going through.
It's difficult to understand from what you're saying if they think there's still a curative treatment path available to him, or just a managed treatment path.
Assuming a curative treatment path is not ruled out, at this moment, I would be pushing for a PSMA PET/CT scan to find where the cancer cells are (his PSA is well above the level for that to be viable in the case of no prostate), and ideally you do that before starting hormone therapy as hormone therapy will make the scan less sensitive. Hospitals which have PSMA PET/CT scanners on site tend to do this when PSA reaches 0.2 or 0.5 (depending on their scanner), but hospitals which don't tend to wait much longer before referring to get a scan elsewhere (for no good reason), or skip it, so you may need to push to get this.
When they know where the cancer is, then they can say which treatment path is viable. Also, it enables radiotherapy (the next curative treatment option) to be targeted to where the cancer cells are, rater than just given blindly to the prostate bed (where they're assumed to be in the absence of a scan).
Where abouts do you live and which hospital is he being treated at?