"Unless there's reason to suppose that the cancer will require treatment while you're away,"
or that circumstances may change in between booking and going, or there is a rise in PSA requiring RT that hadn't been anticipated, etc, etc.
Also worth considering that if you ask for PCa to be excluded and then get struck down by an awful UTI that ruins your holiday they will argue like hell that the UTI was linked to the PCa. Likewise if you go into retention, have a priapism, suffer breathlessness and suspect a heart attack but it turns out to be the HT or any other thing that could even slightly be associated.