HT is only temporarily effective (that can be very many years, but less likely with G4+5). It will eventually stop working.
Getting your PSA < 0.1 when you start RT is excellent - it results in improved outcomes. You should have the RT ASAP, before the HT becomes less effective on your cancer.
I delayed my RT to let the HT get my PSA down to 0.1 (actually, I was 0.12 just before first session), but not everyone can get their PSA that low on HT, and if you wait too long, you will start to see it increase.