My diagnosis and treatment exactly the same as yours. Mine was just labeled T3 without any letters, but MRI said extra capsular extension.
I didn't have much problem with HT so went the full two years. If it is having a big impact on your QoL then stopping is reasonable and we have a one or two members who have.
Worth considering is how HT and RT are interacting. In a fetus all cells start as stem cells, they then become organ specific cells. Hormones play a part in this. Apoptosis is a process where if a cell finds it isn't needed or is in the wrong place it commits suicide. If testosterone is present then some cells will become male genitalia, prostate etc. When on HT these cells become less active, it is almost as if the cells are thinking we are male specific cells but I think we may be in a female body. I don't think this is strong enough to cause apoptosis, but it is enough to make the cells inactive. RT does not kill cancer cells, it damages them such that when they reproduce they fail to divide correctly and then they die. Cancer cells are very bad at repairing RT damage. Prostate cells normally divide after about 6 to 24 months. So it should take about two years before most of the cancer cells have died. Some cancer cells might not divide for five or ten years, but if the RT did its job, and the cell has not repaired the RT damage it will then die.
In summary: RT is damaging the cancer cells, cancer cells are very bad at repairing the damage and HT is making the cells less active so even less able to repair the damage. No matter when you stop HT there will be damaged cancer cells which have not yet died, but they probably will die when they divide. The longer you are on HT the fewer dieing cancer cells will be left when you stop, so the less chance that one may repair the RT damage and start a new tumour.
Certainly within six months of RT there would be a lot of cells to potentially start a new tumour, the chances of any doing that are tiny, but with a lot of cells the odds are stacked against you, after one year, the number of cells has declined massively, after two years, even further.
No one knows what your risk is, but the oncology has taken a guess, based on a lot of experience. He is balancing that risk against the side effects in an average patient. You do know what your side effects are and how tolerable they are to you, so you get to make the final decision and as long as it is informed, it is the right decision even if it results in an early death.