Fatigue, breathlessness and muscle loss are known side effects of hormone treatment - so exercise at the same level you are used to doing can become more tiring because your muscles have to work harder. Radiotherapy can also cause fatigue, particularly from half way through the programme until a couple of weeks after it finishes. So yes, what you are experiencing is very common.
Continuing to exercise as much as you feel able, but perhaps at a slightly slower pace or with more breaks, taking an afternoon nap if needed (close your office door and rest your head on the desk for a while if necessary), making adjustments to the regime if your joints are suffering, low impact exercise such as swimming, etc.
PCUK ran a project on managing fatigue during HT a while back - you could call them on the number at the top of this page to get some ideas of what might help.