I agree. I had 32 sessions of RT and got pretty fatigued at times (on Zoladex/Enzalutimide/Abiraterone also), I always tried to keep up with exercise, as much as I could, which I think is advised anyway. Drawbacks as mentioned are the toilet issues. The fact I was having RT didnt stop me, only the effects.
Peter