Markymark, I think things vary somewhat here in the UK - your doctor will assess what he or she thinks is your particular risk. But my guess from what you say is that you would be close to the point where most would recommend treatment.
After which there is of course the question of which treatment, etc. From my experience (surgery as first treatment, but radiotherapy needed when PSA rose again) it would be extremely tough going through surgery without support but radiotherapy was better - though you might feel pretty lonely at times.