Genetic variation cannot be the only answer. Yes it has a role in all this but it leaves too many questions unanswered, and probability deems it highly unlikely that outliers are so common and responses so different. In the end, we are more similar than we are different. We have the same HPTA, which produces and responds to the same substances, in a predictable manner, and which substances produce predictable effects across every human bar those with genetic insensitivity, who are extremely rare (documented cases you can count using your fingers). That's how we have any clue what we're doing with TRT in the first place, because cause and effect are generalisable with these medications and hormones. There is a precise and measurable reason why someone is a high or low responder, and we cannot chalk it up to medication response so easily in the first place. when we don't know that reason, we say "genetics". But we can't just say "genetics" whenever we reach a point that our current understanding fails to predict. It allows too wide a degree of variation in an otherwise too predictable system. Saying "I have no clue" is more accurate, and trying to improve our understanding so that it can predict that outcome as well is necessary if we are to know what we're doing eventually.
A good theory should be testable, coherent, economical, generalizable, explain known findings and be predictive of new ones. The more it explains and predicts the better it is.