Little is known about the impact of HIV on the mental health of older gay men. In a nationwide online survey, 1,135 Australian gay men aged 40 years and older reported on their physical and mental health. Compared with HIV-negative men, those with HIV were more likely to report currently receiving treatment for a mental health condition (37 vs. 25 %, p = 0.001), but no more likely to report treatment for a major physical health condition other than HIV (39 % vs. 40 %). On a measure of psychological distress, more HIV-positive men were highly distressed (14 vs. 8 %, p = 0.01). There were no significant differences in mental health across age groups for either HIV-positive or HIV-negative men. Underlying factors for poorer mental health included lower income (p = 0.003), not working (p = 0.004), living alone (p = 0.03), treatment for a major physical health condition (p < 0.001), recent experiences of discrimination (p = 0.03), and regarding one's sexual orientation as an essential part of self-identity (p < 0.001). With disparities between older HIV-positive and HIV-negative gay men greater for mental than physical health, more attention may need to be given to the psychological well-being of those living with HIV, particularly by addressing underlying psychosocial factors.