The clinical benefits of androgen-deprivation therapy (ADT) for men with prostate cancer (PC) have been well documented and include living free from the symptoms of metastases for longer periods and improved quality of life. However, ADT comes with a host of its own serious side effects. There is considerable evidence of the adverse cardiovascular, metabolic, and musculoskeletal effects of ADT. Far less has been written about the psychological effects of ADT. This review highlights several adverse psychological effects of ADT. The authors provide evidence for the effect of ADT on men's sexual function, their partner, and their sexual relationship. Evidence of increased emotional lability and depressed mood in men who receive ADT is also presented, and the risk of depression in the patient's partner is discussed. The evidence for adverse cognitive effects with ADT is still emerging but suggests that ADT is associated with impairment in multiple cognitive domains. Finally, the available literature is reviewed on interventions to mitigate the psychological effects of ADT. Across the array of adverse effects, physical exercise appears to have the greatest potential to address the psychological effects of ADT both in men who are receiving ADT and in their partners.