Background: Although obtaining sexual history from patients is essential, the attitudes of physicians can become a barrier to sexual health care. Iranian–American physicians may face particular challenges because talking about sexuality is considered a taboo within their culture. Our study examined these physicians’ attitudes when taking a sexual history from their patients. Methods: In 2013, a self-administrated questionnaire was sent to 1550 Iranian–American physicians in California, USA. Using factor analysis, the principal components approach with a Varimax rotation was used on a set of 12-item questions (five-point Likert scales) to detect latent factors that explain attitudes affecting sexual history taking. Scores are generated to determine physicians’ attitudes towards sexual history taking. Results: In total, 354 questionnaires were returned (23% response rate). Three factors were identified as internally consistent (Cronbach’s α = 0.84 – 0.94): (1) attitude towards various patients; (2) female sexuality; and (3) age and marriage. Significant association were found between these three factors and some variables such as physicians’ gender, country of medical graduation, religion, birthplace and age. Conclusions: Results revealed that cultural attitudes are important factors affecting physicians’ involvement in sexual history taking. Additional studies from this population and other subpopulations of US physicians are needed. New strategies that reflect on physicians’ attitude on sexual healthcare delivery is needed. If confirmed in other studies, our findings could have implications for the training of medical graduates globally.