Objectives: “Karoshi” (death due to overwork) of doctors occurred frequently and attracted increasing attention in recent years in China. This study aimed to determine the prevalence of work-related accumulated fatigue of doctors and its associated factors in tertiary hospitals of China. Methods: A cross-sectional questionnaire survey was conducted on 1729 full-time doctors employed by 24 tertiary hospitals across eastern developed, central developing, and western underdeveloped regions of China. Accumulated fatigue was categorized into four levels using the “Self-diagnosis Checklist for Assessment of Workers’ Accumulated Fatigue” rated on a scale matrix considering both overwork and fatigue symptoms. Ordinal logistic regression analyses were performed to identify factors associated with work-related accumulated fatigue. Results: About 78.8% of respondents reported a “high level” of work-related accumulated fatigue, including 42.0% at a “very high” level. Male doctors and those aged between 30 and 45 years and who had a professional title were found to have higher levels of accumulative fatigue than others. Low salary and poor working conditions (in the western region) were also significantly associated with high levels of work-related accumulated fatigue (p < 0.05). Conclusion: High levels of work-related accumulated fatigue are prevalent in doctors working in tertiary hospitals in China. Male doctors establishing their early- and mid-careers are the high-risk group. Poor working conditions are associated with work-related accumulated fatigue.