Background: One methodological difficulty in research into suicide attempts and suicide is distinguishing these phenomena from nonsuicidal self-harming behaviors and accidents. This is problematic because a reliable assessment of the presence or absence of the outcome variable is fundamental for the validity of the findings. Aims: To develop a standardized rating system, the Classification Algorithm for the Determination of Suicide Attempt and Suicide (CAD-SAS), and to investigate its psychometric properties. Methods: To examine the test-retest reliability, one investigator rated 217 narratives of real-life self-harming incidents at initial assessment and 4 weeks later. To establish the interrater reliability, three independent raters assessed a random sample of 70 narratives using the CAD-SAS. To examine the validity, one investigator using the CAD-SAS compared ratings to clinical judgments made by a consultant psychiatrist without the CAD-SAS on the same random set of 70 narratives. Results: Test-retest reliability was excellent (97.2% agreement) and interrater reliability was substantial (70.0% agreement, κ = 0.70). Agreement in the classification of incidents with the “real-world” clinical judgments supports the validity of the CAD-SAS (64.3% agreement, κ = 0.46). Conclusions: The reliability and validity of future studies can be enhanced through the standardized assessment and classification of incidents.