The aim was to develop an electronic adverse event (AE) screening tool applicable to acute care hospital episodes for patients admitted with chronic heart failure (CHF) and pneumonia.Consensus building using a modified Delphi method and descriptive analysis of hospital discharge data.Consultant physicians in general medicine (n = 38).In-hospital acquired (C-prefix) diagnoses associated with CHF and pneumonia admissions to 230 hospitals in Victoria, Australia, were extracted from the Victorian Admitted Episodes Data Set between July 2004 and June 2007. A 9-point rating scale was used to prioritize diagnoses acquired during hospitalization (routinely coded as a 'C-prefix' diagnosis to distinguish from diagnoses present on admission) for inclusion within an AE screening tool. Diagnoses rated a group median score between 7 and 9 by the physician panel were included.Selection of C-prefix diagnoses with a group median rating of 7-9 in a screening tool, and the level of physician agreement, as assessed using the Interpercentile Range Adjusted for Symmetry.Of 697 initial C-prefix diagnoses, there were high levels of agreement to include 113 (16.2%) in the AE screening tool. Using these selected diagnoses, a potential AE was flagged in 14% of all admissions for the two index conditions. Intra-rater reliability for each clinician ranged from kappa 0.482 to 1.0.A high level of physician agreement was obtained in selecting in-hospital diagnoses for inclusion in an AE screening tool based on routinely collected data. These results support further tool validation.