QUESTION: What is the inter-rater reliability of the Assessment of Physiotherapy Practice (APP) instrument, and what is the error associated with individual scores? DESIGN: Cross-sectional inter-rater reliability study. Thirty pairs of clinical educators each assessed one student after observing student practice over a 5-week clinical placement. PARTICIPANTS: Sixty clinical educators from five Australian universities formed 30 independent pairs of assessors. OUTCOME MEASURES: Each pair completed two independent assessments of one student, providing 60 completed APP assessments and an associated Global Rating Scale score for analysis. ANALYSIS: Correlational coefficients and measurement error expressed in APP scale units were computed to provide a comprehensive analysis of the likely utility of APP scores and to enable score and change score interpretation. RESULTS: Percentage of agreement between assessors for each item ranged from 56% (Item 19, evidence-based practice) to 83% (Item 20, risk management) and across all items averaged 70% (SD 7). The ICC(2,1) was 0.92 (95% CI 0.84 to 0.96) for the total APP score and 0.72 (95% CI 0.50 to 0.86) for the Global Rating Scale. The standard error of measurement for the total score (scale width 0-80) was 3.2 APP points and the MDC(90) was 7.86 representing 9% of the scale width. Bland-Altman analyses identified no systematic differences between raters. CONCLUSION: Clinical educators demonstrated a high level of reliability when using the APP instrument to assess physiotherapy students' level of professional competence in workplace-based practice.