To investigate a relationship between maximum rectal diameter (MRD) on pre-treatment cone beam computed tomography (CBCT) and intra-fraction prostate motion, in the context of an adaptive image-guided radiotherapy (IGRT) method.The MRD was measured on 2125 CBCTs from 55 retrospective patient datasets and related to prostate displacement from intra-fraction imaging. A linear regression model was developed to determine a threshold MRD associated with a high probability of small prostate displacement. Standard and reduced adaptive margin plans were created to compare rectum and bladder normal tissue complication probability (NTCP) with each method.A per-protocol analysis carried out on 1910 fractions from 51 patients showed with 90% confidence that for a MRD≤3 cm, prostate displacement will be ≤5 mm and that for a MRD≤3.5 cm, prostate displacement will be ≤5.5 mm. In the first scenario, if adaptive therapy was used instead of standard therapy, median reductions in NTCP for rectum and bladder were 0.5% (from 9.5% to 9%) and 1.3% (from 6.6% to 5.3%), respectively. In the second scenario, the NTCP for rectum and bladder would have median reductions of 1.1% and 2.6%, respectively.We have identified a potential method for adaptive prostate IGRT based upon predicting small prostate intra-fraction motion by measuring MRD on pre-treatment CBCT.