Ventricular enlargement (VE) is commonly observed in aging and fragile X-associated tremor/ataxia syndrome (FXTAS), a late-onset neurodegenerative disorder. VE may generate a mechanical force causing structural deformation. In this longitudinal study, we examined the relationships between VE and structural changes in the corpus callosum (CC) and putamen. MRI scans (2-7/person over 0.2-7.5 years) were acquired from 22 healthy controls, 26 unaffected premutation carriers (PFX-), and 39 carriers affected with FXTAS (PFX+). Compared with controls, PFX- demonstrated enlarged fourth ventricles, whereas PFX+ displayed enlargement in both third and fourth ventricles, CC thinning, putamen atrophy/deformation (thinning and increased distance), and accelerated expansions in lateral ventricles. Common for all groups, baseline VE predicted accelerated CC thinning and putamen atrophy/deformation and conversely, baseline CC and putamen atrophy/deformation and enlarged third and fourth ventricles predicted accelerated lateral ventricular expansion. The results suggest a progressive VE within the 4 ventricles as FXTAS develops and a deleterious cycle between VE and brain deformation that may commonly occur during aging and FXTAS progression but become accelerated in FXTAS.