β-Amyloid deposition is one of the main hallmarks of Alzheimer's disease thought to eventually cause neuronal death. Post-mortem and neuroimaging studies have consistently reported cases with documented normal cognition despite high β-amyloid burden. It is of great interest to understand what differentiates these particular subjects from those without β-amyloid deposition or with both β-amyloid deposition and cognitive deficits, i.e. what allows these subjects to resist the damage of the pathological lesions. [¹¹C]Pittsburgh compound B positron emission tomography and magnetic resonance brain scans were obtained in 149 participants including healthy controls and patients with subjective cognitive impairment, mild cognitive impairment and Alzheimer's disease. Magnetic resonance data were compared between high versus low-[11C]Pittsburgh compound B cases, and between high-[¹¹C]Pittsburgh compound B cases with versus those without cognitive deficits. Larger temporal (including hippocampal) grey matter volume, associated with better episodic memory performance, was found in high- versus low-[¹¹C]Pittsburgh compound B healthy controls. The same finding was obtained using different [¹¹C]Pittsburgh compound B thresholds, correcting [¹¹C]Pittsburgh compound B data for partial averaging, using age, education, Mini-Mental State Examination, apolipoprotein E4 and sex-matched subsamples, and using manual hippocampal delineation instead of voxel-based analysis. By contrast, in participants with subjective cognitive impairment, significant grey matter atrophy was found in high-[¹¹C]Pittsburgh compound B cases compared to low-[¹¹C]Pittsburgh compound B cases, as well as in high-[¹¹C]Pittsburgh compound B cases with subjective cognitive impairment, mild cognitive impairment and Alzheimer's disease compared to high-[¹¹C]Pittsburgh compound B healthy controls. Larger grey matter volume in high-[¹¹C]Pittsburgh compound B healthy controls may reflect either a tissue reactive response to β-amyloid or a combination of higher 'brain reserve' and under-representation of subjects with standard/low temporal volume in the high-[¹¹C]Pittsburgh compound B healthy controls. Our complementary analyses tend to support the latter hypotheses. Overall, our findings suggest that the deleterious effects of β-amyloid on cognition may be delayed in those subjects with larger brain (temporal) volume.