Spermatogenesis is a complex sequential process that converts mitotically dividing spermatogonia stem cells into differentiated haploid spermatozoa. Not surprisingly, this process involves dramatic nuclear and chromatin restructuring events, but the nature of these changes are poorly understood. Here, we linked the appearance and nuclear localization of the essential histone variant H2A.Z with key steps during mouse spermatogenesis. H2A.Z cannot be detected during the early stages of spermatogenesis, when the bulk of X-linked genes are transcribed, but its expression begins to increase at pachytene, when meiotic sex chromosome inactivation (MSCI) occurs, peaking at the round spermatid stage. Strikingly, when H2A.Z is present, there is a dynamic nuclear relocalization of heterochromatic marks (HP1beta and H3 di- and tri-methyl K9), which become concentrated at chromocenters and the inactive XY body, implying that H2A.Z may substitute for the function of these marks in euchromatin. We also show that the X and the Y chromosome are assembled into facultative heterochromatic structures postmeiotically that are enriched with H2A.Z, thereby replacing macroH2A. This indicates that XY silencing continues following MSCI. These results provide new insights into the large-scale changes in the composition and organization of chromatin associated with spermatogenesis and argue that H2A.Z has a unique role in maintaining sex chromosomes in a repressed state.