Although archaea are ubiquitous in various environments, the knowledge gaps still exist regarding the biogeographical distribution of archaeal communities at regional scales in agricultural soils compared with bacteria and fungi. To provide a broader biogeographical context of archaeal diversity, this study quantified the abundance and community composition of archaea across the black soil zone in northeast China using real-time PCR and high-throughput sequencing (HTS) methods. Archaeal abundances across all soil samples ranged from 4.04 × 107 to 26.18 × 107 16S rRNA gene copies per gram of dry soil. Several soil factors were positively correlated with the abundances including soil pH, concentrations of total C, N, and P, and available K in soil, and soil water content. Approximately 94.2, 5.7, and 0.3% of archaeal sequences, and 31, 151, and 3 OTUs aligned within the phyla Thaumarchaeota, Euryarchaeota, and Crenarchaeota, respectively. Within the phylum of Thaumarchaeota, group 1.1b was a dominating genus accounting for an average of 87% archaeal sequences and phylogenetically classified as Nitrososphaera, a genus of ammonia oxidizing archaea. The response of dominating OTUs to environmental factors differed greatly, suggesting the physiological characteristics of different archaeal members is diversified in the black soils. Although the number of OTUs was not related with any particular soil parameters, the number of OTUs within Thaumarchaeota and Euryarchaeota was marginally related with soil pH. Archaeal community compositions differed between samples, and a Canonical correspondence analysis (CCA) analysis indicated that soil pH and the latitude of sampling locations were two dominating factors in shifting community structures. A variance partitioning analysis (VPA) analysis showed that the selected soil parameters (32%) were the largest drivers of community variation, in particular soil pH (21%), followed by geographic distances (19%). These findings suggest that archaeal communities have distinct biogeographic distribution pattern in the black soil zone and soil pH was the key edaphic factor in structuring the community compositions.