Haldane observed that where interspecies hybrids show sex differences in sex ratio or fertility, it is always heterogametic sex which is affected. Many hypotheses have been advanced to explain Haldane's rule, but it is difficult to find a unifying explanation that will accommodate male and female heterogamety, and explains why fertility, especially, is affected. Here we propose that asymmetry can be accounted for by interactions involved in determining male dimorphisms or fertility between genes which diverged from allele pairs on the X and Y. Failure of one or more heterospecific interactions will render the male sterile. Similar arguments may be made for other groups with male or female heterogamety, and the argument can be extended to account for Y- (or W-) autosome interactions.