Most accounts of vision assume that its function is largely perceptual, providing an internal model or representation of the external world that serves as the visual foundation for thought and action. However, the evolutionary origins of vision are not representational. Vision evolved not to provide perception of the world but to provide distal control of the many different movements that animals make. Moreover, many of these visuomotor control systems are quite modular in their input-output organization. In humans and other primates, these visuomotor modules function quite independently from the neural mechanisms mediating perception-based knowledge of the world. This division of labour between action systems and perception systems can be seen in the organization of the visual pathways in the primate cerebral cortex. The ventral stream of projections from striate cortex to inferotemporal cortex is critical to the visual perception of objects and is intimately connected with the cognitive operations, while the dorsal stream projecting from striate cortex to the posterior parietal region is essential for the required visuomotor transformations for the on-line control of skilled actions directed at those objects. The perceptual representations constructed by the ventral stream allow us to select a particular course of action with respect to objects and events in the world; the visuomotor transformations carried out by the dorsal stream allow us to program and direct any visually guided movements that are required to carry out that course of action. Thus, to understand the organization of the visual system(s), it is necessary to understand the requirements of the different output systems that vision serves.