Functional magnetic-resonance imaging was used to identify areas involved in naming objects and to examine which of these areas adapted to either physical or semantics features of objects. We presented successive pairs of objects that were either the same exemplar of an object, different exemplars of that object, or different objects. By controlling for differences in physical features between pairs of different exemplars and different objects, visual areas in the occipital-temporal cortex were subject to repetition suppression when the same exemplars of an object were repeated, but not when different exemplars of an object were repeated. This was true independent of whether or not participants named objects. Repetition suppression in visual areas appeared therefore bound to physical features. Nevertheless, repetition suppression for physical features was greater in left visual areas when objects were named, suggesting that naming, known to depend on mechanisms in the left hemisphere, may induce greater attentional modulation in the left than in the right visual areas. Taken together, we propose that the difference between our findings and those of earlier studies that report semantic influences can be explained by the failure of those studies to control for differences in the appearance of different exemplars. Left frontal areas showed repetition suppression when either the same or different exemplars of an object were repeated, but only when participants named objects. These results suggest that visual areas process information about physical features without semantic modulation by higher-order areas, and that left frontal areas process semantic features, but the engagement of these processes is task modulated.