Patients with optic ataxia, a deficit in visually guided action, paradoxically improve when pantomiming an action towards memorized stimuli. Visual form agnosic patient D.F. shows the exact opposite pattern of results: although being able to grasp objects in real-time she loses grip scaling when grasping an object from memory. Here we explored the dissociation between immediate and delayed grasping in a patient (F.S.) who after a parietal-occipital stroke presented with severe left visual neglect, a loss of awareness of the contralesional side of space. Although F.S. had preserved grip scaling even in his neglected field, he was markedly impaired when asked to pretend to grasp a leftward object from memory. Critically, his deficit cannot be simply explained by the absence of continuous on-line visual feedback, as F.S. was also able to grasp leftward objects in real-time when vision was removed. We suggest that regions surrounding the parietal-occipital sulcus, typically damaged in patients with optic ataxia but spared in F.S., seem to be essential for real-time actions. On the other hand, our data indicates that regions in the ventral visual stream, damaged in D.F but intact in F.S., would appear to be necessary but not sufficient for memory-guided action.