Physiological studies of color processing have typically measured responses to spatially varying chromatic stimuli such as gratings, while psychophysical studies of color include color naming, color and light, as well as spatial and temporal chromatic sensitivities. This raises the question of whether we have one or several cortical color processing systems. Here we show from non-linear analysis of human visual evoked potentials (VEP) the presence of distinct and independent temporal signatures for form and surface color processing. Surface color stimuli produced most power in the second order Wiener kernel, indicative of a slowly recovering neural system, while chromatic form stimulation produced most power in the first order kernel (showing rapid recovery). We find end-spectral saturation-dependent signals, easily separable from achromatic signals for surface color stimuli. However physiological responses to form color stimuli, though varying somewhat with saturation, showed similar waveform components. Lastly, the spectral dependence of surface and form color VEP was different, with the surface color responses almost vanishing with yellow-grey isoluminant stimulation whereas the form color VEP shows robust recordable signals across all hues. Thus, surface and form colored stimuli engage different neural systems within cortex, pointing to the need to establish their relative contributions under the diverse chromatic stimulus conditions used in the literature.