The spectral transfer function of the imaging spectrometer is defined on the basis of the linear system theorem. The spectral transfer functions including spectral modulation transfer functions and the spectral phase transfer functions of the temporally modulated Fourier transform spectral imager based on Michelson interferometer and those of the spatially modulated Fourier transform spectral imagers based on Sagnac interferometer,Fresnel interferometer, and Lloyd interferometer are deduced. These spectral transfer functions could be regarded as an objective criteria in the spectral domain of spectral imagers. Together with the well known optical transfer function, the performance of the imaging spectrometer could be described comprehensively.