We report the experimental set-up and measurement system combining with a fast-scan Fourier transform spectrometer for photothermal ionization spectroscopy. They have been successfully applied for the investigation and detection of residual shallow impurity centers in ultra-pure germanium and silicon single crystals with high resolution as well as high sensitivity. Boron acceptor at a concentration of the order of magnitude of 108 cm-3 has been detected in ultra-pure germanium, and the detection limit of at least 107 cm-3 has been demonstrated. Lithiumoxygen donor complex D(Li, O) has been revealed in n-type highpurity silicon. Transition line series of both boron acceptor and phosphorus donor in p-type high-purity silicon have been observed simultaneously with the excitation of intrinsic light. In addition, the transitions associated with higher excited states of shallow impurities in both germanium and silicon have also been observed.