Incoherent scatter radar is one of the most important detection instruments of the space plasma. But because of the low dust density in natural space plasma, the contribution of charged dust to incoherent scatter spectrum can be completely ignored, therefore the incoherent scattering theory has not appeared in dusty plasma. In the solid rocket plume, the propellant combustion can form a large number of nanometer- and micronmeter-sized dusty particles, and produce a high electron density from high temperature ionization, which makes considerable contributionto charged dusty particles with the high density. Therefore, we develop the incoherent scattering theory of dusty plasma in order to calculate the scattering characteristics of high density dusty plasma produced by rocket plume, for example. The theoretical model including electrons, ions and dusty particles is established by combining effects of charged dusty particles. The incoherent scatter spectral lines of ion resonance region and dust resonance regionare calculated. The effects of dusty particle radius, temperature and density on spectral line structure are discussed. With the increases of dusty particle radius and density, the amplitude of power spectrum increases. With the increase of dust temperature, the amplitude of power spectrum decreases. In the dust resonance region, the control mechanism of dust in spectrum is similar to that of the ions. With the increase of particle size (mass) and decrease of the temperature, the spectrum width narrows, and amplitude and area increase with the increase of density. But in the ion resonance region, the dust control mechanism is completely different, and the influence of the dust on ion line is in the way of attracting ions. So with the increase of dust density, ion line characteristics do not show that the area increases, and dust controls ions by adjusting the Debye radius or electrostatic shielding ball size. By comparing the ion lines with and without dust under the same parameters conditions, the amplitude of the ion line with dust is much larger than that without dust, and the resonance frequency of the ion line is greatly changed. With the dust particles of a relatively high density, one can enhance the ion line, hence the incoherent scattering phenomenon can be more easily observed in rocket plume. On the other hand, due to significant changes of frequency and amplitude in the ion line spectrum, the incoherent scattering inversion method based on the traditional theory will cause a large error in the inversion parameter, even a failure of parameter retrieval. The incoherent scattering theory and relevant physical laws of dusty plasma are presented, which are of great significance for establishing the incoherent scattering theory system and studying the rocket plume parameters.