The diamond stylus extrudes the metal film and causes coupled elastic-plastic deformation which shapes the profile of the ruled grating. The grating profile function which is hard to get would influenced accurate analysis of the diffraction characteristic. An analytical method of the diffraction characteristic of the mechanically ruled grating is presented based on the profile fitting function. The problem of excessive departure between the measured and theoretical value of the diffraction efficiency of high density ruled grating is solved by this method. We take a 1200l/mm UV grating，the grating profile is scanned in the principal section and then the profile curve function is got. The reason why the diffraction efficiency wasn't high is found by calculation. And then the diffraction efficiency increased about 20% by improving the technology.