Polarization imaging technology is a powerful tool in remote sensing, bioscience, or other scientific areas science, and it can extract the polarization information of target effectively. As a novel polarization imaging technology, the channel modulating polarization imaging has been widely investigated in recent years, owing to its prominent advantages of compact, snapshot and full-Stokes acquirable. In the polarization imaging system based on Savart Plate, a half wave plate with its optical axis at an angle of 22.5 is used to rotate the vibration direction of each incident light to 45. Thus, the amplitude of the light could be equally divided by the second Savart Plate polariscope. Finally, the different components of light will interfere with each other on the focal plane and the target polarization information will be modulated in the interference pattern. Since the intensity distribution of interference pattern is sensitive to the orientation of the half wave plate, a small mismatch angle of half wave plate will lead to a wrong polarization image. In order to solve this problem, we investigate the relationship between the mismatch angle and the image intensity grabbed by focal plane array (FPA) and propose an error eliminating method to improve the accuracy of polarization imaging. We analyze how the mismatch angle affects the light intensity and deduce the expression of the image obtained by the FPA. According to the expression, the raw image we grabbed directly by the FPA is a superposition of the modulated Stokes images with different carrier frequencies. Compared with the ideal expression, the expression we obtained shows that the channels of Stokes parameter S1 and S2,3 each contain a constant factor which is related to the mismatch angle . On the basis of this expression, we propose a method to measure the mismatch angle by imaging a target twice, one is behind a polarizer that is oriented at 0 and the other at 45. Then we can calibrate the system by calculating the mismatch angle through the demodulated images. To image a target with the calibrated system, we just demodulate the raw image obtained by the FPA and then divide the reconstructed stokes images by the constant factor which is determined by . For a mismatch angle of 0.5, a computer simulation is conducted. The result shows that through the compensation method, the accuracies of S1, S2 and S3 can be increased by 0.06%, 3.49% and 3.49%, respectively.