In conventional phase-only holographic display, the phase-only computer generated hologram is usually calculated based on the fast Fourier transform (FFT) algorithm, in which the Nyquist theory should be satisfied. However, due to the pixel structure of the liquid crystal spatial light modulator and a fixed spatial sampling rate, the size of the reconstructed image is limited by the space-bandwidth product of the liquid crystal phase modulator. The traditional solution is to use convolution algorithm or double-step Fresnel diffraction algorithm to calculate the Fresnel hologram, but FFT has to be calculated many times in both of the methods, thereby increasing the burden of hologram computation. Therefore, in this paper we propose a method to calculate the phase-only hologram based on setting a virtual hologram plane. This virtual hologram plane is set based on the principle of lens imaging. So the calculation of the hologram can be divided into two steps: the first step is to calculate the Fresnel diffraction from the object plane to the virtual hologram plane, and the second step is to calculate the hologram from the virtual hologram plane by being multiplied with a quadratic phase term. In this way, the hologram can be calculated from the original object with any sampling rate we need by adjusting the corresponding parameters of distance. By this method one can calculate the Fresnel diffraction between hologram plane and object plane with variable sampling rates, without considering the space-bandwidth product of the liquid crystal phase modulator, and this algorithm uses only one FFT calculation, which can speed up the calculation of hologram compared with the convolution based method (using three FFTs in calculation) and the double-step Fresnel method (using two FFTs in calculation). Both the computer simulation and the optical experiments demonstrate that the object can be reconstructed with different sizes in the holographic display system. In the optical experiment, the zero-order diffraction can be removed by placing a filter on the back focal plane of the imaging lens and the speckle noise can also be eliminated in order to improve the reconstruction quality by displaying multiple phase-only holograms at a high speed. The proposed method in this paper shows a potential application in zoom-able liquid crystal spatial light modulator based holographic display system.