Probability safety assessment has been widely used in nuclear plant safety research, but the reliability research of passive system is at a beginning level. The failure of passive system consists of components failure and physical process failure. Calculating the failure probability of physical process is a very difficult task due to the uncertainties of the factors which affect the physical process, and the method for the calculation is quite different from that of active system. The paper introduces response surface method to calculate the failure probability, and applies the method to get the approximate failure probability of residual heat removal system in HTR-10, the 10MW high temperature gas reactor designed by the Institute of Nuclear and New Energy Technology at Tsinghua University.