With more satellites launched into orbits during recent years, monitoring and cataloging of satellites play an important role in improving the utilization rate of space resource and alleviating the pressure of orbit resource. Groundbased radar, a kind of sensor in space surveillance system, does not consider the influences of the weather and other special circumstances. And it is a key technology in space target tracking by using the measurement data for real-time orbit determination. Due to the influence of orbital perturbation, the satellite orbital dynamic model is a nonlinear system. The optimal estimation of the orbital state can be achieved by means of nonlinear filtering based on the measured ranging, velocity and angle data with measurement noise, which is the essence of real time orbit determination and has important research value. The extended Kalman filter (EKF) and unscented Kalman filter (UKF) are most widely used nonlinear Kalman filters. However, the first-order Taylor expansion of nonlinear function in EKF degrades the filtering accuracy. And the weight value in UKF might be negative for the high-dimensional system, which may directly affect the filtering stability. As an important method in nonlinear filtering, cubature Kalman filter (CKF) has better accuracy and stability than UKF. However, CKF only has third-degree filtering accuracy. In order to improve the filtering accuracy further, some fifth-degree cubature Kalman filters are proposed, mainly including the fifth-degree cubature Kalman filter and the fifth-degree spherical simplex-radial cubature Kalman filter. The optimality of the radial integral cannot be guaranteed by using the moment matching method in these fifth-degree filters, so a high-degree cubature quadrature Kalman filter (HDCQKF) is proposed. The radial integral is calculated using the high-degree Gauss-Laguerre formula in HDCQKF. However, the aforementioned filtering algorithm leads to an increase in the number of cubature points, thereby improving the accuracy, and the number of cubature points increases polynomially with the increase of system dimension. Once the algorithm is applied to a high-dimensional system, or the processor has a relatively poor performance, it may impose a heavier computing burden, thus the real-time performance decreases. Therefore, it is necessary to study how to reduce the computational complexity of the fifth-degree filtering algorithm. In order to improve the real-time performance of orbit determination on condition that the accuracy of orbit determination is kept, a novel fifth-degree cubature Kalman filter for orbit determination is proposed at the lower bound approaching to the number of cubature points. The key problem in the nonlinear Kalman filter is to calculate the multidimensional integral in the form of nonlinear functionGaussian probability density function, and the integral is approximated using a fifth-degree numerical cubature rule, in which the number of cubature points required is only one more than the theoretical lower bound. The abovementioned cubature rule is embedded into the nonlinear Kalman filtering framework, from which the update steps of the novel cubature Kalman filter are derived. Then, the equations of state and measurement for real-time orbit determination are obtained. The J2 perturbation and atmospheric drag perturbation are taken into account in the state equation, and the coordinate transformation is used to derive the nonlinear relationship between the orbital state and measurement element. The simulation results show that the proposed fifth-degree cubature Kalman filter can achieve a higher filtering accuracy than the CKF and the same accuracy as the existing fifth-degree filters, but has the fewest cubature points and the best real-time performance, which proves the effectiveness of the proposed algorithm.