Two-phase learning method is considered in this paper to configure the RBF neural networks for chaotic time series prediction. When determining the hidden-layer centers with the unsupervised learning algorithm, a new distance measure is presented based on Gaussian basis, and the strategy of input-output clustering is employed in combination. The punishment factor in Gaussian basis distance is designed based on Fisher separable ratio, which can improve the clustering performance. Moreover, the introduction of input-output clustering strategy establishes the relation between the clustering performance and the prediction performance. Therefore, the RBF neural networks constructed by this method can not only assure the compact structure, but also improve the prediction performance. This method is applied to Mackey-Glass, Lorenz and Logistic chaotic time series prediction, and the results indicate its validity.