The rapid measuring of the underwater temperature on a large scale is very important for marine monitoring, and it has vital significances in the civilian and military fields. In this paper, a new coherent Rayleigh scattering method to measure underwater temperature is presented. A wide-band photodetector is used to receive the heterodyne signal combined by the local oscillator laser and water Rayleigh backscattering light, and the water Rayleigh scattering spectrum can be acquired by transform analysis, then the water temperature can be obtained. Firstly, theoretical study and simulation are made on the basic principles of measuring underwater temperature based on Rayleigh scattering. Secondly, theoretical analysis and simulation are made on how to measure water Rayleigh scattering spectrum using coherent detection. Finally, the water temperature measurement accuracy based on Rayleigh scattering is analyzed, showing that for 1 MHz measurement accuracy of Rayleigh scattering spectrum half-width, about 0.35 K temperature measurement accuracy can be achieved.