In this paper the evolution of the radiation field from frequency bifurcation to chaos and the temporal and frequency characteristics of the nonlinear unsteady states are investigated, based on the threshold analysis of nonlinear instability for the radiation field in a traveling wave tube amplifier. In the soft nonlinear regime,the nonlinear radiation field is characterized by limit cycle oscillations with frequency bifurcated, and the output power spectrum is discrete and asymmetric around the carrier frequency. This instability is intermittent so that the device can operate in expected steady states or limit cycle oscillation states by slightly adjusting the parameter. In the hard nonlinear regime in which the radiation field is characterized by randomly non-periodic oscillations with frequency chaos, the output power spectrum is continuous and wide, the components with a larger amplitude are restricted in the low-frequency range near zero. This instability is continuous so that it cannot be avoided by slightly adjusting the parameter.