We study the stochastic behavior of a single self-exciting model neuron with additive noise, a system that has bistable stochastic dynamics. We use Langevin and Fokker-Planck equations to obtain analytical expressions for the stationary distribution of activities and for the crossing rate between two stable states. We adjust the parameters in these expressions to fit observed histograms of neural activity, thus obtaining what we call an “effective single neuron” for a given network. We construct an effective single neuron from an activity histogram of a representative hidden neuron in a recurrent learning network. We also compare our result with an effective single neuron previously obtained analytically through the adiabatic elimination approximation.