The connection between robust statistical estimates and nonsmooth optimization is established. Based on the resulting family of optimization problems, robust learning problem formulations with regularization-based control on the model complexity of the multilayer perceptron network are described and analyzed. Numerical experiments for simulated regression problems are conducted, and new strategies for determining the regularization coefficient are proposed and evaluated.