## Abstract

Divisive normalization has been proposed as a nonlinear redundancy reduction mechanism capturing contrast correlations. Its basic function is a radial rescaling of the population response. Because of the saturation of divisive normalization, however, it is impossible to achieve a fully independent representation. In this letter, we derive an analytical upper bound on the inevitable residual redundancy of any saturating radial rescaling mechanism.

## 1. Introduction

This simple but elegant mechanism is so apt in capturing the behavior of neurons throughout the brain that it has rececently been termed *canonical computation* (Carandini & Heeger, 2011). One possible computational goal of this widespread mechanism could be the reduction of redundancies among neural responses to natural signals in accordance with Barlow's idea of redundancy exploitation (Barlow, 1961, 2002) as demonstrated in the seminal paper by Schwartz and Simoncelli (2001).

*L*-spherically symmetric distributions, which provide a good fit to the joint filter responses

_{p}**y**on natural images, there is even a unique and optimal radial rescaling mechanism called

*radial factorization*(for general

*p*) or

*radial gaussianization*(for

*p*=2) (Sinz & Bethge, 2009, 2013; Lyu & Simoncelli, 2009; Sinz, Simoncelli, & Bethge, 2009). The basic underlying mechanism is to map the radial distribution of the filter reponses

**y**into the radial distribution of a

*p*-generalized normal distribution with independent marginals via , where denotes the cumulative distribution function of the respective probability density (Sinz, Gerwinn, & Bethge, 2009; see also Figure 1a). Since these distributions have infinite support, full redundancy reduction can be achieved through radial rescaling mechanism only if it does not saturate but maps onto the entire positive real axis (Lyu & Simoncelli, 2009). Most divisive normalization mechanisms as well as real neurons, however, do saturate for large . Thus, an important issue is how critical this principal limitation of saturating radial rescaling mechanisms is.

*p*-generalized normal. Here,

*h*[

*Z*] and

_{i}*h*[

**Z**] denote the marginal and the joint differential Shannon entropy, respectively. The truncation threshold will be denoted by , as it is this parameter in the divisive normalization model (see equation 1.1) that determines the saturation threshold. We show numerically that the multi-information rate is a decreasing function of and then address the two limiting cases and analytically. The limit of the latter case is simply the

*p*-generalized normal without truncation and thus has independent marginals (Sinz, Gerwinn, & Bethge, 2009). For , the limiting case is the multi-information of the uniform distribution within the

*L*-unit sphere because for the

_{p}*p*-generalized normal distribution

*N*(

_{p}**x**). The main contribution of this letter is an analytic expression for this case, which provides a useful upper bound on the minimal multi-information that can be achieved with a saturating radial rescaling mechanism.

## 2. Analytical Results

*L*-unit ball , we first compute the entropy of its univariate marginal. The marginal densities of the uniform distribution belong to the family of -distributions (Sinz & Bethge, 2010). The expression for the marginal results from solving the integral in the general formula for the marginal distribution of

_{p}*L*-spherically symmetric distributions (Gupta & Song, 1997), where denotes the gamma function.

_{p}*L*-unit ball is computed via its volume

_{p}*V*, with (Gupta & Song, 1997).

## 3. Discussion

After whitening, natural image patches can be well modeled by *L _{p}*-spherically symmetric distributions (Sinz & Bethge, 2009). These can be transformed into factorial distribution by a nonlinear radial rescaling similar to divisive normalization (see Figure 1a). Since the radial rescaling of divisive normalization saturates at , it cannot achieve full redundancy reduction. Numerical computations show that the multi-information rate of a radially truncated

*p*-generalized normal distribution is monotonicly decreasing with the truncation threshold (see Figure 1b). Therefore, the limiting case provides an upper bound on the information rate for arbitrary radially truncated

*p*-generalized normal distributions. This upper bound is given by the multi-information rate of the uniform distribution within the

*L*-unit ball that we derived here. It turns out that the upper bound is quite low compared to a lower bound on the multi-information rate of natural images of nats/pixel (Hosseini, Sinz, & Bethge, 2010; see also Figure 1c). This means that the dependencies due to radial truncation are negligible compared to the dependencies present in unnormalized natural images. Therefore, the multi-information of the uniform distribution on the

_{p}*L*-unit ball can serve as a meaningful lower bound on the redundancy reduction that radial rescaling mechanisms should be able to achieve at least.

_{p}## Acknowledgments

This work was supported by the Bernstein Center for Computational Neuroscience (FKZ 01GQ1002) and the German Excellency Initiative through the Centre for Integrative Neuroscience Tübingen (EXC307). Fabian Sinz wants to thank Oleksandr Pavlyk for helpful discussions on generalized hypergeometric functions.