## Abstract

In a pioneering classic, Warren McCulloch and Walter Pitts proposed a model of the central nervous system. Motivated by EEG recordings of normal brain activity, Chvátal and Goldsmith asked whether these dynamical systems can be engineered to produce trajectories that are irregular, disorderly, and apparently unpredictable. We show that they cannot build weak pseudorandom functions.

Electroencephalogram recordings of normal brain (or of an epileptic brain well before a seizure) are usually irregular and disorderly, with no apparent pattern (Liu, Hahn, Heldt, & Coen, 1992; Lehnertz et al., 2001; Da Silva et al., 2003; Iasemidis, Shiau, Sackellares, Pardalos, & Prasad, 2004; Chaovalitwongse, 2009; Ocak, 2009; Altunay, Telatar, & Erogul, 2010). Chvátal and Goldsmith (2012) asked whether the McCulloch-Pitts model of the brain can be engineered to exhibit similar behavior. The same question, although without its physiological interpretation, was also asked in Elyada and Horn (2005). We begin by briefly describing the McCulloch-Pitts model.

*H*is the Heaviside step function defined by for all nonnegative

*d*and for all negative

*d*. Warren McCulloch and Walter Pitts (1943) proposed a model of the central nervous system built from linear threshold functions. When this system has

*n*neurons and no peripheral afferents, its McCulloch-Pitts model is a mapping defined by for some linear threshold functions . We will refer to such mappings as

*McCulloch-Pitts dynamical systems*.

Chvátal and Goldsmith (2012) asked whether these dynamical systems can produce trajectories that are irregular, disorderly, and apparently unpredictable in the sense of generating random numbers. In making the meaning of their question precise, they took the point of view of the practitioners, who mean by a “random number generator” any deterministic algorithm that, given a short sequence of numbers called a *seed*, returns a longer sequence of numbers; such a random number generator is considered to be good if it passes statistical tests from some commonly agreed on battery. (This point of view is expounded in Knuth, 1998.)

In this note, we take the point of view of the theorists: we are going to prove that McCulloch-Pitts dynamical systems cannot produce trajectories that are irregular, disorderly, and apparently unpredictable in the sense of providing weak pseudorandom functions. These have been introduced in Naor and Reingold (1995) and subsume pseudorandom functions, introduced in Goldreich, Goldwasser, and Micali (1986) under the original name of *poly-random collections*. Roughly speaking, a weak pseudorandom function is a probability distribution on a set *F _{n}* of functions from to with the following property: if are chosen independently and uniformly at random from , then no polynomial-time randomized algorithm can distinguish with a nonnegligible probability between (1) a sequence where

*f*is chosen at random from

*F*and (2) a sequence where are chosen independently and uniformly at random from . (Distinguishing between these two is a trivial matter when

_{n}*f*is known, and that is why an unknown

*f*must be drawn from a probability distribution on

*F*.) Our result shows that weak pseudorandom functions cannot be built from McCulloch-Pitts dynamical systems:

_{n}There is a polynomial-time deterministic algorithm that, given a sequence of *n*-bit vectors, returns either the message - or the message - in such a way that

If for some McCulloch-Pitts dynamical system , then the algorithm returns -.

If are chosen independently and uniformly at random from , if are chosen independently and uniformly at random from , and if for some positive constant , then the algorithm returns - with probability at least , where is a positive constant depending only on .

*X*is its partition into two disjoint sets. Unlike Cover (1965), for whom a dichotomy is an unordered pair of sets, we view every dichotomy as an ordered pair of sets. A dichotomy of a subset of is linearly separable if there are numbers such that When

*f*is a function from to and are points in , the dichotomy is linearly separable if and only if

*f*is a threshold function. Our proof of theorem

^{1}evolves from the propositions that linearly separable dichotomies are easy to recognize and linearly separable dichotomies are rare:

Linearly separable dichotomies of *m*-point subsets of can be recognized in time polynomial in *m* and *n*.

For every positive , there is a positive with the following property: if is a finite subset of such that , then a dichotomy chosen uniformly at random from all dichotomies of *X* is linearly separable with probability at most .

Following the seminal report (Rosenblatt, 1957), the subject of learning a hyperplane that separates, or at least nearly separates, the two parts of a dichotomy received much attention in the machine learning community. None of it is relevant to the following standard argument, implicit in the linear programming proof of Minkowski’s separating hyperplane theorem for convex polytopes (Tucker, 1955).

Deciding whether a prescribed dichotomy of an *m*-point subset of is linearly separable amounts to solving system (1) of *m* strict linear inequalities in variables , where each coefficient *x _{j}* is 0 or 1; the epoch-making result of Khachiyan (1979) guarantees that this can be done in time polynomial in

*m*and

*n*.

Without loss of generality, we may assume that . Let *m* denote and let *p* denote the probability that a dichotomy chosen uniformly at random from all dichotomies of *X* is linearly separable.

*X*, at most are linearly separable (this is at least implicit in Winder, 1966, and Cover, 1965), and so Since and , we have . A special case of the well-known bound on the tail of the binomial distribution (see, e.g., Hoeffding, 1963, theorem

^{1}) guarantees that for every positive smaller than 0.5, there is a positive such that Setting , we conclude that , which proves the lemma.

An alternative proof of lemma ^{3}, proposed by one of the reviewers, relies on the Sauer-Shelah lemma (Sauer, 1972; Shelah, 1972): *If a family of subsets of an m-point set has Vapnik-Chervonenkis dimension d, then it includes at most sets.* Its other ingredient is the following corollary of Radon’s theorem (Radon, 1921):

*If is a family of half-spaces in and if*Putting the two together, we conclude that

*X*is a finite subset of , then family has Vapnik-Chervonenkis dimension at most .*X*has at most linearly separable dichotomies. This upper bound, although weaker than our , also yields the lemma’s conclusion.

*y*and define If this dichotomy is linearly separable, then return

^{i}*McCulloch-Pitts*; else return

*not McCulloch-Pitts*.

Lemma ^{2} guarantees that the algorithm can be implemented to run in polynomial time.

To prove assertion i, assume that for some McCulloch-Pitts dynamical system defined by . Now for all , which means that *f*_{1} takes value 1 on all points of and value 0 on all points of ; since *f*_{1} is a threshold function, the dichotomy is linearly separable, and so the algorithm returns *McCulloch-Pitts*.

*not McCulloch-Pitts*increases as

*m*increases, we may replace the assumption that by the assumption that . Write . Since are chosen independently and uniformly from , they are pairwise distinct with probability . Since this probability is at least . When , the assumption that are chosen independently and uniformly from implies that the dichotomy of

*X*is chosen uniformly from all dichotomies of

*X*, in which case lemma

^{3}guarantees that is linearly separable with probability at most for some positive constant depending only on . We conclude that the algorithm returns

*not McCulloch-Pitts*with probability at least , which is at least for some positive constant depending only on .

^{1}: with standing for the

*j*th bit of

*y*, test each of the

^{i}*n*dichotomies, and return

*McCulloch-Pitts*if and only if all

*n*of them are linearly separable. In the context of distinguishing McCulloch-Pitts functions from truly random functions, the extra work required in this refinement is pointless. The probability of returning

*McCulloch-Pitts*when are chosen independently and uniformly at random from is at most in the original version, and that is good enough. Reducing it further to in the refinement is nice but unnecessary. In addition, the assumption cannot be significantly relaxed even in the refinement: it is at least implicit in Winder (1966) and Cover (1965) that a dichotomy chosen uniformly at random from all dichotomies of a set of fewer than points in is linearly separable with probability at least .

Theorem ^{1} implies that certain simple devices (namely, McCulloch-Pitts dynamical systems) cannot generate pseudorandomness. In the opposite direction, it has been proved that certain simple devices can generate pseudorandomness: examples can be found in Naor, Pinkas, and Reingold (1999), Krause and Lucks (2001), Nielsen (2002), Naor and Reingold (2004), and Applebaum, Ishai, and Kushilevitz (2010).

The question of whether McCulloch-Pitts networks can produce trajectories that are irregular, disorderly, and apparently unpredictable remains open: all depends on the interpretation of the terms *irregular, disorderly, and apparently unpredictable*. When clinical neurologists visually inspect an electroencephalogram, their vague criteria for declaring it random-like are a far cry from the distinguishers that cryptographers use to separate deterministic sequences from random sequences. As Avi Wigderson (2009, p. 6) put it, “Randomness is in the eye of the beholder, or more precisely, in its computational capabilities.A phenomenon (be it natural or artificial) is deemed “random enough,” or pseudorandom, if the class of observers/applications we care about cannot distinguish it from random!”

Many examples of generators that appear random to observers with restricted computational powers are known. In particular, pseudorandom generators for polynomial size constant-depth circuits have been constructed in Ajtai and Wigderson (1985); later, this work was greatly simplified and improved in Nisan (1991). O’Connor (1988) proved that an infinite binary sequence appears random to all finite-state machines if and only if it is -distributed. Pseudorandom generators for space-bounded computation have been constructed in Nisan (1992). It is conceivable that McCulloch-Pitts dynamical systems could fool neurologists into finding their trajectories unpredictable just as they find normal electroencephalograms unpredictable. Proving this in a formal setting with a suitable definition of *neurologists* is an interesting challenge.

A variation on our theme comes from the idea that in a brain of *n* neurons, only *m* neurons may be visible to the observer and the remaining *n*−*m* are hidden from view. Formally, given positive integers such that and given a McCulloch-Pitts dynamical system , we may consider the mapping such that is the *m*-bit prefix of . Can such mappings provide pseudorandomness? Our theorem ^{1} shows that the answer is negative when ; one of the reviewers argued that under the usual assumption that one-way functions exist, the answer is close to affirmative when . Here is the argument. Every one-way function *f* (as every Boolean function) can be computed by a threshold circuit (Parberry, 1994). When this circuit has *n* gates and depth *d*, it can be embedded in a McCulloch-Pitts dynamical system , where the results of its computation show up with the time delay of *d* units. Now *f* is represented in the *d*-fold iteration of and there is an appropriate projection , the hard-core bit (Goldreich & Levin, 1989), such that the sequence , , , … is pseudorandom.

Statistical properties of have been studied in Goldsmith (2015). For instance, there is a McCulloch-Pitts dynamical system such that the restriction of the trajectory of on the first bit passes all 10 statistical tests of the battery implemented in the software library TestU01 of L’Ecuyer and Simard (2007, 2009).

## Acknowledgments

This research was undertaken in part with funding from the Canada Research Chairs program and from the Natural Sciences and Engineering Research Council of Canada. We are grateful to Péter Gács for helpful comments on a draft of this note and to Avi Wigderson for telling us about Nisan’s papers (Nisan, 1991, 1992). We also thank the two anonymous reviewers for their thoughtful comments that helped us improve the presentation considerably.

## References

*TC*

^{0}and cryptographic limitations to proving lower bounds

*N*-space by hyperplanes