In a pioneering classic, Warren McCulloch and Walter Pitts proposed a model of the central nervous system. Motivated by EEG recordings of normal brain activity, Chvátal and Goldsmith asked whether these dynamical systems can be engineered to produce trajectories that are irregular, disorderly, and apparently unpredictable. We show that they cannot build weak pseudorandom functions.
Electroencephalogram recordings of normal brain (or of an epileptic brain well before a seizure) are usually irregular and disorderly, with no apparent pattern (Liu, Hahn, Heldt, & Coen, 1992; Lehnertz et al., 2001; Da Silva et al., 2003; Iasemidis, Shiau, Sackellares, Pardalos, & Prasad, 2004; Chaovalitwongse, 2009; Ocak, 2009; Altunay, Telatar, & Erogul, 2010). Chvátal and Goldsmith (2012) asked whether the McCulloch-Pitts model of the brain can be engineered to exhibit similar behavior. The same question, although without its physiological interpretation, was also asked in Elyada and Horn (2005). We begin by briefly describing the McCulloch-Pitts model.
Chvátal and Goldsmith (2012) asked whether these dynamical systems can produce trajectories that are irregular, disorderly, and apparently unpredictable in the sense of generating random numbers. In making the meaning of their question precise, they took the point of view of the practitioners, who mean by a “random number generator” any deterministic algorithm that, given a short sequence of numbers called a seed, returns a longer sequence of numbers; such a random number generator is considered to be good if it passes statistical tests from some commonly agreed on battery. (This point of view is expounded in Knuth, 1998.)
In this note, we take the point of view of the theorists: we are going to prove that McCulloch-Pitts dynamical systems cannot produce trajectories that are irregular, disorderly, and apparently unpredictable in the sense of providing weak pseudorandom functions. These have been introduced in Naor and Reingold (1995) and subsume pseudorandom functions, introduced in Goldreich, Goldwasser, and Micali (1986) under the original name of poly-random collections. Roughly speaking, a weak pseudorandom function is a probability distribution on a set Fn of functions from to with the following property: if are chosen independently and uniformly at random from , then no polynomial-time randomized algorithm can distinguish with a nonnegligible probability between (1) a sequence where f is chosen at random from Fn and (2) a sequence where are chosen independently and uniformly at random from . (Distinguishing between these two is a trivial matter when f is known, and that is why an unknown f must be drawn from a probability distribution on Fn.) Our result shows that weak pseudorandom functions cannot be built from McCulloch-Pitts dynamical systems:
There is a polynomial-time deterministic algorithm that, given a sequence of n-bit vectors, returns either the message - or the message - in such a way that
If for some McCulloch-Pitts dynamical system , then the algorithm returns -.
If are chosen independently and uniformly at random from , if are chosen independently and uniformly at random from , and if for some positive constant , then the algorithm returns - with probability at least , where is a positive constant depending only on .
Linearly separable dichotomies of m-point subsets of can be recognized in time polynomial in m and n.
For every positive , there is a positive with the following property: if is a finite subset of such that , then a dichotomy chosen uniformly at random from all dichotomies of X is linearly separable with probability at most .
Following the seminal report (Rosenblatt, 1957), the subject of learning a hyperplane that separates, or at least nearly separates, the two parts of a dichotomy received much attention in the machine learning community. None of it is relevant to the following standard argument, implicit in the linear programming proof of Minkowski’s separating hyperplane theorem for convex polytopes (Tucker, 1955).
Deciding whether a prescribed dichotomy of an m-point subset of is linearly separable amounts to solving system (1) of m strict linear inequalities in variables , where each coefficient xj is 0 or 1; the epoch-making result of Khachiyan (1979) guarantees that this can be done in time polynomial in m and n.
Without loss of generality, we may assume that . Let m denote and let p denote the probability that a dichotomy chosen uniformly at random from all dichotomies of X is linearly separable.
An alternative proof of lemma 3, proposed by one of the reviewers, relies on the Sauer-Shelah lemma (Sauer, 1972; Shelah, 1972): If a family of subsets of an m-point set has Vapnik-Chervonenkis dimension d, then it includes at most sets. Its other ingredient is the following corollary of Radon’s theorem (Radon, 1921): If is a family of half-spaces in and if X is a finite subset of , then family has Vapnik-Chervonenkis dimension at most . Putting the two together, we conclude that X has at most linearly separable dichotomies. This upper bound, although weaker than our , also yields the lemma’s conclusion.
Lemma 2 guarantees that the algorithm can be implemented to run in polynomial time.
To prove assertion i, assume that for some McCulloch-Pitts dynamical system defined by . Now for all , which means that f1 takes value 1 on all points of and value 0 on all points of ; since f1 is a threshold function, the dichotomy is linearly separable, and so the algorithm returns McCulloch-Pitts.
Theorem 1 implies that certain simple devices (namely, McCulloch-Pitts dynamical systems) cannot generate pseudorandomness. In the opposite direction, it has been proved that certain simple devices can generate pseudorandomness: examples can be found in Naor, Pinkas, and Reingold (1999), Krause and Lucks (2001), Nielsen (2002), Naor and Reingold (2004), and Applebaum, Ishai, and Kushilevitz (2010).
The question of whether McCulloch-Pitts networks can produce trajectories that are irregular, disorderly, and apparently unpredictable remains open: all depends on the interpretation of the terms irregular, disorderly, and apparently unpredictable. When clinical neurologists visually inspect an electroencephalogram, their vague criteria for declaring it random-like are a far cry from the distinguishers that cryptographers use to separate deterministic sequences from random sequences. As Avi Wigderson (2009, p. 6) put it, “Randomness is in the eye of the beholder, or more precisely, in its computational capabilities.A phenomenon (be it natural or artificial) is deemed “random enough,” or pseudorandom, if the class of observers/applications we care about cannot distinguish it from random!”
Many examples of generators that appear random to observers with restricted computational powers are known. In particular, pseudorandom generators for polynomial size constant-depth circuits have been constructed in Ajtai and Wigderson (1985); later, this work was greatly simplified and improved in Nisan (1991). O’Connor (1988) proved that an infinite binary sequence appears random to all finite-state machines if and only if it is -distributed. Pseudorandom generators for space-bounded computation have been constructed in Nisan (1992). It is conceivable that McCulloch-Pitts dynamical systems could fool neurologists into finding their trajectories unpredictable just as they find normal electroencephalograms unpredictable. Proving this in a formal setting with a suitable definition of neurologists is an interesting challenge.
A variation on our theme comes from the idea that in a brain of n neurons, only m neurons may be visible to the observer and the remaining n−m are hidden from view. Formally, given positive integers such that and given a McCulloch-Pitts dynamical system , we may consider the mapping such that is the m-bit prefix of . Can such mappings provide pseudorandomness? Our theorem 1 shows that the answer is negative when ; one of the reviewers argued that under the usual assumption that one-way functions exist, the answer is close to affirmative when . Here is the argument. Every one-way function f (as every Boolean function) can be computed by a threshold circuit (Parberry, 1994). When this circuit has n gates and depth d, it can be embedded in a McCulloch-Pitts dynamical system , where the results of its computation show up with the time delay of d units. Now f is represented in the d-fold iteration of and there is an appropriate projection , the hard-core bit (Goldreich & Levin, 1989), such that the sequence , , , … is pseudorandom.
Statistical properties of have been studied in Goldsmith (2015). For instance, there is a McCulloch-Pitts dynamical system such that the restriction of the trajectory of on the first bit passes all 10 statistical tests of the battery implemented in the software library TestU01 of L’Ecuyer and Simard (2007, 2009).
This research was undertaken in part with funding from the Canada Research Chairs program and from the Natural Sciences and Engineering Research Council of Canada. We are grateful to Péter Gács for helpful comments on a draft of this note and to Avi Wigderson for telling us about Nisan’s papers (Nisan, 1991, 1992). We also thank the two anonymous reviewers for their thoughtful comments that helped us improve the presentation considerably.