Abstract

The leaky integrator, the basis for many neuronal models, possesses a negative group delay when a time-delayed recurrent inhibition is added to it. By means of this delay, the leaky integrator becomes a predictor for some frequency components of the input signal. The prediction properties are derived analytically, and an application to a local field potential is provided.

In “How Delays Affect Neural Dynamics and Learning” Baldi and Atiya (1994) state, “Integration and communication delays are ubiquitous, both in biological and man-made neural systems… . Indeed, delays should be considered as an additional media through which evolution, or skilled engineers, can achieve particular dynamical effects.” In fact, time delays have an impact on the dynamics of neuronal networks, for example, by causing oscillations and waves (Crook, Ermentrout, & Vanier, 1997). In this note, I point out how time delays added to leaky integrators are defining predictors for smooth input signals. The underlying mechanism is based on negative group delay (NGD) and to the best of my knowledge has not been used in the neurosciences so far.

The basic model is a leaky integrator with a recurrent, time-delayed inhibition. The leaky integrator is defined as usual as a capacitance, the integrator, in parallel to a resistance, the leak (Stein, 1965; Schiff, 2011). The recurrent inhibition is modeled as a linear time-delayed feedback term with negative gain (Plant, 1981). The leaky integrator with recurrent inhibition follows as
formula
1
where is the leakage coefficient, the input signal (zero-mean, generated by a stationary process), the input scaling, the (inhibitory) feedback gain, and a time delay. For c = 0, equation 1 would simply be a leaky integrator, but for , it has an inhibitory feedback that enters the model as a delayed leak. Therefore, model 1 is referred to as a delayed-leak integrator (DLI). For a = 0 and , it describes a pure DLI without a conventional leak, which would have similar properties as the DLI with but is not further considered here.
Equation 1 is linear and thus can be described by its frequency response function,
formula
2
with , , (Munakata, Iwama, & Kimizuka, 2009; Voss, 2016). It defines the steady-state input-output relationship between x and y in Fourier space as , where f is frequency, , , and . If written as , its gain is , and its phase is . The frequency-dependent group delay is
formula
3
It can be positive, zero, or negative. Negative group delay in general means a group advance, or real-time prediction of the input signal (Mitchell & Chiao, 1997; Nakanishi, Sugiyama, & Kitano, 2002). To characterize the group delay for low-frequency signal components, is expanded for small Neglecting quadratic and higher-order terms in the counter and denominator of the expansion, it follows that
formula
4
This result has two important consequences for input signal components with small :
  1. For , the group delay is negative, a necessary condition for prediction.

  2. The group delay is approximately independent of , a necessary condition for distortion-free signal transfer (Kandic & Bridges, 2013).

These ideas were applied to a local field potential (LFP) from the left hippocampus (CA1) of a rat. The data consisted of the first 80 s of the “hc-5” set from CRCNS.org (Pastalkova, Itskov, Amarasingham, & Buzsaki, 2008). The input x was defined as the average over all electrodes, normalized, and slightly low-pass-filtered (cutoff at 27 Hz). The parameters and were estimated from a fit to the first 5 s of the data set with a simplex search algorithm and then used to model 16 contiguous intervals of 5 s each (). Equation 1 was solved with a Runge-Kutta scheme with ms and for . All computations were performed with Matlab R2015a (MathWorks, Inc., Natick, MA).

In Figure 1A, 1.4 s of the input x and corresponding output y are shown. It is evident that the DLI output y at time t (red) predicts the LFP input x at a time (black) on average. The group delay is −16.2 ms (see Figure 1B, red dashed line shows ). Figure 1B also depicts a section of the estimated and analytic phase and gain of the frequency response function, including the first interval with NGD. Therefore, frequency components of x within this interval are predicted by y. More specifically, the cross-correlation function (CCF) between x and y (see Figure 1C) has a global maximum of 0.81 at ms. This result is reproducible. Out of the 16 data sections, 11 yielded a CCF with a global maximum between and −8.8 ms. The importance of signal frequency content is demonstrated in Figure 1D; spectral components near the resonance of the frequency response function are amplified and become detrimental to prediction if dominating the signal.

Figure 1:

Simulation of the DLI system with experimental input data. Refer to the text for a detailed description.

Figure 1:

Simulation of the DLI system with experimental input data. Refer to the text for a detailed description.

The real-time prediction of the LFP does not violate causality but follows from the delay-induced NGD (Voss, 2016) of the DLI, equation 1. Prediction performance depends on model parameters and spectral properties of the data. For improper conditions the DLI might not predict or cause oscillatory instabilities (Bernard, Belair, & Mackey, 2001). Very recently it has been emphasized in this journal that anticipatory systems can defy the inference of the direction of information flow from data (Schumacher, Wunderle, Fries, Jakel, & Pipa, 2015). It would be interesting if this holds true for NGD systems, too (Voss, 2000; Pethel, Corron, Underwood, & Myneni, 2003; Blakely, Pruitt, & Corron, 2008). DLIs might augment the related concept of neuronal anticipatory synchronization (Calvo, Chialvo, Eguiluz, Mirasso, & Toral, 2004; Ciszak, Marino, Toral, & Balle, 2004; Pyragiene & Pyragas, 2013; Sausedo-Solorio & Pisarchik, 2014), which recently has been used to explain observations in brain dynamics (Matias et al., 2014). Note that DLIs do not require a memory of past signal values, only of past predicted, already internalized states, as equation 1 does not contain delayed inputs. It would be worth investigating how NGD systems fit into general theories of prediction (Crutchfield, 2012) or how they perform as predictors in artificial neuronal networks. Since delay-induced NGD does not depend on a specific model for the signal, it is quite conceivable that biological neuronal networks might use this simple mechanism for real-time prediction, such that Baldi and Atiya’s insights could be corroborated once again.

References

Baldi
,
P.
, &
Atiya
,
A. F.
(
1994
).
How delays affect neural dynamics and learning
.
IEEE Transactions on Neural Networks
,
5
(
4
),
612
621
.
Bernard
,
S.
,
Belair
,
J.
, &
Mackey
,
M. C.
(
2001
).
Sufficient conditions for stability of linear differential equations with distributed delay
.
Discrete and Continuous Dynamical Systems—Series B
,
1
(
2
),
233
256
.
Blakely
,
J. N.
,
Pruitt
,
M. W.
, &
Corron
,
N. J.
(
2008
).
Time shifts and correlations in synchronized chaos
.
Chaos
,
18
(
1
),
013117
.
Calvo
,
O.
,
Chialvo
,
D. R.
,
Eguiluz
,
V. M.
,
Mirasso
,
C.
, &
Toral
,
R.
(
2004
).
Anticipated synchronization: A metaphorical linear view
.
Chaos
,
14
(
1
),
7
13
.
Ciszak
,
M.
,
Marino
,
F.
,
Toral
,
R.
, &
Balle
,
S.
(
2004
).
Dynamical mechanism of anticipating synchronization in excitable systems
.
Physical Review Letters
,
93
(
11
),
114102
.
Crook
,
S. M.
,
Ermentrout
,
G. B.
, &
Vanier
,
M. C.
(
1997
).
The role of axonal delay in the synchronization of networks of coupled cortical oscillators
.
Journal of Computational Neuroscience
,
4
(
2
),
161
172
.
Crutchfield
,
J. P.
(
2012
).
Between order and chaos
.
Nature Physics
,
8
(
1
),
17
24
.
Kandic
,
M.
, &
Bridges
,
G. E.
(
2013
).
Limits of negative group delay phenomenon in linear causal media
.
Progress in Electromagnetics Research
,
134
,
227
246
.
Matias
,
F. S.
,
Gollo
,
L. L.
,
Carelli
,
P. V.
,
Bressler
,
S. L.
,
Copelli
,
M.
, &
Mirasso
,
C. R.
(
2014
).
Modeling positive Granger causality and negative phase lag between cortical areas
.
NeuroImage
,
99
,
411
418
.
Mitchell
,
M. W.
, &
Chiao
,
R. Y.
(
1997
).
Negative group delay and ”fronts” in a causal system: An experiment with very low frequency bandpass amplifiers
.
Physics Letters A
,
230
(
3–4
),
133
138
.
Munakata
,
T.
,
Iwama
,
S.
, &
Kimizuka
,
M.
(
2009
).
Linear stochastic system with delay: Energy balance and entropy production
.
Physical Review E
,
79
(
3
),
031104
.
Nakanishi
,
T.
,
Sugiyama
,
K.
, &
Kitano
,
M.
(
2002
).
Demonstration of negative group delays in a simple electronic circuit
.
American Journal of Physics
,
70
(
11
),
1117
1121
.
Pastalkova
,
E.
,
Itskov
,
V.
,
Amarasingham
,
A.
, &
Buzsaki
,
G.
(
2008
).
Internally generated cell assembly sequences in the rat hippocampus
.
Science
,
321
(
5894
),
1322
1327
.
Pethel
,
S. D.
,
Corron
,
N. J.
,
Underwood
,
Q. R.
, &
Myneni
,
K.
(
2003
).
Information flow in chaos synchronization: Fundamental tradeoffs in precision, delay, and anticipation
.
Physical Review Letters
,
90
(
25
),
254101
.
Plant
,
R. E.
(
1981
).
A Fitzhugh differential-difference equation modeling recurrent neural feedback
.
SIAM Journal on Applied Mathematics
,
40
(
1
),
150
162
.
Pyragiene
,
T.
, &
Pyragas
,
K.
(
2013
).
Anticipating spike synchronization in nonidentical chaotic neurons
.
Nonlinear Dynamics
,
74
(
1–2
),
297
306
.
Sausedo-Solorio
,
J. M.
, &
Pisarchik
,
A. N.
(
2014
).
Synchronization of map-based neurons with memory and synaptic delay
.
Physics Letters A
,
378
(
30–31
),
2108
2112
.
Schiff
,
S. J.
(
2011
).
Neural control engineering: The emerging intersection between control theory and neuroscience
.
Cambridge, MA
:
MIT Press
.
Schumacher
,
J.
,
Wunderle
,
T.
,
Fries
,
P.
,
Jakel
,
F.
, &
Pipa
,
G.
(
2015
).
A statistical framework to infer delay and direction of information flow from measurements of complex systems
.
Neural Computation
,
27
(
8
),
1555
1608
.
Stein
,
R. B.
(
1965
).
A theoretical analysis of neuronal variability
.
Biophysical Journal
,
5
(
2
),
173
194
.
Voss
,
H. U.
(
2000
).
Anticipating chaotic synchronization
.
Physical Review E
,
61
(
5
),
5115
5119
.
Voss
,
H. U.
(
2016
).
Signal prediction by anticipatory relaxation dynamics
.
Physical Review E
,
93
,
030201(R)
.