Abstract

A significant feature of spiking neural networks with varying connection delays, such as those in the brain, is the existence of strongly connected groups of neurons known as polychronous neural groups (PNGs). Polychronous groups are found in large numbers in these networks and are proposed by Izhikevich (2006a) to provide a neural basis for representation and memory. When exposed to a familiar stimulus, spiking neural networks produce consistencies in the spiking output data that are the hallmarks of PNG activation. Previous methods for studying the PNG activation response to stimuli have been limited by the template-based methods used to identify PNG activation. In this letter, we outline a new method that overcomes these difficulties by establishing for the first time a probabilistic interpretation of PNG activation. We then demonstrate the use of this method by investigating the claim that PNGs might provide the foundation of a representational system.

1.  Introduction

Spiking neural networks are connected networks of nodes that exchange messages across their connections in the form of temporally discrete firing events called spikes. The significance of these spiking messages is determined by the weight of the connection bearing each message: messages borne on strong connections carry greater significance than those borne on connections with lesser weight. The nodes in a spiking neural network are model neurons that receive spiking messages on their input connections, evaluate their significance, and optionally produce a spike on their output connections. Message evaluation involves the integration of messages over a limited time frame, with the significance of individual messages decreasing exponentially the moment they are received. If the combined significance of recent messages reaches a threshold, the neuron fires, producing an output spike that provides an input message to other neurons.

The pattern of messages flowing through the network has both spatial and temporal components: each neuron receives input messages from many neighboring neurons distributed across the network space, and these messages arrive at the neuron at discrete times. Each neuron therefore sees distinct spatiotemporal stimuli across its many input connections. The network parameters in a simulated network are typically chosen such that a neuron requires at least two nearly simultaneous inputs in order to fire. Incoming spikes from neighboring neurons must therefore converge to produce sufficient combined input for the neuron to exceed its firing threshold.

If the connections to neighboring neurons are of variable length, the precise firing time of each of the neighboring neurons interacts with the transit times of each of the resulting spikes to determine whether the neuron will fire. In the network shown in Figure 1A, if neuron 1 fires at t=0 and neuron 2 fires at t=5, then the combined input from two nearly simultaneous spikes is sufficient for neuron 4 to reach the firing threshold. While neuron 3 receives the same spike messages, they do not coincide in time, and the threshold is therefore not reached. In Figure 1B, the input firing pattern is reversed: neuron 2 fires at t=0, and neuron 1 fires at t=5. The result is that convergent input arrives at neuron 3, and only neuron 3 will fire. To summarize, a neuron will reach the firing threshold only when the input connection lengths are congruent with the pattern of firing times of the input neurons. Each neuron therefore has the ability to selectively respond to spatiotemporal stimuli in which the precise timing of its input neurons matches the corresponding connection delays.

Figure 1:

The significance of variable connection lengths. Each of the four neurons in this model network requires at least two nearly simultaneous inputs in order to reach firing threshold. Neurons 1 and 2 are input neurons that provide synaptic input to output neurons 3 and 4. Due to the variable delays imposed by the different connection lengths, the precise firing times of each input neuron determine which of the two output neurons is likely to fire.

Figure 1:

The significance of variable connection lengths. Each of the four neurons in this model network requires at least two nearly simultaneous inputs in order to reach firing threshold. Neurons 1 and 2 are input neurons that provide synaptic input to output neurons 3 and 4. Due to the variable delays imposed by the different connection lengths, the precise firing times of each input neuron determine which of the two output neurons is likely to fire.

The connected structure of a spiking neural network can be viewed as a weighted digraph in which the graph vertices are replaced by neurons and the directed edges are replaced by connections. Izhikevich (2006a) has shown that certain strongly connected groups of neurons known as polychronous neural groups (PNGs) exist in large numbers within this connection graph. A defining feature of polychronous groups is that the connected subgraph that distinguishes the group forms a broad pathway of congruent connections through the network that has the potential to sustain a causal cascade of neural firing. Activation of a PNG produces polychronization, a reproducible and precisely timed sequence of firing events that is observable in the firing data generated by the network (Izhikevich, Gally, & Edelman, 2004; Izhikevich, 2006a).

1.1.  PNG Activation

Polychronous group activation requires an appropriate spatiotemporal triggering pattern that is able to match some portion of the PNG subgraph and produce convergent firing. If the resulting polychronization is to be sustained, multiple group neurons must fire at precise times over the course of activation. The interaction of the convergent connections within the group with these precisely timed firing events allows individual group neurons to reach their firing thresholds, supporting further polychronization. Although these structural PNGs provide pathways of converging connections that have the potential to support polychronization, PNG activation also requires that the connections between group members be sufficiently strong to allow the combined inputs of group neurons to pass the firing threshold (see the distinction between structural and dynamic PNGs outlined by Martinez & Paugam-Moisy, 2009). However, if we provide a training regimen that selectively strengthens the connection weights within the group, then polychronization will occur with increasing frequency over the course of training. Results from neurophysiological studies have demonstrated that spike-timing dependent plasticity (STDP) can provide such a training scheme (Markram, Lübke, Frotscher, & Sakmann, 1997; Caporale & Dan, 2008).

1.2.  Detecting PNG Activation

The unique pattern of firing events generated by each PNG activation provides a distinct activation signature that can be detected in the network firing data. In order to resolve these signatures within the massive flood of firing events generated by the network, two approaches have been used. In one approach, PNGs that are triggered by parts of the stimulus are used as spatiotemporal templates that are matched against the firing data (Izhikevich, 2006a); in the other approach, the triggering patterns that make up the stimulus are detected directly, on the assumption that the presence of the triggering pattern will entail PNG activation (Martinez & Paugam-Moisy, 2009).

One difficulty with these techniques is that in a network with recurrent connections and random background firing, PNG activation is not deterministic: the same PNG can polychronize in different ways, with variation in both the neurons that participate in each activation and in the precise time of their firing (see Guise, Knott, & Benuskova, 2013b, Figure 6). This variability is caused by perturbations in the internal dynamics of each neuron due to the integration of recent events. For example, if random or recurrent input to a PNG neuron has recently caused it to fire, the neuron may resist participating in the current activation for a small interval (the neural refractory period) or may fire with a small delay. In addition, Izhikevich et al. (2004) have noted considerable competition between polychronous groups for the affiliation of individual neurons, causing the synaptic weights to be constantly adjusted to support the activation of first one group and then the other. Methods for detecting group activation must take these variations into account, typically by reducing the number of firing events in a search template that must be matched and by incorporating a term that accounts for an allowed temporal jitter in the neural firing time. However, both of these solutions reduce the accuracy of PNG identification.

Another difficulty with current template-matching techniques is that the templates match only PNGs that are directly triggered by the stimulus. However, the activation of multiple stimulus-triggered PNGs may in turn provide triggers for additional PNG activations that are not directly triggered by the stimulus. A template-matching technique that employs only stimulus-triggered templates cannot detect these indirect PNG activations.

1.3.  A PNG-Based Representational System

Using techniques such as the template-matching method described above, Izhikevich (2006a) has used a network trained on two different stimuli to show that different groups of PNGs are activated in response to each stimulus. Paugam-Moisy, Martinez, and Bengio (2008) have extended this work with a system based on a polychronizing reservoir that successfully classifies handwritten numerals from the USPS data set. Results such as these have led Izhikevich to propose that polychronous groups might provide a neural basis for representation and memory and that the activation of a PNG is equivalent to evoking a specific representation. In this letter, we also consider the representational potential of PNGs. In this section, we first consider the criteria that a representational system must satisfy, and in following sections we introduce a general proposal for how PNGs might meet these criteria.

Minimally, if a PNG functions to represent a given stimulus, it must be activated whenever the stimulus is presented, and it must not be activated if the stimulus is not presented. We use the term consistency to refer to the requirement for activation on each stimulus presentation and selectivity to refer to the stimulus specificity of the PNG activation. The reservoir-based system of Paugam-Moisy et al. (2008) provides initial evidence for the consistency of PNG activation, as does the work of Guise, Knott, and Benuskova (2013a) using a template-matching technique. However, the issue of selectivity is less clear. For example, we have observed near identical spatiotemporal patterns of polychronization resulting from different but overlapping stimuli (see Figure 2); individual PNGs, when defined using templates, are therefore not always selective for the input stimulus.

Figure 2:

Near-identical patterns of polychronization in the PNG activation response to two different stimuli (filled circles) that fire the same neurons but in a different order. The spatiotemporal pattern of firing produced in response to an ascending pattern (open circles, left) is identical to that produced in response to a descending pattern (open circles, right) if a temporal jitter of 4 milliseconds is allowed. This subset of the PNG activation response is therefore insufficient to select the stimulus. Note that only selected portions of the stimulus and PNG activation firing events are shown.

Figure 2:

Near-identical patterns of polychronization in the PNG activation response to two different stimuli (filled circles) that fire the same neurons but in a different order. The spatiotemporal pattern of firing produced in response to an ascending pattern (open circles, left) is identical to that produced in response to a descending pattern (open circles, right) if a temporal jitter of 4 milliseconds is allowed. This subset of the PNG activation response is therefore insufficient to select the stimulus. Note that only selected portions of the stimulus and PNG activation firing events are shown.

While individual PNGs may not have the property of selectivity, both our results and those of Izhikevich (2006a) and Paugam-Moisy et al. (2008) are consistent with the idea that sets of PNG activations could be selective. This is the intuition we will pursue in this letter. It will lead to a new way of conceptualizing PNGs in which they are defined not by single templates but by sets of partially independent spatiotemporal firing events. We will show that PNGs when defined in this way can be shown to be both consistent and selective.

1.4.  The Activation Response

The PNG activations that comprise the network's response to a stimulus presentation together provide a stimulus-specific activation signature that will be referred to here as the activation response. On first exposure to a stimulus, there may be few, if any, PNG activations making up the activation response, and any activations that do occur may be partial. As we will show in section 4.1, this initially weak response becomes increasingly well defined over the course of training as the connection weights supporting PNG activation are strengthened.

As discussed in section 1.2, previous study of the activation response has used a template-based approach that allows for only a limited variability (temporal jitter) in PNG activation. In order to avoid missing significant features of the activation response, it is important that any alternative method for studying the activation response not only incorporates this temporal jitter but also includes a stochastic component that models the variable participation of PNG neurons as they respond to changes in the network dynamics. In addition, an alternative method must allow for changes in neural recruitment that occur over training and for the possibility of indirect PNG activations that are not directly caused by the stimulus. Although the template-based approach can extend to a set-based view of the PNG activation response by including multiple templates, it can see only the activations of known templates; the activation of PNGs triggered by the activation of these known templates would not be visible.

In this letter, we abandon the template-based approach in favor of a probabilistic representation of the activation response called a response fingerprint. Rather than viewing the activation response as a fixed set of PNG activations, this new representation views the activation response as a probabilistic set of firing events that are conditional on the stimulus. Unlike the existing template-based method, the response fingerprinting method can be shown to satisfy both the selectivity and consistency criteria discussed in section 1.3. In addition, this new method allows the activation response to be quantified, whereas the template-based approach is not readily quantifiable without first decomposing the set of template matches into a more finely grained set of matching firing events. The ability to quantify the PNG activation response is useful for studying many aspects of PNG activation, including the effects of stimulus degradation, the variability of the response, and the changes that occur with training (see Guise et al., 2013b, and sections 4.1 and 4.2).

2.  Response Fingerprinting

To introduce the concept of response fingerprinting, we examine the statistical behavior of the neurons involved in a stimulus-related PNG activation. A significant feature of PNG activation is the variability in the pattern of polychronization produced by the network dynamics, and the resulting mis-firings, or temporal jitter in the firing of PNG neurons. The firing of an individual PNG neuron following stimulus presentation is therefore far from certain. However, if the neuron belongs to a stimulus-related polychronous group, it will often be fired as part of the group despite the variability imposed by network dynamics. Over the course of many stimulus presentations, the firing of PNG neurons can be observed to be strongly correlated with the presence of the stimulus. In particular, the firing of the neuron at a specific time after stimulus presentation is strongly correlated with stimulus presentation.

This statistical consistency in the firing times of PNG neurons is a consequence of the causal nature of PNG activation. Figure 3 shows a schematic representation of PNG activation. The external stimulus produces firing of neurons in the input layer that produces subsequent firing of neurons in layer 1, and so on into deeper layers of the network. Given repeated stimuli, then, the firing of input layer neurons should be highly correlated with the subsequent firing of neurons in layer 1. We might also expect to see correlations between the stimulus firing events and firing events in deeper layers of the network, although these correlations should get progressively weaker due to the nondeterministic nature of polychronization.

Figure 3:

Stimulation of a trained network produces the activation of a PNG, represented here as the propagation of firing activity through multiple layers due to convergent patterns of firing. Nodes in the figure represent the firing of PNG neurons, and lines represent causal links between firing events. Activation is triggered by an external stimulus, represented as a set of three firing events with a gray background.

Figure 3:

Stimulation of a trained network produces the activation of a PNG, represented here as the propagation of firing activity through multiple layers due to convergent patterns of firing. Nodes in the figure represent the firing of PNG neurons, and lines represent causal links between firing events. Activation is triggered by an external stimulus, represented as a set of three firing events with a gray background.

The consistent firing of PNG neurons over multiple stimulus presentations can be represented as a histogram of spike counts over time, as shown in Figure 4. Consistency in the firing of PNG neurons produces peaks in the spike counts that occur at a fixed interval following stimulus presentation at t=0. While PNG neurons show peaks in the spike counts, neurons that are not part of any activated polychronous group fire at random times over the same interval and therefore fail to produce peaks. If spike count histograms are produced for each of the neurons in the network, the neurons that are involved in stimulus-related PNG activations show strong peaks, while those that do not participate in the activation response show only random spiking. For the participating neurons, the timing of each PNG-related peak in their respective histograms reflects the relative timing of the firing event produced by the neuron as it participates in PNG activation.

Figure 4:

A histogram of spike counts accumulated over a 250 millisecond response frame and drawn at two different y-axis scales. The spiking data of a single PNG neuron were collected from a trained network following repeated stimulation with a triggering stimulus at regular (250 msec) intervals. The y-axis scales are chosen to show only high-amplitude peaks in the top figure and low-amplitude peaks in the bottom figure. A single high-amplitude peak in the accumulated spike counts (top) is formed by a consistent spiking response to each repeated stimulus presentation at t = 0. The same neuron also spikes at times that are unrelated to stimulus presentation (bottom) due to random background firing and recurrent input.

Figure 4:

A histogram of spike counts accumulated over a 250 millisecond response frame and drawn at two different y-axis scales. The spiking data of a single PNG neuron were collected from a trained network following repeated stimulation with a triggering stimulus at regular (250 msec) intervals. The y-axis scales are chosen to show only high-amplitude peaks in the top figure and low-amplitude peaks in the bottom figure. A single high-amplitude peak in the accumulated spike counts (top) is formed by a consistent spiking response to each repeated stimulus presentation at t = 0. The same neuron also spikes at times that are unrelated to stimulus presentation (bottom) due to random background firing and recurrent input.

The timing of these peaks over multiple neurons produces a spatiotemporal signature that reflects the PNG activations produced by the stimulus. This unique response signature describes a statistical average over many activations of the connectivity subgraphs within the underlying polychronous groups. We can define a temporal window that overlaps each peak in the histograms, specifying a temporal range within which the majority of the spikes counts are captured. Given this framework, each presentation of the stimulus produces an activation response whose firing events are statistically likely to fall within these temporal windows.

This unique spatiotemporal signature within the statistical activation response is utilized by the response fingerprinting method to create a stimulus-specific fingerprint that can then be used for the study of the PNG activation response (for details, see Guise et al., 2013b). Each temporal window in the fingerprint specifies a small range of temporal offsets within which the probability of a spike occurring is significantly greater than average. If the majority of windows capture a spike, then it is probable that the response produced by the stimulus associated with the fingerprint is specific to the stimulus. But if few windows capture a spike, then it is unlikely that the network firing response was related to the stimulus.

The response fingerprinting method not only imposes a probabilistic view on the activation response, it also usefully allows the quantification of the nondeterministic relationship between the stimulus and the activation response. In particular, it allows us to make statements such as, “Given some spiking of selected neurons in these selected time ranges, the probability that this response was specific to the stimulus is X.” To implement this statement requires computation of a conditional probability: the probability that the response is stimulus related given the pattern of spiking in the activation response. Of particular interest is a spiking pattern that produces the activation of the stimulus-related temporal windows within the activation response, where window activation is defined as the event of one or more spikes occurring within a temporal window. Our objective in producing a response fingerprint is therefore to compute the conditional probability that the activation response is stimulus related, given the evidence of coincident spiking within a specified time frame. Using a more succinct notation:
formula

3.  Methods

3.1.  Software

An implementation of the response fingerprinting method is provided in the Spinula software package, a spiking network simulation engine for the Microsoft Windows platform (Guise, Knott, & Benuskova, 2013c). Spinula is based on the reference software provided by Izhikevich (2006b) and is composed of a set of Microsoft Windows dynamic link libraries that provide functions for network construction, execution, and analysis. These libraries can be incorporated into a user program or interactively executed using a Microsoft. Net scripting language.

3.2.  Networks

Twenty independent networks, each composed of 1000 Izhikevich neurons (800 excitatory and 200 inhibitory) and with parameters as described in Izhikevich (2006a) were generated. Weights were initialized to the values +3.0 (for excitatory weights) and −2.0 (for inhibitory weights). Each network was matured for 2 hours (simulation time) by exposure to 1 Hz random input generated by a Poisson process. Following maturation, the weights were found to be redistributed in a strongly bimodal fashion with the majority of weights at their maximum or minimum values, as is typical following random stimulation. The networks used in experiments were then generated from these matured networks as follows: 20 networks were trained on a single stimulus, 20 networks were trained on two stimuli, and 20 networks were left untrained.

3.3.  Stimuli

Internally to the Spinula software, stimuli are represented as spatiotemporal firing patterns that are repeatedly presented to the network at regular intervals. Each spatiotemporal firing pattern consists of a sequence of firing events, each of which records the firing of a specific neuron at a specific time. Two different spatiotemporal patterns were used as stimuli, composed from the same set of neurons but with a different firing order (see Izhikevich, 2006a). In each pattern, 40 neurons are arranged in an ascending sequence: 1, 21, 41, 61, , 741, 761, 781. The first pattern fires these 40 neurons at 1 millisecond intervals in the specified order and is termed the ascending pattern. The second, descending, pattern inverts the neuron sequence order, producing a step-like descending firing sequence.

The stimuli used for training each network were produced by repeating the selected pattern every 200 milliseconds, producing an overall stimulation frequency of 5 Hz. When training with a single stimulus, the selected 40-neuron pattern was presented at five times per second, while for multistimulus training, the stimulus pattern was alternated once per second, producing an alternating set of five stimulus presentations to the network in each second. While a 5 Hz stimulation frequency was used for training, the analysis of the stimulus response in trained networks used a 1 Hz stimulation frequency; analysis of the stimulus response requires that the stimulus is presented once per frame, corresponding to a 1 Hz stimulation frequency when using a 1 second response frame (see section 3.5).

In addition to the external stimulus, a random pattern of firing was generated within the network by causing each neuron in the network to fire at random times in each second. The frequency of this random background firing was set to 1 Hz (i.e., each neuron was randomly fired once per second).

3.4.  Learning Rule

Spike-timing-dependent plasticity (STDP) is a learning rule that changes the strength of each synapse onto a postsynaptic neuron according to the arrival time of spikes at the synapse relative to the firing time of the neuron (Markram et al., 1997; Caporale & Dan, 2008). We apply this learning rule throughout the course of network training, although synaptic plasticity is suspended in later experimental phases involving the analysis of the stimulus response. With STDP enabled, network training produces a selective strengthening of the synaptic connections between PNG neurons, resulting in a rapidly increasing probability of firing events occurring within the temporal windows that map out the spatiotemporal pattern of PNG activation.

3.5.  Response Histograms

Response histograms were generated from the network firing data by aggregating the firing data over multiple 1 second intervals called response frames (see Figure 5). Each response frame is evenly divided into 1 millisecond slots, representing fixed temporal offsets relative to the start of the frame. Within each response frame, the stimulus was presented at the start of the frame, leaving the remainder of the frame for the activation response.

Figure 5:

A schematic of a single response frame showing the presentation of a stimulus at the time indicated by the arrow, and the subsequent activation response caused by one or more PNG activations. The firing events in this schematic have been filtered so as to show just the stimulus events and the events that are part of the resulting activation response. This single response frame is assumed to be one of many in a long, consecutive sequence of frames.

Figure 5:

A schematic of a single response frame showing the presentation of a stimulus at the time indicated by the arrow, and the subsequent activation response caused by one or more PNG activations. The firing events in this schematic have been filtered so as to show just the stimulus events and the events that are part of the resulting activation response. This single response frame is assumed to be one of many in a long, consecutive sequence of frames.

3.6.  The Response Fingerprint

A response histogram is generated for each neuron by accumulating the number of spikes that occur within each response frame slot. The collection of response histograms for each neuron in the network together provides a unique profile of the response of the network to a stimulus (a frame profile). Although many neurons produce no significant peaking in response to a stimulus, the peaks that occur in the remainder reflect a strong correlation between stimulus presentation and subsequent neural firing. We can define a temporal window for each of the neurons whose firing strongly correlates with the stimulus, with each window specifying the temporal range that captures the largest peak in the spike counts for that neuron. The combined temporal windows for these selected neurons define a response fingerprint, so called because each fingerprint provides a unique spatiotemporal signature of the PNG activation response produced by the stimulus.

3.7.  Window Activation

Generation of a response fingerprint requires aggregation of firing events across multiple response frames. However, the response fingerprint produced by a given network and stimulus need only be generated once and can be used thereafter for the analysis of the stimulus response. This analysis involves the examination of the PNG activations that occur in each frame and is produced by filtering the firing events in the frame through the temporal windows of the selected stimulus-specific fingerprint. The activation of multiple PNGs triggered by the stimulus is referred to as the PNG activation response, while the occurrence of one or more spikes within a temporal window is termed window activation. Each window activation provides accumulating evidence that a stimulus-specific PNG activation has occurred within the response frame.

3.8.  Probability Scores

We can compute a conditional probability for each temporal window to represent the probability of window activation given presentation of the stimulus. For convenience, we define resp as an activation response that may be related to the stimulus and act as a window activation. The probability that the activation response is stimulus related is therefore P(resp), and the probability of a window activation is P(act).

Applying Bayes' theorem, we get
formula
3.1
Given that each response frame typically includes multiple items of evidence (i.e., multiple window activations), we need to compute For this we can use an extended form of Bayes' theorem that supports multiple events. Or more simply, we can assume conditional independence of act1, act2, act3 such that
formula
3.2
This conditional independence assumption allows a more general equation as follows:
formula
3.3
where respi represents a stimulus-related response related to stimulus i.
A naive Bayes classifier probabilistically selects the best hypothesis from a range of competing hypotheses based on evidence provided by a set of class features. For our purposes, we wish to decide whether the response is related to a particular stimulus, given the set of coincident window activations in the frame. In equation 3.3, the denominator () is constant and independent of the hypothesis on the stimulus, allowing us to simplify:
formula
3.4
where the function argmax selects the maximum value of its arguments, that is, the best hypothesis.

We can apply equation 3.4 to multiple fingerprints in order to determine which of multiple potential stimuli the response is most closely related to. This is achieved by simply selecting the result with the largest probability (see Guise et al., 2013b, for further details).

3.9.  Learning Curves

Learning curves show changes in the PNG activation response over time relative to the activation response of a fully trained network. Learning curve experiments have two phases: first, a network is trained on a specified stimulus while saving the network state at regular intervals; second, probability scores and window activation counts are calculated for each saved state using the response fingerprint from the fully trained network (the last of the saved network states). The first phase produces a sequence of networks in progressively more advanced stages of training. These different network states are then used in the following phase to generate the learning curve. Learning is assessed for each network state by running the classifier through multiple trials and generating averages for the window activation counts and probability scores over each run. For the learning curve experiments, window activation counts were computed using the fingerprint generated from each fully trained network at 600 seconds and averaged over 100 frames.

Additional method details are available in Guise et al. (2013b).

4.  Results

In the remainder of this letter, we demonstrate the use of the response fingerprinting method in two experiments. Both experiments examine some of the properties of PNG activation that are relevant to the question of whether polychronicity might provide the foundation of a representational system. The first uses the response fingerprinting method to investigate the effects of learning on PNG activation, measuring changes in the number of window activations over the course of training. The second experiment examines the PNG activations produced by fully trained networks, paying special attention to the selectivity and consistency of these activations and how these attributes impact on the requirements of a PNG-based representational system.

4.1.  The Rate of Learning

Many of the structural PNGs that reside within the connection graph of each experimental network have the potential for activation. However, the activation of these structural groups typically occurs only after a network training regimen that involves repeated exposure to a triggering stimulus. Over the course of training, the STDP learning rule produces selective reinforcement of the connections between PNG neurons for those structural PNGs that are spatiotemporally congruent with the stimulus. As the training program proceeds, the probability of producing stimulus-specific PNG activations following each stimulus presentation progressively increases until it eventually reaches certainty. This growth in the PNG activation probability reflects a corresponding increase in the size of the stimulus-specific firing response and therefore also the number of window activations captured using a stimulus-specific fingerprint.

A fully trained network produces a consistent PNG activation response that provides an appropriate benchmark for studying the rate of stimulus learning. A response fingerprint can be derived from this consistent response and used to test the number of window activations produced over the course of training. If the network state is sampled at regular intervals during training, each state in the training sequence produces a progressively larger stimulus response until the number of window activations reaches a plateau. The size of the stimulus response can then be measured in each of the sampled network states by counting the average number of window activations produced by the repeated stimulus over multiple response frames. Plotting each of these window activation counts produces a sigmoid-shaped learning curve representing changes in the size of the stimulus response over the course of learning.

Figure 6 shows the learning curves produced from 20 independent networks as they learn the ascending stimulus. The window activation counts for each network are initially very low due to the low probability of firing events occurring within the fingerprint windows defined by a fully trained network. As training proceeds and the networks become increasingly familiar with the stimulus, the number of window activations produced by each network increases exponentially, reaching a plateau at around 130 seconds.

Figure 6:

Changes in the number of window activations following repeated stimulation with the ascending stimulus. Each line represents the changes in window activation counts for each of 20 independent mature networks trained on the ascending stimulus at 5 Hz.

Figure 6:

Changes in the number of window activations following repeated stimulation with the ascending stimulus. Each line represents the changes in window activation counts for each of 20 independent mature networks trained on the ascending stimulus at 5 Hz.

The time course of learning a single stimulus is very consistent across networks, with only small internetwork variation in the rate of learning and the size of the maximal stimulus response. In a later experiment, we will examine the specificity of this stimulus response, and for this we will need a set of networks that have been trained on multiple stimuli. It is therefore of interest to examine whether training with multiple stimuli has an effect on the network learning curves. The multistimulus version of the learning curve experiment is identical to the single-stimulus version except that the window activations produced by each sampled network state are measured for two different stimuli using two unique stimulus-specific fingerprints. As with the single stimulus version of the experiment, the set of measurements that constitute the learning curve are produced for each of the 20 test networks, using the fingerprint (or pair of fingerprints) from each fully-trained network state.

The results shown in Figure 7 highlight some important differences in the learning behavior of networks during multistimulus training relative to single stimulus training. The 20 networks in each panel of Figure 7 are the same as in Figure 6 except that each network in Figure 7 was trained on two different stimuli—the ascending and descending stimuli. Changes in the PNG activation response to each stimulus are shown in separate panels in Figure 7 as each is measured separately using the fingerprint corresponding to each stimulus. The results using the ascending fingerprint are shown in the left panel and those for the descending fingerprint in the right panel.

Figure 7:

Changes in the number of window activations following repeated stimulation with the ascending stimulus (left) or the descending stimulus (right). Each line represents the changes in window activation counts for each of 20 independent mature networks trained on both the ascending and descending stimuli at 5 Hz.

Figure 7:

Changes in the number of window activations following repeated stimulation with the ascending stimulus (left) or the descending stimulus (right). Each line represents the changes in window activation counts for each of 20 independent mature networks trained on both the ascending and descending stimuli at 5 Hz.

Many of the networks in both panels produce a near asymptotic learning curve similar to those seen in Figure 6, although no obvious plateau is reached by the networks over the 600 seconds of training. Some networks also achieve large activation responses, reaching a similar level of window activation counts to the networks produced by single-stimulus training. However, a large number of networks show a more flattened learning response. In both panels, there is a clear divide between networks that learn quickly and those that do poorly. In the left panel, there are 13 networks that perform well in response to the ascending stimulus, eventually converging on a window activation count close to 400. In the right panel, just 7 networks perform well in response to the descending stimulus, and the remaining networks perform relatively poorly. Interestingly, the networks that perform well on the ascending stimulus are poor performers on the descending stimulus (and vice versa), suggesting a limitation in network capacity using this training regimen.

The response fingerprinting method is also able to compute a probability score from the patterns of window activations produced by each stimulus presentation. This score represents the probability that the window activation response is specific to the stimulus and not just some random pattern of firing. In the early stages of training, this probability score is very low, reflecting a low probability that any firing events captured by fingerprint windows are representative of a coherent stimulus response. However, single-stimulus training produces a steep sigmoidal change in probability score in all networks, mirroring an underlying growth in the size of the PNG activation response (result not shown). The shape of this curve is sensitive to the width of the Bayesian classifier threshold and reflects the degree of variability in the average window count values as they approach this threshold. Importantly, the probability scores typically reach their maximum within 300 to 500 stimulus presentations (60–100 seconds), well before the window activation counts reach a plateau. This property of the response fingerprinting method allows the method to differentiate between the stimulus response of different stimuli even in partially trained networks.

Figures 8 and 9 show changes in the probability scores over the course of learning multiple stimuli. Figure 8 shows the 13 networks favoring the ascending stimulus, while Figure 9 shows the 7 networks favoring the descending stimulus. As with the window activation counts, each network is measured twice—once with the ascending fingerprint (left) and once with the descending fingerprint (right). Each set of networks produces a tight sigmoidal curve in the probability scores, but only when the networks are measured using the fingerprint of their favored stimulus. When measured using the nonfavored stimulus fingerprint, the probability scores are much more scattered and erratic, reflecting a lack of consistency in the stimulus responses.

Figure 8:

Changes in the probability scores during multistimulus training for the 13 networks favoring the ascending stimulus. (Left) Changes in the probability scores produced by stimulation with the ascending stimulus and measured using the ascending stimulus fingerprint. (Right) Corresponding values for the descending stimulus.

Figure 8:

Changes in the probability scores during multistimulus training for the 13 networks favoring the ascending stimulus. (Left) Changes in the probability scores produced by stimulation with the ascending stimulus and measured using the ascending stimulus fingerprint. (Right) Corresponding values for the descending stimulus.

Figure 9:

Changes in the probability scores during multistimulus training for the 7 networks favoring the descending stimulus. (Left) Changes in the probability scores produced by stimulation with the the ascending stimulus and measured using the ascending stimulus fingerprint. (Right) Corresponding values for the descending stimulus.

Figure 9:

Changes in the probability scores during multistimulus training for the 7 networks favoring the descending stimulus. (Left) Changes in the probability scores produced by stimulation with the the ascending stimulus and measured using the ascending stimulus fingerprint. (Right) Corresponding values for the descending stimulus.

4.2.  Selectivity and Consistency of the Activation Response

The availability of networks that have been trained on two different stimuli provides an opportunity to investigate the selectivity and consistency of the activation response. The question of whether the stimulus response is specific to each stimulus and occurs consistently on each stimulus presentation is relevant to the idea that polychronization might provide an underlying mechanism for a PNG-based representational system. In the following experiment, the PNG activation response produced by each of the two stimuli is tested over multiple response frames, and the response data are then separately evaluated from the perspective of either selectivity or consistency.

A single stimulus is presented in each frame over multiple trials, and the network response to each trial is assessed using the fingerprint for each of the two stimuli. The selectivity requirement is tested as follows. if the response is specific to the stimulus, then the ascending fingerprint should identify a large set of PNG activations in frames where the ascending stimulus is presented and few PNG activations in frames where the descending stimulus is presented (and vice versa for the descending fingerprint). The probability scores computed for each fingerprint will therefore favor the corresponding stimulus in frames where that stimulus is presented, reflecting the selective activation of PNGs related to the stimulus, that is, comparing the response measured by each fingerprint should favor the stimulus presented in each frame due to the selective activation of PNGs triggered by that stimulus.

The same experiment also addresses the consistency requirement by comparing the pattern of scoring over multiple frames with the chosen stimulus in each frame to determine the consistency of the stimulus-specific response. An inconsistent response will produce a low number of window activations and therefore a low probability score using the fingerprint that is specific to the stimulus in each frame. However, if each stimulus consistently produces PNG activations, then the stimulus-specific fingerprint will compute a high number of window activations and a high probability score.

In each trial, one of the two stimuli is presented, and the resulting stimulus response is collected. Filtering the stimulus response through the fingerprint for each stimulus generates a pair of window activation counts, and from these, a pair of probability scores is computed. The stimulus fingerprint that generates the largest probability score is selected as the trial winner. If the stimulus response is highly selective, then the probability scores will be substantially different, generating an unambiguous winner for each trial. In addition, if the stimulus response is consistent, then the trial winner in each frame will correspond to the selected stimulus.

Figure 10 shows the results of this experiment over 75 trials for each of 20 networks trained on both ascending and descending stimuli. In each of the two panels (top and bottom), each row consists of 75 positions—one for each response frame trial. A pair of results is produced for each trial (the trial winner and loser), and these are represented in the corresponding rows in the top and bottom panels for each network. The top panel shows the results using the ascending fingerprint, and the bottom panel shows the results using the descending fingerprint.

Figure 10:

The PNG activation response of 20 networks measured using two different response fingerprints. Each of the 20 independent networks was trained on both the ascending and descending stimuli and tested on both the ascending fingerprint (top) and the descending fingerprint (bottom). The horizontal axis for each figure shows trial winners (a dot) and losers (an empty space) for the first 75 response frames. Three different stimuli were used to test each network: the ascending stimulus was presented for the first 25 of the 75 frames, the descending stimulus for the second 25, and no stimulus was provided in the third set of 25 frames.

Figure 10:

The PNG activation response of 20 networks measured using two different response fingerprints. Each of the 20 independent networks was trained on both the ascending and descending stimuli and tested on both the ascending fingerprint (top) and the descending fingerprint (bottom). The horizontal axis for each figure shows trial winners (a dot) and losers (an empty space) for the first 75 response frames. Three different stimuli were used to test each network: the ascending stimulus was presented for the first 25 of the 75 frames, the descending stimulus for the second 25, and no stimulus was provided in the third set of 25 frames.

Stimuli were presented in blocks of 25 over the 75 response frames shown in the figure. The ascending stimulus was chosen for the first stimulus block, the descending stimulus for the second block, and no stimuli were presented in the remaining 25 frames (see the stimulus ranges at the top of the figure). In each trial position, the presence of a small filled circle represents the trial winner, while the absence of a filled circle in the corresponding row represents the trial loser. In order to meet the consistency requirement, the stimulus selection and the trial winner must match. The block of filled circles in the top panel of Figure 10 corresponds to trials in which the ascending stimulus was selected and the trial winner was the ascending fingerprint, and therefore the response to the ascending stimulus is shown to be consistent. The block of filled circles in the bottom panel of Figure 10 corresponds to trials in which the descending stimulus was selected and the trial winner was the descending fingerprint, showing that the descending stimulus also produces a consistent response in all 20 networks.

In stimulus blocks where no stimulus was provided, very few window activations were counted, producing no clear winner. However, in trials where a stimulus was presented, an unambiguous winner was selected in all trials, providing support for the selectivity of the stimulus response. In each trial in which a stimulus was presented, the probability scores computed for each stimulus fingerprint differed by orders of magnitude, depending on which of the two stimuli was presented (results not shown).

5.  Discussion

Izhikevich (2006a) has previously shown that structural PNGs can be found in large numbers within the connection structure of spiking neural networks. Repeated exposure to a stimulus causes the PNGs that are structurally compatible to the stimulus to adapt, supporting PNG activation and polychronization. The set of adapted PNGs triggered by a stimulus constitutes the PNG activation response, although a more sophisticated view of the activation response is developing (see below).

In previous research, the PNG activation response has been conveniently viewed as a fixed and deterministic sequence of firing events, with the timing of each event corresponding to the fixed axonal delays between the neurons constituting the underlying structural PNG. However, this view is at odds with the observed variability in neural participation and firing times between different activations of the same PNG. Another source of variability is the change in the PNG activation response that occurs over the course of training as neurons are recruited into the polychronous group.

In response to this observed variability, the response fingerprinting method takes a probabilistic view of PNG activation, viewing the PNG activation response as a set of firing probabilities conditional on the presentation of a triggering stimulus. Each neuron in the network has an associated firing probability given the presentation of a stimulus: for neurons involved in PNG activation, the firing probability is high, while for other neurons it is very low or zero. The firing probability of PNG neurons reaches a peak within windows that have a fixed temporal position relative to stimulus onset, and these windows therefore capture the majority of firing events related to PNG activation.

The spatiotemporal arrangement of windows describes the averaged spatiotemporal path of polychronization that occurs in the PNG activation response. This unique fingerprint can therefore be used to selectively filter the poststimulus firing events for events related to stimulus-specific PNG activation, creating a new method for the study of the PNG activation response. The fingerprint also provides a means for the visualization of PNG activation by using the underlying network structure to identify the causal connections within the filtered firing events (see Guise et al., 2013b).

An initial demonstration of the response fingerprinting method was provided in a learning experiment. The growth in the number of window activations following repeated exposure to the ascending stimulus produced a sigmoid-shaped learning curve that reached a plateau after around 650 repetitions. At this point, the network can be deemed to have learned the stimulus, although the probability scores show that a unique and consistent PNG activation response can be detected at an earlier stage of training, within 300 to 500 repetitions, suggesting that learning may be largely complete at this earlier stage. Nevertheless, this requirement for repeated exposure in stimulus learning may at first seem incompatible with a perceptual world in which stimuli are not so conveniently repeated. However, the replay of temporally compressed spatial memories during the sharp-wave-associated ripple state might provide a bridge between a mechanism for neocortical memory consolidation that requires repetition and a short-term memory system in the hippocampus (Cutsuridis & Hasselmo, 2011; Girardeau & Zugaro, 2011).

When training with multiple stimuli, the networks that produced a large activation response for one stimulus generally produced a small response with the other stimulus. This result suggests that the networks are optimizing themselves for one stimulus at the expense of the other, an interpretation that has significant implications for the representational capacity of these networks. However, the optimum training regimen for multiple stimuli is not yet known, and therefore a poor training strategy may explain this apparent compromise in the network capacity for each stimulus.

The second experiment showed that a PNG activation response consistently followed stimulus presentation in trained networks. The probabilistic view imposed by the fingerprinting method allows this response to be readily detectable despite the variability of the activation response. The same experiment also provides initial support for the selectivity of PNG activation as previously demonstrated by Izhikevich (2006a) and Paugam-Moisy et al. (2008). Previous experiments in our lab have highlighted the complexity of the PNG activation response, emphasizing the need for a sophisticated view of what selectivity in the activation response might entail (Guise et al., 2013b). In particular, we support a set-based view of the PNG activation response in which each representation requires activation of a unique set of PNGs (Izhikevich, 2006a; Paugam-Moisy et al., 2008). The results shown in Figure 2 further emphasize that individual PNG activations cannot be representational.

In addition, visualization experiments suggest that a representational activation response may be more than just the sum of the individual PNG activations that are directly triggered by the stimulus. Izhikevich (2006a) has proposed that networks dynamically respond to different stimuli and recurrent input with both competition and cooperation between PNGs. In such an environment, individual PNG activations can interact and merge to trigger further PNG activations that would never otherwise be triggered in isolation. This emerging view of the PNG activation response arises naturally from a probabilistic model based on individual firing events that strongly correlate with stimulus events. Presentation of the stimulus causes the firing events produced from the interaction of PNG activations to co-occur with high probability, and therefore also to correlate with stimulus presentation.

In future work we hope to use the response fingerprinting method to examine the issue of capacity raised in this study and further explore the selectivity of the activation response. We will also use this new method to examine the effect of changes in network parameters on the PNG activation response.

References

Caporale
N.
, &
Dan
Y.
(
2008
).
Spike timing-dependent plasticity: A hebbian learning rule
.
Annu. Rev. Neurosci.
,
31
,
25
46
.
Cutsuridis
V.
, &
Hasselmo
M.
(
2011
).
Spatial memory sequence encoding and replay during modeled theta and ripple oscillations
.
Cognitive Computation
,
3
(
4
),
554
574
.
Girardeau
G.
, &
Zugaro
M.
(
2011
).
Hippocampal ripples and memory consolidation
.
Current Opinion in Neurobiology
,
21
(
3
),
452
459
.
Guise
M.
,
Knott
A.
, &
Benuskova
L.
(
2013a
).
Evidence for response consistency supports polychronous neural groups as an underlying mechanism for representation and memory
. In
S.
Cranefield
&
A.
Nayak
(Eds.),
AI 2013: Advances in Artificial Intelligence, vol. 8272 of Lecture Notes in Computer Science
(pp.
86
97
).
New York
:
Springer
.
Guise
M.
,
Knott
A.
, &
Benuskova
L.
(
2013b
).
Response Fingerprinting: A probabilistic method for evaluating the network response to stimuli
(
Technical Report OUCS-2013-10
).
Dunedin
:
Department of Computer Science, University of Otago
. http://www.cs.otago.ac.nz/research/techreports.php
Guise
M.
,
Knott
A.
, &
Benuskova
L.
(
2013c
).
Spinula: Software for simulation and analysis of spiking network models
(
Technical Report, OUCS-2013-03
).
Dunedin
:
Department of Computer Science, University of Otago
. http://www.cs.otago.ac.nz/research/techreports.php
Izhikevich
E. M.
(
2006a
).
Polychronization: Computation with spikes
.
Neural Computation
,
18
(
2
),
245
282
.
Izhikevich
E. M.
(
2006b
).
Reference software implementation for the Izhikevich model: Minimal spiking network that can polychronize
. http://www.izhikevich.org/publications/spnet.htm
Izhikevich
E. M.
,
Gally
J. A.
, &
Edelman
G. M.
(
2004
).
Spike-timing dynamics of neuronal groups
.
Cerebral Cortex
,
14
(
8
),
933
944
.
Markram
H.
,
Lübke
J.
,
Frotscher
M.
, &
Sakmann
B.
(
1997
).
Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS
.
Science
,
275
(
5297
),
213
215
.
Martinez
R.
, &
Paugam-Moisy
H.
(
2009
).
Algorithms for structural and dynamical polychronous groups detection
. In
C.
Alippi
,
M.
Polycarpou
,
C.
Panayiotou
, &
G.
Ellinas
(Eds.),
Artificial Neural Networks ICANN 2009
,
vol. 5769 of Lecture Notes in Computer Science
(pp.
75
84
).
New York
:
Springer
.
Paugam-Moisy
H.
,
Martinez
R.
, &
Bengio
S.
(
2008
).
Delay learning and polychronization for reservoir computing
.
Neurocomputing
,
71
(
7
),
1143
1158
.