Correlations between responses in visual cortex and perceptual performance help draw a functional link between neural activity and visually guided behavior. These correlations are commonly derived with ROC-based neural-behavioral covariances (referred to as choice or detect probability) using boxcar analysis windows. Although boxcar windows capture the covariation between neural activity and behavior during steady-state stimulus presentations, they are not optimized to capture these correlations during short time-varying visual inputs. In this study, we implemented a matched-filter technique, combined with cross-validation, to improve the estimation of ROC-based neural-behavioral covariance under short and dynamic stimulus conditions. We show that this approach maximizes the area under the ROC curve and converges to the true neural-behavioral covariance using a Poisson spiking model. We also demonstrate that the matched filter, combined with cross-validation, reveals the dynamics of the neural-behavioral covariations of individual MT neurons during the detection of a brief motion stimulus.
Small trial-by-trial fluctuations in sensory neurophysiological activity are often weakly correlated with measurable changes in perceptual behavior (referred to here as neural-behavioral covariations). Accurately estimating neural-behavioral covariations has become an important tool for understanding how neural activity is linked to behavior and perception. For example, neural-behavioral covariations during visually guided behavior have provided models for how activity in visual cortex is pooled (Zohary, Shadlen, & Newsome, 1994; Britten, Newsome, Shadlen, Celebrini, & Movshon, 1996; Shadlen, Britten, Newsome, & Movshon, 1996; Uka & DeAngelis, 2004; Purushothaman & Bradley, 2005; Gu, DeAngelis, & Angelaki, 2007), modulated by attention or perceptual learning (Law & Gold, 2008; Herrington & Assad, 2009), processed differently depending on sensory or task demands (Krug, Cumming, & Parker, 2004; Cohen & Newsome, 2008; Bosking & Maunsell, 2011), and linked to feedforward versus feedback cortical processing (Nienborg & Cumming, 2009; Smith, Zhan, & Cook, 2011).
Most trial-by-trial neural-behavioral covariations of neurons in visual cortex are estimated by comparing the distribution of neural responses associated with two behavioral outcomes (e.g., correct versus failed trials). This analysis usually employs receiver operating characteristics (ROC) methods and is commonly referred to as choice or detect probability because it reports the probability that an ideal observer could correctly predict the behavioral outcome given the neural response (Celebrini & Newsome, 1994; Britten et al., 1996). Because the functional link between the activity of a single neuron and behavior is weak, observing measurable neural-behavioral covariations usually requires many trials and can depend on the temporal window used in the analysis (Ghose & Harrison, 2009; Price & Born, 2010).
Most previous studies have applied boxcar windows to compute neural-behavioral covariations, usually based on the duration of the visual stimulus. Such a fixed analysis window, however, may not capture the true differences between correct and failed neural responses. For example, both Ghose and Harrison (2009) and Price and Born (2010) provide particularly compelling examples of how a window's position and length can affect neural-behavioral covariations. In this study, we examined if the shape of the analysis window can be optimized using the well-known signal-processing method referred to as matched filtering.
Analysis windows that are shaped to match the signal of interest have been widely used in signal processing and digital communication applications as correlators that minimize noise (Wozencraft & Jacobs, 1965; Davenport & Root, 1987; Papoulis & Pillai, 2002; Vaseghi, 2008). Given their adaptive characteristics, matched filters could be used to predict behavior based on the fluctuations of the neural responses. Importantly, the shape of the matched filter would describe the temporal dynamics of the neural-behavioral covariations.
In this letter, we first provide an overview of using ROC analysis to estimate neural-behavioral covariations using data previously recorded from the middle temporal (MT) area of visual cortex. We then derive the matched-filter theory based on firing rates and show that this theory optimizes the ROC analysis commonly used to compute neural-behavior covariations. Using a Poisson neuron model, we demonstrate that the matched-filter approach, combined with cross-validation, provides an unbiased estimate of the true neural-behavioral covariation. Finally, we apply the matched-filter analysis to individual MT neural responses to reveal the time course over which neural-behavioral covariations arise while a subject detects a brief motion stimulus.
2. Background: ROC-Based Neural-Behavioral Covariations
To highlight the importance of the analysis window for estimating neural-behavioral covariances, we analyzed previously published neurophysiological recordings from visual cortical area MT (Smith et al., 2011). Neural-behavioral covariation measures how well one can predict the behavioral outcome on a trial-by-trial basis using the neural responses. To illustrate the effect of the analysis window on estimating neural-behavioral covariations, we used single-neuron recordings from monkeys performing a motion detection task while viewing two coherent random dot patches shown on a CRT monitor (see Figure 1A and section 3.4). The animals’ task was to release a lever after detecting the occurrence of a brief 50 ms coherent motion pulse that occurred at a random time in the random dot patches. Figure 1B shows an example spike raster from one MT neuron, aligned to the onset of the motion pulse (vertical line) and separated into correct (black) and failed (gray) trials. Only spikes that occurred within 100 ms before and 300 ms after the stimulus onset are shown.
To illustrate how neural-behavioral covariations are affected by the placement of the analysis window, we applied either a 100 ms (solid bar) or 400 ms (dashed bar) boxcar window to the example neural responses shown in Figure 1B. The firing rate distributions computed for the 100 ms window are shown in Figure 1C and are grouped into correct (black) and failed (gray) distributions. As can be seen in both the spike raster and firing rate histograms, this neuron's activity was correlated with the behavioral outcome on a trial-by-trial basis. Correct trials were associated with higher firing rates, while failed trials tended to be associated with lower firing rates. Thus, observing the response of this MT neuron on a single trial would allow one to predict the behavioral outcome of that trial at better than chance levels.
The correlation between neural activity and behavior can be quantified using an ROC-based analysis. The ROC curve in Figure 1D describes the relationship between hit and false alarm probabilities (PH(r) vs. PFA(r) where r is the firing rate) for the two firing rate distributions in Figure 1C. PH(r) is the probability of categorizing the trial as correct when the monkey detected the occurrence of the motion pulse, while PFA(r) is the probability of categorizing the trial as correct when the monkey failed to detect the motion pulse. Note that PFA(r) should not be confused with behavioral false alarms that occurred when the monkeys responded before the motion pulse (these trials were not included in the analysis). Thus, the ROC curve in Figure 1D plots both probabilities as a function of a firing rate threshold.
The area under an ROC curve (aROC) is used to compute a single value that quantifies the neural-behavioral covariance. This area corresponds to the probability that an ideal observer could correctly classify a random draw from each distribution associated with correct and failed trials (Celebrini & Newsome, 1994; Britten et al., 1996). An aROC at the chance level of 0.5 occurs when the two distributions perfectly overlap and indicates no neural-behavioral covariance. An aROC value that approaches either 0 or 1 indicates strong neural-behavioral covariance.
Figure 1D shows two ROC curves corresponding to the two boxcar windows shown in Figure 1D. The shorter 100 ms window (solid line) produced a larger aROC neural-behavioral value of 0.87, while the longer 400 ms window produced a weaker aROC score of 0.74. The reason for this aROC difference is fairly clear from the spike raster as the largest difference between correct and failed responses occurs immediately after the 50 ms motion pulse. Thus, the 100 ms window is better positioned to capture the differences in neural activity associated with correct and failed trials. This example illustrates our central question: What is the analysis window (in terms of both shape and location) that best captures the neural-behavioral covariation?
Only two neurophysiological studies have examined the effect of length and position of boxcar analysis windows on neural-behavioral covariations (Ghose & Harrison, 2009; Price & Born, 2010). Although both studies suggested that neural-behavioral covariations arise over relatively short boxcar windows, neither study examined other analysis window shapes. Other theoretical studies have addressed optimal ROC scores using enhanced machine learning models such as decision tree learning (Ferri, Flach, & Hernandez-Orallo, 2002), rule sets learning (Fawcett, 2001), gradient descent learning (Herschtal & Raskutti, 2004; Lee, Gader & Wilson, 2007; Castro & Braga, 2008), and boosting learning (Freund & Schapire, 1997; Freund, Iyer, Schapire, & Singer, 2003; Rudin & Schapire, 2009). Others (Brefeld & Scheffer, 2005) have implemented a support vector machine algorithm for aROC maximization, while Takenouchi, Komori, and Eguchi (2012) adopted a novel boosting-type algorithm applied to concave functions when solving the optimization problem. In this study, we tackle the problem of optimizing the analysis window for ROC-based neural-behavioral estimates by using a simple matched-filter technique that can be applied to single neurons.
3.1. Optimal Matched-Filtering Maximizes the aROC.
Matched filters have long been used in applications of radar and sonar detection (Nathanson, Reilly, & Cohen, 1999; Skolnik, 2001; Levanon & Mozeson, 2004). Additionally, they have been largely implemented in signal transmission demodulators as front-end correlators to optimum receivers, thus forming the basis for noise reduction and cancellation in signal processing applications (Lee & Messerschmitt, 1994; Sklar, 2001). Based on the maximum a posteriori probability rule, these detectors generally consist of a bank of matched filters operating on a received time-varying signal, which is then passed through a maximum likelihood decoder to predict an optimal estimate (Turin, 1960; Proakis & Salehi, 2002). A similar problem arises in computing neural-behavioral covariations, where the objective is to quantify the distance (or separation) between two distributions of neural responses, one from correct trials and the other from failed trials (see Figure 1C). Figure 2 illustrates a theoretical framework for how a matched filter can be derived to maximize this distance.
What is the optimal shape of the matched filter, such that convolving it with a neuron's spike response will result in the maximum separation between the distributions of correct and failed behavioral outcomes? This has been clearly explained in problems of optimal detection in the presence of additive white noise channels (Cooper & McGillem, 1999; Oppenheim & Verghese, 2010) and other detection structures of hypothesis testing or pattern classification problems (Pickholtz, Shilling, & Milstein, 1982; Gilhousen et al., 1991; Pickholtz, Milstein, & Shilling, 1991). We optimized neural-behavioral covariations by adopting an approach similar to the matched-filter optimum receiver used in data transmission applications.
A lowercase f denotes the probability density function of either a correct or failed distribution with respect to the random variable R(i) and an uppercase P denotes the integration of f over a specific range of firing rates. For a given threshold gamma, hit (see equation 3.9) and false alarm (see equation 3.10) probabilities are measured by integrating fR(r(i)|C) or and fR(r(i)|F) or over a range of firing rates greater than respectively. It is also important to note that by considering the matched filter as a linear time-invariant system, the likelihood ratio as a function of rho, equation 3.7, becomes equivalent to the likelihood ratio as a function of r in equation 3.8, in an optimal sense.
The assumptions of white gaussian-distributed noise channels are primarily used to simplify the theoretical derivation of the matched filter and link the modulation process of the matched filter to the optimal detection algorithm of the aROC decoder. These parameters, however, are not stringent conditions for the implementation of the matched-filtering technique. Matched filters have also been derived with generalized noise distributions and colored spectrums (Papoulis & Pillai, 2002) for digital communication, optimal radar, and sonar detection applications (Sklar, 2001; Skolnik, 2001).
In summary, we have shown that optimizing the neural-behavioral covariance (see Figure 2) is accomplished by using a matched-filter analysis window designed by subtracting the average firing rate on failed trials from the average rate on correct trials. The matched filter maximizes the separation of the mean firing rates associated with correct and failed distributions, and thus maximizes the likelihood ratio corresponding to the curvature of the ROC curve. Using a Poisson spiking computer model below (see section 3.3), we empirically demonstrate that this method converges to the true neural-behavioral covariation (see section 4.1).
3.2. Cross-Validation of Neural-Behavioral Estimates with Matched Filters.
When computing the “correct minus failed” matched filter, the high variability in neural recordings introduces a positive bias in our estimate of the neural-behavioral covariation due to random differences in the spike responses between correct and failed trials. This bias can be eliminated by a common cross-validation approach, where the neural responses are randomly divided into two equally sized groups (Fawcett, 2006). Matched filters were estimated for each set of neural responses and then applied to the other group (i.e., crossed). Thus, spike responses were weighted by the crossed matched filters to produce two aROC scores of the neural-behavioral covariation.
In our analysis of the computer model and real MT data, we repeated this random cross-validation 100 times for each set of neural responses to produce 200 aROC scores and 200 matched filters. From these we used the mean aROC score as the neural-behavioral estimate and the mean matched filter as the analysis window associated with the neural-behavioral estimate. For each set of aROC scores, a t-test was used to measure if the mean was significantly different from 0.5 (p < 0.05).
To estimate the matched filters for both the model and MT neurons, we fit a single gaussian to the average neural response (PSTH) for the correct and failed trials separately. A gaussian was used because the correct and failed average neural responses tended to be gaussian-like for the MT data in response to the 50 ms motion pulse. The gaussian function had four free parameters: baseline firing rate (spikes/sec), peak firing rate (spikes/sec), center of the gaussian (ms), and standard deviation of the gaussian (ms). The matched filter was then computed as the difference of the gaussian fits to the correct and failed PSTHs.
When estimating aROC scores for the fixed boxcar in Figure 4A, we randomly divided the data in half in order to use the same number of trials as in the cross-validation of the matched filters. Thus, a boxcar aROC estimate is the average of 200 individual estimates, each computed with half the data.
3.3. Poisson Spiking Model.
To test the matched-filter with cross-validation approach, we applied it to a Poisson spiking model. The model produced two sets of spike trains corresponding to 300 ms correct and failed trials (see Figure 3A for an example set of spike trains). Spike trains were modeled with a gaussian rate function with a baseline of 15 spikes/sec. For failed trials, the gaussian rate function had a peak of 50 spikes/sec, center at 150 ms and a standard deviation of 25 ms. We varied two of the gaussian rate function parameters, peak and center, to produce correct trial responses with different neural-behavioral covariances when compared to the failed trial responses. This allowed us to mimic changes in both firing rate and latency commonly observed in neural responses.
For the model results in Figure 3, the peak and center of the gaussian rate function for correct trials were varied to produce seven true aROC scores: 50 spikes/sec and 150 ms (same as failed); 50 spikes/sec and 147 ms (example shown in Figure 3A); 52 spikes/sec and 144 ms; 53.5 spikes/sec and 140.7 ms; 53.6 spikes/sec and 134.5 ms; 60 spikes/sec and 130 ms; and 70 spikes/sec and 125 ms. Each set of correct firing parameters corresponded to a true neural-behavioral aROC score of 0.5, 0.53, 0.56, 0.60, 0.65, 0.72, and 0.80, respectively. The true aROC score for the model was computed by applying the true matched filter to 250,000 trials. The true matched filter was the difference between the correct and failed rate functions that were used to produce the spike responses. The mean and standard deviation shown for the model results in Figure 3 were computed from 100 separate simulations. For the boxcar filter comparison shown in Figure 3, we used the same cross-validation technique described above, but optimized the boxcar by varying its center and width by “brute force” over all possible values so as to maximize the aROC score.
The reason we varied both firing rate and latency in the model was twofold. First, we wanted to examine the neural-behavioral aROC error that was introduced when estimating a matched filter with cross-validation. Second, we wanted to illustrate the effect of using a suboptimal analysis window to compute neural-behavioral covariances. For example, a boxcar filter would not be optimal for small latency differences between correct and failed neural responses with no overall change in firing rate, as illustrated by the analysis window in Figure 3B.
3.4. MT Neural Responses during a Motion-Detection Task.
To illustrate the matched-filter approach for estimating neural-behavioral covariations using real data, we analyzed single-neuron responses recorded from two monkeys performing a motion detection task (Smith et al., 2011). While the animals fixated on a central cross, the motion pulse occurred at a random time (0.5–10 sec, flat hazard function) and the animals had to release the lever between 200 and 800 ms after the motion pulse to receive a reward (see Figure 1A). Thus, the animals could not easily guess when the motion pulse occurred. In addition, the strength of the motion coherence was reduced to produce similar proportions of correct- and failed-detection trials. Trials were not included in this analysis if the monkey released the lever before the motion pulse or looked away from the central fixation spot. All procedures were approved by the McGill University Animal Care Committee under guidelines set forth by the Canadian Council on Animal Care.
While the animals performed the motion detection task, we simultaneously recorded the neural activity of two MT neurons (124 neurons total) using electrodes that were spaced approximately 1 to 2 mm apart. Neurons in area MT are direction sensitive and encode the coherent motion pulse (Smith et al., 2011). The direction and speed of the 50 ms motion pulses were matched to the receptive field (RF) properties of the neurons under study (dashed circles, Figure 1A). Thus, the coherent motion pulse that the monkeys detected was likely to have been encoded in the activity of the neurons under observation. The reason we recorded from two neurons was to study how MT neural-behavioral covariations depended on which neuron encoded the motion signal (see Smith et al., 2011). For this study, we examined neural responses only when the motion pulse occurred in a neuron's RF.
The theoretical derivation above (see section 3.1) shows that using a matched filter based on the difference between correct and failed trials maximizes the aROC score used to quantify neural-behavioral covariances. We assumed gaussian noise in this analytical derivation mainly for convenience. Since the matched-filter theory has been shown to apply to other noise distributions (Sklar, 2001; Papoulis & Pillai, 2002), the nonparametric advantage of aROC is preserved when using matched filters with noisy spike trains. The difficulty in applying this method to real data, however, is that neural responses are noisy and usually consist of a finite number of trials. These factors contribute a positive bias in the neural-behavioral estimate when using a matched filter. For example, random fluctuations in neural responses between correct and failed trials would tend to produce both a nonzero matched filter and neural-behavioral covariations, even when none existed.
To eliminate this bias, we used a common cross-validation approach (see section 3.2). The matched filter was estimated from half of the neural responses and used to compute the aROC score from the second half of the responses. Although cross-validation is theoretically sound for eliminating the bias, it raises a practical consideration in that splitting the data increases the amount of noise used to compute the aROC estimate. Thus, gains in aROC estimation using a matched filter could potentially be offset by reductions due to the introduction of noise during cross-validation. Even if this was the case, however, the shape of the matched filter would still provide a nonbiased estimate of the temporal dynamics over which the neural-behavioral covariations occurred.
To get an empirical sense of how these various factors affect matched-filter and neural-behavioral estimates, we first applied our approach to synthetic neural responses where the true aROC score was known. To explore what can be learned from the shape of the matched filter, we then applied our method to estimate neural-behavioral covariations for single MT neural responses recorded during a motion detection task.
4.1. Computer Simulations Using a Spiking Model Neuron.
We wanted to test that the matched filter combined with cross-validation worked as predicted by the theory and also to understand the effects of noise on our ROC-based neural-behavioral covariations. To accomplish this, we simulated neural responses with a Poisson spiking model. A series of 300 ms trials of spike activity was generated, and each trial was associated with two hypothetical behavioral outcomes (referred to as correct and failed). Spike activity was generated using a gaussian rate function to simulate a neuron's temporal response to a brief sensory event (see section 3.3). Figure 3A illustrates one example set of responses. The failed trials for all simulations were generated using the same rate function. Different sets of correct trials were generated by varying the rate function to produce different ROC-based neural-behavioral covariations. For example, the spike raster shown in Figure 3A corresponded to a true aROC score of 0.53 when computed using the true matched filter shown in Figure 3B (thick line).
We varied the magnitude and latency of the correct trial gaussian rate function to produce seven true aROC scores from 0.5 to 0.8 (see Figure 3C and section 3.3). The reason for varying both parameters was to test that our matched-filter with cross-validation procedure converged to the true aROC score when faced with changes in both latency and firing rate. For comparison, we also optimized a boxcar filter to illustrate the effects of using a nonmatched filter. To study how random noise affected aROC estimates, we ran three sets of simulations corresponding to trial sizes of 50, 100, and 1000. For each set of correct and failed trials, we repeated the simulations 100 times to measure the variance of our cross-validated aROC estimator.
Figure 3C shows an example of how well the matched-filter with cross-validation approach estimated the true aROC score using 100 trials (filled circles; error bars are SD). When there was no difference between the correct and failed rate functions (true aROC = 0.5), our aROC estimate showed no bias (i.e., suggesting a neural-behavioral covariation when none exists). At its core, our neural-behavioral analysis is a measure of correlation, and noise always reduces estimates of correlations (which one might consider as a less egregious form of bias). Thus, our estimated aROC scores were always pulled toward 0.5 and away from the true aROC as shown in Figure 3C.
The effect of noise due to a limited number of trials is reduced for higher true aROC conditions. This is mainly because the difference between the correct and failed responses at these higher aROC values is larger than the effects of noise introduced by the combined Poisson spike generation and cross-validation. As expected from the theory, an aROC estimator based on an optimized boxcar filter (gray circles) does not do as well because it cannot take advantage of the differences in latency between correct- and failed-trial activity. Plotting the error of our estimated aROC scores in Figure 3D, shows that the matched-filter with cross-validation technique converges toward the true aROC score as noise is reduced with an increased number of trials, while the nonmatched boxcar filter does not converge.
Each cross-validated estimate of the aROC score also provides an estimate of the matched filter (computed as the difference in the gaussian fits to the correct and failed responses; see section 3.2). For example, Figure 3B shows the mean and SD of the estimated matched filter (thin line and gray area, respectively) corresponding to the true aROC condition of 0.53 and 100 trials. Although the noise in the gaussian fits to the correct and failed responses added significant variability to individual matched-filter estimates, in the limit, our filter estimates converged to the true matched filter. This would be expected because unlike the aROC score, noise would not bias the matched-filter estimate in either the positive or negative direction.
The modeling results confirm the matched-filter theory by demonstrating that our aROC estimator was unbiased (i.e., it did not suggest a neural-behavioral covariance when none existed) and converged toward the true neural-behavioral covariance for Poisson spiking noise conditions. Our results also demonstrate that the increased noise due to cross-validation is more of an issue when the true aROC is small. Finally, the simulations highlight that for the matched filter to be an effective estimator, it needs to accurately capture the temporal dynamics of the firing rates for the two behavioral conditions. In our simulations, the shape of the estimated matched filter tended to capture the temporal differences between the correct and failed neural responses. As our analysis will demonstrate below, the latency and duration of the matched filter were key components for revealing the temporal dynamics of the neural-behavioral covariance for individual MT neurons.
4.2. Estimating the Neural-Behavioral Matched Filter for Single MT Neurons.
To provide an empirical example, we applied the matched-filter with cross-validation approach to estimate the neural-behavioral covariation of 124 MT neurons recorded during a motion detection task. We analyzed the MT neural responses in the same way as the model responses above. To examine the effect of noise introduced by the cross-validation, Figure 4A compares the matched-filter aROC scores with those computed using a 100 ms fixed boxcar filter that begins 50 ms after the start of the motion pulse (black bar in Figure 4B bottom). Note that we also used cross-validation when computing aROC scores using the fixed boxcar filter in order to equalize the number of trials used to that of the matched-filter analysis. The mean aROC score was similar for the fixed boxcar filter (0.569) versus the matched filter (0.566; p = 0.32, paired t-test). However, the matched-filter aROC scores tended to be larger than the fixed boxcar filter for the larger aROC scores (more than 0.6) and smaller for aROC scores (less than 0.6). This relationship was suggested by the modeling (see Figure 3D), which showed that matched-filter aROC scores are more susceptible to cross-validation noise when the true neural-behavioral covariations are small.
A benefit of our method is that it provides an estimate of the individual matched filters that were computed when deriving the aROC scores. When examining the individual matched filters, we included only the 111 MT neurons that had matched-filter aROC scores significantly different from 0.5 (black bars in the ordinate marginal histograms of Figure 4A, section 3.2). Figure 4B (bottom panel) shows that the average normalized matched filter, aligned to the onset of the motion pulse, was sightly wider than the 100 ms boxcar filter (gray areas are SEM). This average filter shape follows the difference in the average normalized gaussian fit to the correct and failed neural responses (middle panel).
Closer inspection, however, revealed that the matched filter centers were distributed over a wide time range (histogram in Figure 4B) and suggested that averaging the matched filters by aligning them to the motion onset caused substantial temporal smearing of the average matched-filter shape. Figure 4C shows all matched filters (each row corresponds to a single neuron's matched filter, sorted by the peak time of the matched filter) and highlights that most matched filters were relatively narrow and varied in latency relative to the onset of the motion pulse. A handful of matched filters had distinct negative peaks that appear as black among the white positive peaks. Also note that matched filters with longer peak latencies tended to be slightly wider. The marginal plot in Figure 4C shows the matched-filter neural-behavioral aROC scores aligned to the corresponding matched filter (vertical line is 0.5). There was no correlation between the time of the matched-filter peak and the aROC neural-behavioral covariation. Note that due to noise, a few matched filters did not have well-defined peaks, and this confounded their alignment in Figure 4C.
To better illustrate the fast temporal properties of the matched filters, we computed the average filter aligned to the peak (Figure 4D). The average matched-filter width was about 50 ms at half-height and confirmed that neural-behavioral covariations for single neurons usually occurred over a fairly narrow time window that was similar to the 50 ms duration of the motion stimulus. It is notable that the variability in the latency of the matched filters was much greater than the variability of filter widths.
Further inspection of the matched filters revealed that about 17% had prominent narrow negative peaks (corresponding to failed responses that were greater than correct responses; see Figure 4E). These negative-peaked matched filters can also be seen as black peaks distributed among the white positive peaks in the gray-scale plot of Figure 4C. Negative-peaked matched filters tended to correspond with smaller matched-filter aROC scores (< 0.6) and boxcar aROC values < 0.5 (open symbols in Figure 4A). Compared to the boxcar, the negative-peaked matched filters produced a higher average aROC score (p = 0.04, paired t-test), with 7 out of 11 negative boxcar aROC scores becoming positive with the match filter. Thus, the matched-filter approach appeared to have detected when firing rates were higher on failed versus correct trials. Although other filter shapes were present in our matched filters (e.g., biphasic filters), we did not observe any systematic relationship between filter shape and aROC scores for these filters. Note that for the peak-aligned plot in Figure 4D, matched filters with negative peaks were inverted before averaging. Overall, deriving the individual matched filters highlighted the heterogeneity of the latency in neural-behavioral covariances within a single area of visual cortex.
Measuring the functional link between cortical sensory neurons and behavior has become an important tool for understanding the neural basis of perception. Because a single neuron represents a small component of the overall cortical network, accurately identifying neural-behavioral covariations depends on both the number of trials and the placement and shape of the analysis window. In this study, we developed the theory and used a model to demonstrate that using a matched-filter with cross-validation approach can capture the functional link between neural activity and behavior. Importantly, the matched-filter shape provides insights into the temporal dynamics of the neural-behavioral covariation. Applying this method to individual MT neurons recorded during a brief motion stimulus demonstrated neural-behavioral covariations on timescales much faster and more variable than those measured by a fixed boxcar analysis window.
Although it has been shown that neural-behavioral covariations of sensory neurons in visual cortex are dynamic and peak soon after a stimulus (Cook & Maunsell, 2002; Thiele & Hoffmann, 2008; Cohen & Newsome, 2009; Ghose & Harrison, 2009; Price & Born, 2010; Bosking & Maunsell, 2011), only two studies have examined the timescales that best capture the correlations between neural activity and behavior. Price and Born (2010) varied parameters of a boxcar filter and found that around 80 ms was the optimal width for the population MT and MST neural-behavioral covariations during a speed detection task. The authors did not, however, optimize the boxcar filter for individual neurons. Ghose and Harrison (2009), by comparison, applied a novel mutual information approach to individual MT neurons during a motion detection task that was similar to that used here and found maximal neural-behavioral covariations in windows between 32 and 64 ms wide that were approximately matched to the duration of the motion stimulus. Although they used a different approach, the choice-related information plots for the individual MT neurons of Ghose and Harrison (2009) are very similar to our narrow matched filters with variable latency shown in Figure 4C. However, it is important to note that none of these studies, including ours, is immune to potential confounds, such as non-stationarity of neural and behavioral responses, which can bias aROC scores (Kang & Maunsell, 2012).
Our main contribution is showing that the matched filter maximizes the likelihood ratio of the ROC curve given reasonable statistical assumptions of the neural activity. Modeling demonstrated that when combined with cross-validation, the matched-filter approach converges to the true aROC neural-behavioral score for individual neurons. Although our method is unbiased, the cross-validation procedure of dividing the data in half increased the noise in our aROC estimates. We feel, however, that the benefit of estimating the matched-filter shape is that it reveals how neural-behavioral covariations arise over time. For example, when applied to our MT data, our results showed that individual MT neurons typically had neural-behavioral correlations with highly variable latencies that occurred on a short timescale similar to the 50 ms stimulus duration. As others have suggested, this raises the interesting hypothesis that downstream brain areas are themselves applying an optimal matched filter to MT activity in order to best detect the occurrence of a short motion pulse (Ghose & Harrison, 2009; Price & Born, 2010).
A further benefit of a matched filter is that its parameters can be easily optimized to a clear correct-minus-failed deterministic signal. For example, the response of individual MT neurons to the motion pulse was generally gaussian in shape, and thus it made sense to fit correct and failed responses with gaussian functions in order to estimate the matched filter. The implication of this theory, as demonstrated by applying a nonmatched boxcar filter to the model (see Figure 3), is that using any other analysis window, even one with similar spatial and temporal parameters as synaptic integration (e.g., Cook, Guest, Liang, Masse, & Colbert, 2007), would not converge to the true neural-behavioral aROC scores. This is because an ideal observer can best predict behavior based on single-trial spike trains using matched filters that mimic the time-dependent neural fluctuations of sensory neurons. This result is not surprising because ROC-based neural-behavioral covariations are based on the trial-by-trial differences in neural responses between two behaviors (e.g., correct and failed). Thus, a matched-filter analysis window that emphasizes when these differences occur is guaranteed to provide a more accurate measure of the neural-behavioral covariation.
The neural-behavioral matched filters of our single MT neurons had several interesting features. The most prominent was that the matched filters were narrow and the peaks were widely distributed in time. This variability could be tied to the variability in the latency of individual MT neurons responding to the motion pulse. For example, if we estimate the latency of the matched filter as the time the filter reaches 10% of its peak relative to the baseline, then our median matched-filter latency was 47 ms after the motion onset (10% to 90% range was 47 to 117 ms). Although there are some discrepancies, reported MT latencies range from 35 to 100 ms (Schmolesky et al., 1998; Bair, Cavanaugh, Smith, & Movshon, 2002; but see Raiguel, Lagae, Gulyas, & Orban, 1989). Thus, our range of matched-filter latencies is similar to MT response latencies with the exception of the early matched-filter latencies before 35 ms. There are two possible explanations for these early latencies. First, we have previously shown that neural-behavioral covariations can begin before the response to a sensory stimulus due to the downstream integration of neural activity (Smith, Masse, Zhan, & Cook, 2012). Second, it has also been demonstrated that changes in attentional state can contribute to neural-behavioral covariations (Herrington & Assad, 2009; Nienborg & Cumming, 2009). This is also suggested in our data by the elevated baseline before the motion pulse occurred in our population matched-filter (see Figure 4B, bottom panel). Finally, the wide latency range of matched-filter peaks could be the result of a mixture of neural-behavioral mechanisms. For example, early neural-behavioral covariations soon after the stimulus pulse could be due to fluctuations in feedforward sensory-related signals, while later neural-behavioral covariations could be due to postdetection feedback modulation (Smith et al., 2012).
Another feature of our analysis is that it revealed a small number of neurons that had negative neural-behavioral covariations that resulted in matched filters with narrow negative peaks. Observing a small number of negative neural-behavioral covariations is consistent with many previous studies. However, regardless of whether matched filters were positive or negative, the narrow width was a consistent feature of the neural-behavioral covariations for our population of MT neurons.
We do not propose that estimating neural-behavioral covariations using a matched filter with cross-validation is a replacement for the traditional fixed-window analysis in all cases. For example, the initial dynamics of the neural response during long stimulus presentations of several seconds do not seem to contribute much to the neural-behavioral covariation (Britten et al., 1996; Nienborg & Cumming, 2006). Thus, when long stimulus presentations are used, a fixed boxcar filter aligned to the stimulus presentation time is essentially the optimal matched filter. However, long stimulus presentation times do not reflect the fact that our visual system usually operates under much more dynamic conditions. For example, as one moves about the world, the visual image on the retina is constantly changing and we are sensitive to brief changes in visual stimuli that could indicate a potential collision. When these behaviorally relevant timescales of a few tens of milliseconds are used to probe the functional link between neural activity and behavior, the temporal dynamics of the cortical responses can have a much larger impact on neural-behavioral estimates (Cohen & Newsome, 2009). It is under these conditions of brief, or dynamic, stimulus presentations that a matched filter with cross-validation provides a systematic approach for revealing the temporal properties of the ROC-based neural-behavioral covariations.
We thank A. Golzar, N. G. Sadeghi, and A. Hashemi for helpful comments on this letter. This research was supported by operating grants from the Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council.