Abstract

Dating back to the 19th century, the discovery of processing stages has been of great interest to researchers in cognitive science. The goal of this paper is to demonstrate the validity of a recently developed method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA), for discovering stages directly from EEG data, in contrast to classical reaction-time-based methods. To test the validity of stages discovered with the HsMM-MVPA method, we applied it to two relatively simple tasks where the interpretation of processing stages is straightforward. In these visual discrimination EEG data experiments, perceptual processing and decision difficulty were manipulated. The HsMM-MVPA revealed that participants progressed through five cognitive processing stages while performing these tasks. The brain activation of one of those stages was dependent on perceptual processing, whereas the brain activation and the duration of two other stages were dependent on decision difficulty. In addition, evidence accumulation models (EAMs) were used to assess to what extent the results of HsMM-MVPA are comparable to standard reaction-time-based methods. Consistent with the HsMM-MVPA results, EAMs showed that nondecision time varied with perceptual difficulty and drift rate varied with decision difficulty. Moreover, nondecision and decision time of the EAMs correlated highly with the first two and last three stages of the HsMM-MVPA, respectively, indicating that the HsMM-MVPA gives a more detailed description of stages discovered with this more classical method. The results demonstrate that cognitive stages can be robustly inferred with the HsMM-MVPA.

INTRODUCTION

A central tenet of theories in cognitive neuroscience is the existence of processing stages. After the seminal work of Donders (1969) and Sternberg (1969), stages are typically identified based on behavioral responses. Although these methods are easy to apply and provide insight in the relative duration of different processing stages, it has proven difficult to identify the exact duration and temporal onsets of individual stages (e.g., Henson, 2011; Posner, 2005). Here, we demonstrate the validity of a recently proposed method, hidden semi-Markov model multivariate pattern analysis (HsMM-MVPA), as a method for discovering processing stages using neural data (EEG/magnetoencephalography [MEG]).

The HsMM-MVPA method was introduced by Anderson, Zhang, Borst, and Walsh (2016) to overcome the limitations of conventional methods based on behavioral data. The main advantage of HsMM-MVPA is that it allows parsing a cognitive task into processing stages based on the many EEG/MEG samples within a trial, unlike one reaction time (RT) measure per trial. For example, when it was applied to EEG data from the classic Sternberg task, five stages were identified: (1) preattention, (2) encoding, (3) memory retrieval, (4) decision, and (5) response (Anderson et al., 2016). The HsMM-MVPA method has been proved to be a versatile method for detecting processing stages in a range of tasks (Imani, Harati, Pourreza, & Goudarzi, 2020; Anderson et al., 2018; Portoles, Borst, & van Vugt, 2018; Zhang, van Vugt, Borst, & Anderson, 2018; Zhang, Walsh, & Anderson, 2017, 2018; Walsh, Gunzelmann, & Anderson, 2017; Zhang, Borst, Kass, & Anderson, 2017; Borst & Anderson, 2015). At the same time, the interpretation of discovered stages remains a challenge. In particular, when dealing with more complex tasks that involve a longer sequence of cognitive processes, it becomes difficult to find support for a specific cognitive interpretation.

The main goal of the current paper is to demonstrate the validity of the HsMM-MVPA as a method for discovering processing stages that can be linked to a specific cognitive interpretation. Because we do not have access to ground truth—we cannot observe cognitive processes directly—we will use experimental manipulations to provide a proxy of ground truth. In this way, robust conclusions on discovered processes can be drawn. In addition, the HsMM-MVPA will be complemented by evidence accumulation models (EAMs) that represent a traditional, well-established approach to modeling decision-making (Evans & Wagenmakers, 2020). Analogous to Donders' and Sternberg's methods, EAMs are applied to RTs; however, they provide a more formal account of behavioral data. In doing so, EAMs are able to separate cognitive processes underlying the RT distribution, in particular, decision processes from nondecision processes (e.g., Bode et al., 2018; Jepma, Wagenmakers, & Nieuwenhuis, 2012).

With this goal in mind, we designed two EEG experiments in which participants were asked to perform a simple visual discrimination task. Each experiment consisted of two conditions where different visual stimuli were used to manipulate perceptual processing. Between experiments, we maintained the same stimuli and response options but made the decision more difficult. Thus, between conditions, perceptual stages should be affected, whereas between experiments, a decision stage(s) is expected to have a different duration. Before applying the HsMM-MVPA method, we will confirm the successful operationalization of these assumptions by using EAMs.

To summarize, (1) we conducted two experiments in which we manipulated perceptual processing and decision difficulty, (2) we quantified the duration of decision and nondecision components utilizing EAMs, (3) we applied the HsMM-MVPA method to discover the underlying processing stages, and (4) we correlated the results of the EAMs with those of HsMM-MVPA models. Overall, this set of analyses confirmed the effectiveness of HsMM-MVPA as a versatile method for stage discovery and resulted in a definition of simple stages that can be used to interpret more complex tasks. In the remainder of this introduction, we will describe the main assumptions underlying the application of the HsMM-MVPA method to EEG data and explain the use of EAMs.

HsMM-MVPA

The HsMM-MVPA method decomposes EEG data into processing stages. The method is based on the assumption that any cognitive event—the start of a processing stage—is accompanied by a negative or positive peak across different brain regions. This assumption is shared between the two main theories explaining the generation of ERPs: the “classical theory” and the “synchronized oscillations theory.” According to the classical theory, phasic bursts of activity are generated when cognitive events occur (Shah et al., 2004; Schroeder et al., 1995). In this framework, background EEG is regarded as noise. The synchronized oscillations theory opposes this view and instead proposes that such peaks result from synchronization in a certain frequency band, or phase resetting, that is triggered by the event (Klimesch, Sauseng, & Hanslmayr, 2007; Makeig et al., 2002). Although these theories suggest different causes on ERP generation, simulation studies revealed no difference in activity generated by phasic bursts and phase resetting: Both theories predict significant positive or negative deflections at the start of a new cognitive process (Yeung, Bogacz, Holroyd, & Cohen, 2004). Our current assumption is that these deflections originate in processing of the BG (e.g., Rektor et al., 2003, 2004). This is consistent with the idea that the BG–thalamus circuit implements goal-directed cognitive processing, where BG events start new cognitive processing stages (e.g., Stocco et al., 2017; Stewart, Bekolay, & Eliasmith, 2012).

To identify the onset of cognitive processes, the HsMM-MVPA searches for such negative or positive deflections across the scalp and in time. These so-called “bumps” mark the transition from one processing stage to another. It is assumed that bumps are separated by “flats” that have a variable duration; together with the bumps, they comprise stages in a cognitive task. To identify the optimal number of bumps to account for EEG data, separate HsMM-MVPA models are fitted that assume different numbers of bumps. By comparing the goodness of fit of these models, the optimal number is derived, providing the temporal locations and scalp topologies of the bumps as well as the distributions of durations of the flats (see Methods for details).

Figure 1 illustrates how the EEG signal from three different trials is modeled with the HsMM-MVPA. The algorithm searches for bumps that represent the onset of a cognitive stage and flats that separate these bumps. In these flats, the EEG signal is described by sinusoidal noise around 0. This assumption is theoretical and is used to highlight that the bumps that signify cognitive events are added to ongoing EEG oscillations. In the original paper by Anderson et al. (2016), it was shown that, even if this assumption does not hold, HsMM-MVPA can robustly discover bumps that signify stages. The goal is to identify the topology and temporal location of each bump on each trial. As there is variability in the duration of cognitive processes and the associated EEG signal on each trial, the algorithm allows for bumps to occur at different time points in each trial (i.e., a perceptual stage will have a different duration on each trial, jittering the onset of the subsequent bump). To account for this, we analyze data at the single-trial level, while taking into account all trials of all participants simultaneously. Because the method assumes that the cognitive processes are the same on each trial, the topologies of the bumps are kept constant across trials (depicted by the red arrows in Figure 1). However, as the length of each process can vary per trial, the durations of the flats are variable for all participants and trials, described by gamma distributions (depicted by the blue arrows). The HsMM-MVPA model illustrated in Figure 1 contains five bumps, each marking the onset of a cognitive stage. As the first stage starts with the stimulus presentation, this model represents six stages in total; the end of the last stage is defined by the participant's response.

Figure 1. 

An illustration of the HsMM-MVPA method applied to EEG data from three trials. The resulting model contains five bumps and six stages. Red dashed arrows indicate the location of the Bump 2 on each trial, whereas blue dashed arrows indicate variability in the duration of Stage 4, which represents the process terminated with Bump 4.

Figure 1. 

An illustration of the HsMM-MVPA method applied to EEG data from three trials. The resulting model contains five bumps and six stages. Red dashed arrows indicate the location of the Bump 2 on each trial, whereas blue dashed arrows indicate variability in the duration of Stage 4, which represents the process terminated with Bump 4.

Modeling Decision Tasks with EAMs

To decompose RTs into underlying cognitive components, EAMs such as the diffusion decision model (DDM; Ratcliff, 1978) are often used. The DDM is designed for modeling data from two-choice experiments and is well established in cognitive science (Ratcliff & McKoon, 2008). The core idea of this method is that participants gather evidence for certain decisions over time before they execute a response. Because this evidence accumulation process is noisy, RTs across trials vary, and participants sometimes make mistakes. This model has been successfully applied to a wide variety of tasks (e.g., Milosavljevic, Malmaud, Huth, Koch, & Rangel, 2010; Gomez, Ratcliff, & Perea, 2007; Smith & Ratcliff, 2004).

Because the current tasks required only very simple decisions, participants' responses were mostly correct. The low error rates preclude the use of DDM for this data set, and for this reason, we opted for an EAM that assumes only a single bound (the shifted Wald model; Anders, Alario, & van Maanen, 2016; Matzke & Wagenmakers, 2009; Heathcote, 2004). The assumption of a single response boundary allows modeling a decision process that is most likely terminated by the correct response option. In Figure 2, an example of a shifted Wald model is shown. The red line represents the process of evidence accumulation with an average drift rate v. This accumulation process continues until a threshold B is reached, and the response is executed. The decision time, therefore, depends on both the drift rate v and the threshold value B. Nondecision time (t0) accounts for non-decision processes such as encoding, pre-attention and motor preparation. Together with decision time, non-decision time comprises observed RTs.

Figure 2. 

A visual representation of a shifted Wald model. The figure illustrates example data of a participant who accumulates evidence in a trial. This noisy accumulation process is represented with the red line with a mean drift rate v and is terminated with a decision boundary, or threshold, B. The time that is not related to the decision process such as perceptual processing and motor preparation is represented as t0, namely, nondecision time.

Figure 2. 

A visual representation of a shifted Wald model. The figure illustrates example data of a participant who accumulates evidence in a trial. This noisy accumulation process is represented with the red line with a mean drift rate v and is terminated with a decision boundary, or threshold, B. The time that is not related to the decision process such as perceptual processing and motor preparation is represented as t0, namely, nondecision time.

The parameters of a Wald model can be interpreted concerning their relation to cognitive processing. The drift rate value has been attributed to decision difficulty, with lower values corresponding to harder decisions (Mulder, van Maanen, & Forstmann, 2014; Basten, Biele, Heekeren, & Fiebach, 2010). The threshold value is typically associated with response caution (e.g., van Maanen et al., 2019; Boehm, van Maanen, Forstmann, & van Rijn, 2014; Bogacz, Wagenmakers, Forstmann, & Nieuwenhuis, 2010). Finally, the value of the non-decision component has been linked to enhanced attention because of stimulus anticipation (Jepma et al., 2012) and “stimulus quality” (Bode et al., 2018).

We will now describe two EEG experiments across which we varied perceptual processing and decision difficulty. Both experiments consisted of two conditions where different types of stimuli were used to manipulate perceptual processing. Between experiments, we used the same stimuli but adapted the task to vary decision difficulty. To exclude effects of task interference between Experiment 1 and Experiment 2 (given that we used different response rules with identical stimuli), two separate experiments were conducted. Overall, assuming that HsMM-MVPA is a versatile method for discovering cognitive stages in a task, we hypothesized that (1) the topological distributions of HsMM-MVPA-identified perceptual stage(s) will differ by stimulus type, (2) the duration of HsMM-MVPA decision stage(s) and the topology will be different when we manipulate task difficulty, and (3) these differences should be consistent with nondecision time and drift rate values in corresponding Wald models.

METHODS

Participants

In Experiment 1, 30 participants performed a simple visual discrimination task. Data of five participants were excluded (three because of the problems in EEG recording, one for excessive noise in the EEG, and one for not following task instructions), resulting in a final set of 25 participants (13 women; mean age = 24.72 years, SD = 4.49, range = 18–33 years).

In Experiment 2, 30 participants performed a more difficult visual discrimination task. Data of five participants were excluded from the analysis (three because of a programming error, one for major eye movement contamination that could not be captured with independent component analysis, and one for EEG malfunction), resulting in a final set of 25 participants (16 women; mean age = 23.92 years, SD = 4.8, range = 18–36 years).

Participants in both experiments were students of the University of Groningen or the Hanze University of Applied Sciences in Groningen, The Netherlands. All participants were right-handed, had normal or corrected-to-normal vision, and had no history of cognitive impairment and normal color vision as assessed by the EnChroma Color Blindness Test (EnChroma, Inc.1). Before the experiments, participants gave written informed consent to the experimental procedures, as approved by the research ethics review committee of the Faculty of Arts at University of Groningen (reference number: 62838174 for Experiment 1, 64834780 for Experiment 2), in accordance with the Declaration of Helsinki. Participants received monetary compensation of 8 euros.

Task Design

In both experiments, participants were presented with geometric shapes or with character strings. Their task was to discriminate between different stimuli by using one of the two randomly assigned response keys (“M” or “N”). On the basis of stimulus type, the conditions are labeled “Shapes” or “Characters.” As we varied the decision difficulty between the two experiments, we will refer to Experiment 1 as “Easy” and Experiment 2 as “Difficult.” We will, therefore, refer to the Shapes condition of Experiment 1 as the “Easy Shapes” condition and the Characters condition of Experiment 1 as the “Easy Characters” condition. In line with that, the two conditions of Experiment 2 will be further referred to as “Difficult Shapes” and “Difficult Characters.”

The Easy Shapes condition was presented in two blocks. Four shapes (circle, triangle, square, and rhombus) of four colors (red = #ff0000, green = #55aa00, yellow = #ffff00, blue = #0000ff) were used as stimuli. Red–green and blue–yellow colors were chosen in line with the opponent process theory where they are considered the highest contrasting pairs (Goldstein, Humphreys, Shiffrar, & Yost, 2008; Hurvich & Jameson, 1957). In Block 1, two shapes of two colors were randomly selected from the set, and participants were instructed to respond based on the shape of the stimulus and to ignore the color. In Block 2, the other two shapes and colors were used, and the task was to respond based on the color of the stimulus and ignore the shape. We will collapse over those two blocks in our analyses (the results of these blocks were also analyzed separately; however, no differences were found).

The Easy Characters condition was presented in one block. In this condition, participants were instructed to respond whether the characters were letters or numbers. The stimuli consisted of four random uppercase letters of the Latin alphabet (e.g., BDRN or AENJ) or four random digits from 0 to 9 (e.g., 3476 or 8168).

The Difficult Shapes condition was presented in two blocks. In this condition, participants were presented with the same stimuli as in the Easy Shapes condition; however, the task was more difficult: They were instructed to respond by considering both shape and color of the objects. For example, if in Block 1, the stimuli were red or green triangles or circles, the participant was instructed to press “M” when a red circle or a green triangle was presented and “N” for the other two stimuli (Figure 3). Thus, both features—shape and color—had to be taken into account. For Block 2, the task was the same—only different shapes and colors were used.

Figure 3. 

Response mapping in all conditions. In Easy Shapes, participants were asked to discriminate between two shapes of different colors (the color should be ignored) or based on two different colors of the shape (then the shape should be ignored). In Easy Characters, participants were asked to discriminate between two characters—a letter or a number. In Difficult conditions, the same stimuli were used while the decision difficulty was manipulated.

Figure 3. 

Response mapping in all conditions. In Easy Shapes, participants were asked to discriminate between two shapes of different colors (the color should be ignored) or based on two different colors of the shape (then the shape should be ignored). In Easy Characters, participants were asked to discriminate between two characters—a letter or a number. In Difficult conditions, the same stimuli were used while the decision difficulty was manipulated.

The Difficult Characters condition was presented in one block. In this condition, participants were presented with the same stimuli as in the Easy Characters condition and were instructed to discriminate between consonants, vowels, odd numbers, and even numbers. An example of this condition is illustrated in Figure 3 where the participant was instructed to press “M” for even numbers and consonants and “N” for odd numbers and vowels.

To summarize, we designed Shapes and Characters conditions where we used stimulus type to manipulate perceptual processing and Easy and Difficult conditions where we manipulated decision difficulty.

Procedure

Participants were seated in front of a 21.5-in. screen with a resolution of 1366 × 768 pixels. Experiment 1, referred to as Easy conditions, contained 480 trials: three blocks consisting of 20 practice trials and 140 main trials. Experiment 2, referred to as Difficult conditions, contained 456 trials: three blocks consisting of 12 practice trials and 140 main trials. The presentation of the blocks was counterbalanced across participants.

Each block started with instructions, directly followed by practice trials. After the experimenter ensured the task was sufficiently understood, the experimental trials commenced. Each trial started with a fixation dot, presented for a duration sampled from a uniform distribution (from 1500 to 2250 msec), followed by the presentation of the stimulus that remained on the screen until one of the two response keys (“N” or “M”) was pressed or the response deadline was reached. In the Easy conditions, the deadline was set to 3000 msec, and in the Difficult conditions, it was increased to 5000 msec to reflect the increased difficulty of the task. Feedback was provided for 500 msec (“correct,” “incorrect,” or “late”). Between blocks, participants were given the option to take a short break. Each block took around 10 min. In total, data acquisition for both Easy and Difficult conditions lasted 1 hr, including EEG setup and instructions.

Behavioral Analysis

For the behavioral analysis, we removed practice trials and incorrect trials. For each of the four conditions (Easy Shapes, Easy Characters, Difficult Shapes, and Difficult Characters) and each participant, we then excluded trials that deviated more than 2 SDs from the mean RTs of the participant and the condition (this preprocessing pipeline was used for all consecutive analyses, and the qualitative results are identical when the “outliers” are kept in the data; we used this rather strict outlier regimen to stay in line with previous HsMM-MVPAs).

To statistically evaluate RTs and accuracy, linear mixed-effects models (LME models) were constructed (Bates & DebRoy, 2004). The evaluation of models was done with a forward stepwise fitting routine: We started with simple models where only one predictor was included (stimulus type or decision difficulty), and then the models with both predictors were evaluated. Finally, the estimation terminated with the full model that included the interaction between stimulus type and decision difficulty. The lmerTest R package was used to obtain p values for fixed effects based on Satterthwaite's method (Kuznetsova, Brockhoff, & Christensen, 2018). Not only fixed effects but also random effects can potentially lead to statistical errors. To prevent the latter, the forward fitting procedure was separately applied to random effects. Thus, we determined the maximum random effects structure allowed by the data (Bates, Kliegl, Vasishth, & Baayen, 2015; Barr, Levy, Scheepers, & Tily, 2013).

Evidence Accumulation Modeling

Only the RTs for correct responses were modeled because the error rates for all conditions and all participants were, on average, lower than 5%. In addition, trials that were marked as outliers were not modeled, consistent with all other analyses in the current paper (see Behavioral Analysis section). The model parameters were estimated using differential evolution Markov chain Monte Carlo algorithm (Sherri, Boulkaibet, Marwala, & Friswell, 2019; Turner, Sederberg, Brown, & Steyvers, 2013) as implemented in the Dynamic Models of Choice software (Heathcote et al., 2019). A prior probability distribution for each of the parameters was specified as a truncated normal distribution. The convergence of samples was assessed with Gelman and Rubin's potential scale reduction factor (Gelman & Rubin, 1992). To reduce the autocorrelation between different Markov chain Monte Carlo samples, thinning was applied.

We hypothesized that different conditions might lead to differences in the parameters of EAMs. To test this assumption, we estimated all possible models where the parameters drift rate (v), threshold (B), and nondecision time (t0) differed by stimulus type, separately for the Easy and Difficult conditions. The estimation started with the simplest model, the intercept-only model, and terminated with the most complex model, where all parameters were assumed to differ per condition. All the constructed models contained v, B, and t0 parameters. However, in the simpler models where parameters were constrained across conditions, we only estimated a single v, B, and t0 parameter. In more complex models, we let one or more of those parameters vary by condition.

Furthermore, for each model, the Watanabe–Akaike information criterion (WAIC) was computed (Vehtari, Gelman, & Gabry, 2017), and WAIC values were derived for each participant. We then calculated WAIC weights analogous to Akaike information criterion weights' computation procedure (Wagenmakers & Farrell, 2004) to allow for a more straightforward interpretation of WAIC differences in the estimated models. To account for individual variability in the fitted models, we applied model averaging (Miletić & van Maanen, 2019; Hoeting, Madigan, Raftery, & Volinsky, 1999). For each participant, the estimated values from the models were weighted by the corresponding WAIC weights. Then, the average over all participants was calculated per parameter (v, t0, B). For the models where parameters (v, t0, B) did not differ per condition, the values for the different conditions were derived from the shared value of this parameter.

EEG Recording and Preprocessing

The EEG was recorded from 32 positions using active Ag–AgCl electrodes (BioSemi ActiveTwo system) digitized with a sampling rate of 512 Hz. The electrodes were placed using the International 10–20 system layout including two “ground” channels—Common Mode Sense and Driven Right Leg. Two horizontal and two vertical electrodes were used to measure eye movements and blinks. Data were post hoc referenced to the average of the mastoids. Scalp impedance for each electrode was kept under 20 kΩ for all except four participants, for whom it was kept under 30 kΩ.

For EEG preprocessing and analysis, the open-source toolbox EEGLAB (Delorme & Makeig, 2004) was used along with custom-made scripts in MATLAB (The MathWorks, Inc.). EEG data were passed through a high-pass filter of 1 Hz and a low-pass filter of 40 Hz. Next, data were downsampled to 256 Hz. Manual artifact rejection was performed on continuous data by visual inspection of the data and rejection of data portions containing noise. On average, 3.7% of the data were removed because of artifacts. For six participants, one or two noisy channels were removed. Furthermore, data were decomposed with independent component analysis to remove eye blinks and muscle artifacts (using EEGLAB's runica function, which is based on a logistic infomax algorithm; Bell & Sejnowski, 1995). Typically, one or two components were identified to account for eye artifacts or muscle movements and were subsequently subtracted from the data. The channels that were removed were topographically reconstructed using spherical spline interpolation.

Preprocessing for HsMM-MVPA

For HsMM-MVPA, additional preprocessing procedures were applied. First, the data was downsampled to 100 Hz for computational trackability. Next, the data were epoched on a trial-by-trial basis relative to the onset of the presentation of the stimuli and consecutive response. That is, the analysis is performed on all data points of all trials between stimulus and response. The trials that were marked as outliers in the behavioral analysis were excluded. Next, a baseline from 400 msec preceding the stimulus appearance was computed and subtracted. Only complete trials were subjected to the analysis (incomplete trials appeared because of artifact rejection). A covariance matrix was computed for each trial and participant separately by multiplying normalized data by its transpose (Portoles et al., 2018; Cohen, 2014). Next, the mean covariance for all trials for all participants was computed and subsequently used as input for principal component analysis (as implemented in MATLAB's pcacov function). The first 10 principal components accounting for 94.8% of the variance were retained. Finally, the data were normalized by calculating z scores.

HsMM-MVPA

The HsMM-MVPA was applied to discover cognitive stages. A hidden Markov model is a type of stochastic model that connects a sequence of hidden states to a sequence of corresponding observations. Whereas the hidden Markov model assumes that one observation corresponds to one state, in the hidden semi-Markov model, several observations can be produced during a state and each of these states, therefore, has a variable duration (Yu, 2010). In the current application of HsMM-MVPA, principal components extracted from the EEG signal act as observations, and the analysis aims to find cognitive stages that are hidden. To identify the onsets of these stages, we search for bumps—positive or negative deflections with a duration of 50 msec. This duration was chosen as it produces robust results even if the actual bumps are slightly shorter or longer (Anderson et al., 2016). The process that is indicated by a bump continues within a flat; the mean amplitude of the signal during the “flat” is equal to zero. Together with bumps, flats comprise cognitive stages. The duration of these flats was modeled with a gamma distribution with a shape parameter of 2. The placement of n bumps results in n + 1 flats because of the first stage starting with a flat.

For Easy and Difficult conditions, different HsMM-MVPA models will be constructed. The model estimation starts with a single-bump model and increases to the number of bumps nmax, which is the maximum number of bumps (each taking 50 msec) that fit in the shortest observed RT. During the estimation of an HsMM-MVPA model, two parameters of the hidden states are obtained: (1) the amplitudes of the bumps that mark the onsets of cognitive stages and (2) the scale parameter of a gamma distribution that describes the stage durations (the shape parameter is fixed at 2). These parameters are estimated by taking into account the data from all trials and all participants simultaneously. This is performed in a way that maximizes the match between the EEG data and the model with a standard expectation–maximization (EM) algorithm.

The fitting process starts with defining initial amplitudes for the bumps and gamma distributions for stage durations. As the convergence of the EM algorithm is sensitive to the choice of a starting point (Wu, 1983) and might end up in a local maximum, one option is to use different random values for bump amplitudes and compare their outputs (e.g., Portoles et al., 2018). Here, we take another approach based on the work by Zhang, Walsh, et al. (2018). We firstly fitted separate HsMM-MVPA models for each condition with the maximum number of bumps possible (nmax) and obtained the bump amplitudes and gamma distributions. Next, these parameters were used for models with fewer bumps (nmax − 1) where we iteratively left out each of the bumps of nmax. Then, all different nmax − 1 models were compared, and the model with the best fit was selected. This process was repeated until a model with only a single bump (n1) was fitted. The idea was to find all potential bump topologies while avoiding local maxima. The amplitudes of the bumps obtained in this way have been further used as inputs for subsequent HsMM-MVPA models.

The log-likelihood of HsMM-MVPA models tends to increase when more bumps are fitted, as there are more parameters to fit the data. To avoid overfitting, we applied a leave-one-out cross-validation procedure. We estimated the HsMM-MVPA model on all participants but one and then tested the fit of this model on the left-out participant, effectively separating training and testing of the models. This was repeated for all participants. Finally, we tested for how many participants the log-likelihoods of the models with n + 1 bumps increased compared to an n-bump model using a sign test. In this way, we evaluated whether a model with one additional bump outperformed the previous model for a sufficiently large number of participants and thus whether the additional complexity of the model was warranted. The main reasoning behind this was to derive the model that generalized across participants (Anderson & Fincham, 2014). As a result, the optimal number of bumps in each condition was obtained.

For more details, a mathematical description, and code of the HsMM-MVPA, we refer to Anderson et al. (2016).

RESULTS

Behavioral Results

Average error rates and RTs along with within-participant standard errors (Morey, 2008) were computed for each condition and are shown in Figure 4. The proportion of errors in all conditions was found to be low (not exceeding 5%). To quantify the differences in behavioral performance for both stimulus type and decision difficulty, we constructed LMEs. Using a stepwise fitting procedure, we ended up with the same fixed and random effects structure for RT and accuracy (p < .001 for RT and p < .05 for accuracy): stimulus type, decision difficulty, and their interaction as fixed effects; participant as a random intercept; and stimulus type and trial as random slopes.

Figure 4. 

Mean RTs and error rates with within-participant standard errors for each condition.

Figure 4. 

Mean RTs and error rates with within-participant standard errors for each condition.

First, RTs were plotted against a theoretical normal distribution, and as they were found to be right-skewed, a logarithmic transformation was applied. Next, an LME model with log (RT) as a dependent variable was fitted. For the Easy conditions, stimulus type reached significance, indicating that participants were faster when discriminating between Easy Shapes than Easy Characters. For Difficult Conditions, Figure 4 suggests the inverse effect of stimulus type, but it did not reach significance. For both Shapes and Characters, decision difficulty had a significant effect on RTs; namely, RTs increased when the same stimuli were presented but the decision was more difficult (for statistics, see Table 1). In addition, for all conditions, the interaction between stimulus type and decision difficulty was significant.

Table 1. 
The Results of LME Models for log (RT) and Accuracy as Dependent Variables
 Response Variable
Log (RT)Accuracy
Estimatet Valuep ValueEstimatez Valuep Value
Reference: Easy Shapes 
Intercept 6.25 168.73 <.001 4.78 22.28 <.001 
Stimulus type 0.14 3.88 <.001 −0.27 −1 .31 
Decision difficulty 0.47 8.99 <.001 −1.27 −4.63 <.001 
Stimulus Type × Decision Difficulty −0.18 −3.51 <.001 0.82 2.47 <.05 
  
Reference: Difficult Characters 
Intercept 6.68 143.34 <.001 4.06 15.2 <.001 
Stimulus type 0.04 1.08 .28 −0.55 −2.35 <.05 
Decision difficulty −0.28 −4.29 <.001 0.44 1.19 .23 
 Response Variable
Log (RT)Accuracy
Estimatet Valuep ValueEstimatez Valuep Value
Reference: Easy Shapes 
Intercept 6.25 168.73 <.001 4.78 22.28 <.001 
Stimulus type 0.14 3.88 <.001 −0.27 −1 .31 
Decision difficulty 0.47 8.99 <.001 −1.27 −4.63 <.001 
Stimulus Type × Decision Difficulty −0.18 −3.51 <.001 0.82 2.47 <.05 
  
Reference: Difficult Characters 
Intercept 6.68 143.34 <.001 4.06 15.2 <.001 
Stimulus type 0.04 1.08 .28 −0.55 −2.35 <.05 
Decision difficulty −0.28 −4.29 <.001 0.44 1.19 .23 

For accuracy as a dependent variable, a binomial LME model was fitted. For the Shapes condition, decision difficulty had a significant effect but did not reach significance for the Characters condition. In addition, for Difficult conditions, stimulus type had a significant effect on accuracy. For all conditions, the interaction between stimulus type and decision difficulty was significant.

To summarize, decision difficulty had the largest effect: It led to significantly longer RTs for both stimulus types. In addition, participants answered slower to Characters than to Shapes, but only when the decision was easy.

Evidence Accumulation Results

To identify how these differences in RTs were caused by decision and nondecision components, we fitted shifted Wald models to RTs. Various models—starting from the intercept-only model to the most complex model where drift rate (v), nondecision time (t0), and threshold (B) could differ—were compared with regard to WAIC weights. These weights represent the probability that a particular model is the “true” model (i.e., the model that generated the data) under the assumption that the “true” model is in the set of compared models (Wagenmakers & Farrell, 2004). The quantified WAIC weights showed that there was no clear winning model across participants. Consequently, model averaging was applied (see Evidence Accumulation Modeling section). The resulting weighted values are shown in Figure 5.

Figure 5. 

The average weighted values (weighted by the corresponding WAIC weights) are presented for each condition and for each of the Wald model parameters: drift rate, nondecision time, and threshold.

Figure 5. 

The average weighted values (weighted by the corresponding WAIC weights) are presented for each condition and for each of the Wald model parameters: drift rate, nondecision time, and threshold.

To test whether these values differed per stimulus type and decision difficulty, separate LME models were constructed with parameter (v, t0, B) as the dependent variable and participant as a random effect. The model estimation was performed following a forward stepwise fitting routine: It started with the simple models (stimulus type or decision difficulty as the only predictor) and terminated with the full models (main effects of Stimulus type and Decision difficulty as well as their interaction as predictors). To compare these models, we computed Bayes factors (BFs) with the BayesFactor package (Morey et al., 2015). BF represents relative evidence in favor of null and alternative hypotheses provided by the data (Kass & Raftery, 1995). In terms of model comparison, reported BFs indicate whether the data favor a model with an effect over a model with no effect, the intercept-only model.

For the drift rate, the BFs provided strong evidence for the model with decision difficulty (see Table 2 for statistics). This model outperformed all other models: versus model with stimulus type, BF = 65762.5 (±0.03%); versus “main-effects model” of stimulus type and decision difficulty, BF = 4.23 (±0.9%); and versus “full model” with main effects of stimulus type and decision difficulty and their interaction, BF = 3.59 (±1.11%). For the threshold parameter, BFs provided evidence against models with stimulus type and decision difficulty. For the nondecision time, BFs provided strong evidence for the model with stimulus type. This model outperformed all the other models except for the “main-effects model”: versus model with decision difficulty, BF = 6056.83 (±0%); versus “main-effects model,” BF = 1.16 (±9.4%); and versus “full model,” BF = 3.78 (±4.16%). Although both models—the model with stimulus type and the “main-effects model”—describe the data well, we were inclined to proceed with the model with stimulus type as it has the highest BF value compared to the intercept-only model and is the more parsimonious model.

Table 2. 
BFs for the Estimated Wald Parameters (Drift Rate, Threshold, and Nondecision Time)
PredictorsResponse Variable
Drift Rate (v)Threshold (B)Nondecision Time (t0)
Stimulus type 0.23 ± 0.03% 0.21 ± 0.03% 3791.48 ± 0%a 
Decision difficulty 15227.53 ± 0%a 0.23 ± 0.03% 0.63 ± 0% 
Main-effects model (stimulus type + decision difficulty) 3595.3 ± 0.9% 0.05 ± 1.86% 3259.01 ± 9.4% 
“Full model” (Stimulus Type × Decision Difficulty) 4245.34 ± 1.11% 0.01 ± 8.82% 1004.02 ± 4.16% 
PredictorsResponse Variable
Drift Rate (v)Threshold (B)Nondecision Time (t0)
Stimulus type 0.23 ± 0.03% 0.21 ± 0.03% 3791.48 ± 0%a 
Decision difficulty 15227.53 ± 0%a 0.23 ± 0.03% 0.63 ± 0% 
Main-effects model (stimulus type + decision difficulty) 3595.3 ± 0.9% 0.05 ± 1.86% 3259.01 ± 9.4% 
“Full model” (Stimulus Type × Decision Difficulty) 4245.34 ± 1.11% 0.01 ± 8.82% 1004.02 ± 4.16% 

Reported BFs indicate relative fit by comparing these models to an intercept-only model.

a

Indicate the models with the highest BFs.

To summarize, shifted Wald models were fitted to the RT data from all conditions. Consistent with our hypothesis, we found evidence that decision difficulty affected the drift rate of the decision-making process and that stimulus type affected the nondecision time.

HsMM-MVPA Results

To gain more insight into the processing stages that comprise decision and nondecision processes, the HsMM-MVPA was applied to all conditions separately to find the optimal number of stages. For all conditions, a model with four bumps and five stages accounted best for the data. Next, to test whether the discovered cognitive stages differed per stimulus type, for Easy and Difficult conditions, a range of HsMM-MVPA models were constructed in which bumps and stage durations were shared in different ways. The model estimation was performed with a forward stepwise fitting routine and included a general model where we hypothesized that scalp topologies and stage durations for both conditions are the same, models where we varied each consecutive bump between conditions, and, finally, completely separate models for the two conditions. The model fits were compared for each participant, and a model was preferred when the fit improved for a significant number of participants compared to a simpler model. Table 3 lists the results.

Table 3. 
The Results of Model Comparison for the Estimated HsMM-MVPA Models
 General ModelBump 1Bump 2Bump 3Bump 4Bump 2 + Stage 3Sum of Separate Models
Easy Shapes and Easy Characters 
General model 
Bump 1 17 
Bump 2 20 17 13 14 
Bump 3 19 16 12 11 
Bump 4 21 19 11 14 
Bump 2 + Stage 3a 23 21 18 18 17 
Sum of separate models 23 22 18 20 20 17 
  
 General ModelBump 1Bump 2Bump 3Bump 4Stage 4Sum of Separate Models
Difficult Shapes and Difficult Characters 
General modela 13 10 13 12 13 
Bump 1 12 10 14 13 12 15 
Bump 2 16 15 14 17 16 15 
Bump 3 15 11 11 16 15 16 
Bump 4 12 12 11 
Stage 4 13 13 10 19 14 
Sum of separate models 12 10 10 14 11 
 General ModelBump 1Bump 2Bump 3Bump 4Bump 2 + Stage 3Sum of Separate Models
Easy Shapes and Easy Characters 
General model 
Bump 1 17 
Bump 2 20 17 13 14 
Bump 3 19 16 12 11 
Bump 4 21 19 11 14 
Bump 2 + Stage 3a 23 21 18 18 17 
Sum of separate models 23 22 18 20 20 17 
  
 General ModelBump 1Bump 2Bump 3Bump 4Stage 4Sum of Separate Models
Difficult Shapes and Difficult Characters 
General modela 13 10 13 12 13 
Bump 1 12 10 14 13 12 15 
Bump 2 16 15 14 17 16 15 
Bump 3 15 11 11 16 15 16 
Bump 4 12 12 11 
Stage 4 13 13 10 19 14 
Sum of separate models 12 10 10 14 11 

The reported values express the number of participants for whom the model improved (row compared to column). The numbers in bold indicate a significantly better model prediction (as determined by a sign test, p < .05).

a

The best models for Easy and Difficult conditions.

For the Easy conditions, the model where Bump 2 varied per stimulus type had the highest average log-likelihood over participants, except for the completely separate models. As a cognitive stage in an HsMM-MVPA model is represented in both bump topology and stage duration, the duration of the consecutive stage (Stage 3) was also varied. The resulting model in which we varied both Bump 2 and the duration of Stage 3 significantly outperformed all other combined models and was not outperformed by any other model. Although this was also the case for completely separate models for each condition, these models require more parameters. We therefore decided to proceed with the more parsimonious model, in which only Bump 2 and Stage 3 were different per stimulus type.

For the Difficult conditions, we also estimated models where each bump was allowed to be different between conditions. However, none of these models outperformed the general model (Table 3). Because we observed a difference in duration of Stage 4 for the different stimulus types in the initial models, we complemented the analysis with a model where Stage 4 was varied. However, this model also did not outperform the general model.

The topographical representations and stage onsets of the final models are presented in Figure 6, and the average duration of these stages and the standard errors are presented in Figure 7. As Figure 6 suggests, even the topologies resulting from different analyses were highly correlated, suggesting robustness of the method (Bump 1: Easy–Difficult = 0.84; Bump 2: Easy Characters–Easy Shapes = 0.81, Easy Characters–Difficult Shapes/Characters = 0.96, Easy Shapes–Difficult Shapes/Characters = 0.76; Bump 3: Easy–Difficult = 0.97; Bump 4: Easy–Difficult = 0.76).

Figure 6. 

The topographical representations and temporal locations of the resulting HsMM-MVPA stages plotted per condition. Our cognitive interpretation of these stages is described in the Functional Interpretation of HsMM-MVPA Stages section.

Figure 6. 

The topographical representations and temporal locations of the resulting HsMM-MVPA stages plotted per condition. Our cognitive interpretation of these stages is described in the Functional Interpretation of HsMM-MVPA Stages section.

Figure 7. 

The average stage durations with standard errors per condition from the HsMM-MVPA models.

Figure 7. 

The average stage durations with standard errors per condition from the HsMM-MVPA models.

Finally, to investigate the effect of Decision difficulty, the discovered HsMM-MVPA stages were compared between Easy and Difficult conditions. The durations of the resulting stages were visually inspected (Figure 7), and the stages that seemed to be different for decision difficulty—Stage 4 and Stage 5—were subjected to statistical analysis. Thus, an LME model with stage durations as a dependent variable, decision difficulty as a predictor, and participant as a random intercept was fitted. To assess the relative fit of these models, BFs were computed (BayesFactor package; Morey, 2008). BFs provided strong evidence for the effect of decision difficulty for Stage 4 (1.9 × 1016 ± 0%) and for Stage 5 (3.5 × 1019 ± 0%). Not only the duration of Stage 4 and Stage 5 but also the topology of the Bump 4 located between these stages varies per decision difficulty (Figure 6). This bump is characterized by higher amplitude for Easy conditions with the most prominent central positivity.

To summarize, for all conditions, an HsMM-MVPA model with five stages accounted best for the data. For the Easy conditions, the HsMM-MVPA demonstrated that Bump 2 and the duration of Stage 3 varied by stimulus type, whereas the other stages were shared. For the Difficult conditions, no stages were found to vary by stimulus type. When comparing Easy and Difficult conditions, the duration of Stage 4 and Stage 5 and the topology of Bump 4 located between them were found to vary by decision difficulty.

Identifying Evidence Accumulation Stages with HsMM-MVPA

We hypothesized that the stages discovered with the HsMM-MVPA method should be consistent with the results of the EAMs. To this end, we compared the duration of processing stages of the HsMM-MVPA models with the results of the Wald models. We assume that the HsMM-MVPA method decomposes the Wald decision time (based on drift rate and threshold parameters) and nondecision time into a more detailed sequence of processing stages. This suggests that a particular sum of stage durations in the HsMM-MVPA model should correspond to either the Wald decision or nondecision durations. If this assumption holds, a regression model fitted to the corresponding durations in HsMM-MVPA and Wald should have a slope not different from 1.

To test this assumption, we first computed the duration of the decision process in Wald model by dividing the threshold value (B) by drift rate (v). The nondecision time is simply given by the nondecision parameter t0. Next, linear regression models with an intercept of 0 were fitted to the durations. In these models, Wald decision or nondecision time acted as the dependent variable, whereas various HsMM-MVPA stages and their combinations were used as predictors. If the slope value for these models is not different from 1, it can be inferred that the methods produce the same values.

For the Wald decision time, various regression models were constructed with a single HsMM-MVPA stage or a sum of consecutive stages as predictors (for an overview of the values from all estimated models, see Table 4). To test which of these stages represent the Wald decision time, the resulting models were compared to a restricted model where the slope value is 1. Next, the F statistic and corresponding p values were computed for these models. The model with the combined duration of the HsMM-MVPA Stages 3, 4, and 5 was the only model with a slope that was not different from 1 (p > .05). Figure 8A illustrates the clear relationship between the duration of these stages and the duration of Wald decision time.

Table 4. 
The Slope Values for the Regression Models with Wald Durations Predicted by HsMM-MVPA Stages
Wald Decision Time as a Response VariableWald Nondecision Time as a Response Variable
PredictorsSlope ValuePredictorsSlope Value
Stage 3 + Stage 4 + Stage 5 0.983a Stage1 + Stage 2 1.032a 
Stage 3 + Stage 4 1.114 Stage 2 + Stage 3 0.796 
Stage 2 + Stage 3 + Stage 4 + Stage 5 0.845 Stage 1 + Stage 2 + Stage 5 0.688 
Stage 1 + Stage 2 + Stage 3 + Stage 4 0.837 Stage 3 1.315 
Stage 4 + Stage 5 1.25 Stage 1 + Stage 2 + Stage 3 0.588 
Stage 4 1.384 Stage 4 0.383 
Stage 1 + Stage 2 + Stage 3 1.58 Stage 4 + Stage 5 0.374 
Stage 2 + Stage 3 2.114 Stage 3 + Stage 4 0.34 
Stage 1 + Stage 2 2.843 Stage 3 + Stage 4 + Stage 5 0.313 
Stage 3 3.422 Stage 2 + Stage 3 + Stage 4 + Stage 5 0.274 
Stage 5 4.247 Stage 1 + Stage 2 + Stage 3 + Stage 4 0.268 
Stage 2 5.3 Stage 5 1.743 
Stage 1 6.115 Stage 1 2.2 
Wald Decision Time as a Response VariableWald Nondecision Time as a Response Variable
PredictorsSlope ValuePredictorsSlope Value
Stage 3 + Stage 4 + Stage 5 0.983a Stage1 + Stage 2 1.032a 
Stage 3 + Stage 4 1.114 Stage 2 + Stage 3 0.796 
Stage 2 + Stage 3 + Stage 4 + Stage 5 0.845 Stage 1 + Stage 2 + Stage 5 0.688 
Stage 1 + Stage 2 + Stage 3 + Stage 4 0.837 Stage 3 1.315 
Stage 4 + Stage 5 1.25 Stage 1 + Stage 2 + Stage 3 0.588 
Stage 4 1.384 Stage 4 0.383 
Stage 1 + Stage 2 + Stage 3 1.58 Stage 4 + Stage 5 0.374 
Stage 2 + Stage 3 2.114 Stage 3 + Stage 4 0.34 
Stage 1 + Stage 2 2.843 Stage 3 + Stage 4 + Stage 5 0.313 
Stage 3 3.422 Stage 2 + Stage 3 + Stage 4 + Stage 5 0.274 
Stage 5 4.247 Stage 1 + Stage 2 + Stage 3 + Stage 4 0.268 
Stage 2 5.3 Stage 5 1.743 
Stage 1 6.115 Stage 1 2.2 

These values are ordered by their absolute difference from a slope of 1 (starting from the model with the slope closest to 1).

a

The values significantly not different from 1 (p > .05).

Figure 8. 

Comparison of the durations from HsMM-MVPA and Wald modeling. The points in the figure indicate the values for each participant and condition and are supplemented by a diagonal line.

Figure 8. 

Comparison of the durations from HsMM-MVPA and Wald modeling. The points in the figure indicate the values for each participant and condition and are supplemented by a diagonal line.

The Wald nondecision time should logically be represented in the remaining two HsMM-MVPA stages: Stage 1 and 2. To test this assumption, various regression models were fitted for Wald nondecision time (Table 4). In line with our hypothesis, the regression model with the combined duration of HsMM-MVPA Stages 1 and 2 was the only model with a slope not different from 1 (p > .05). The Wald nondecision time along with cumulative first two stages from HsMM-MVPA are represented in Figure 8B. It should be noted that, although the two methods produce similar findings for the duration of nondecision time, in the Wald models, these processes have a larger variation between participants.

To summarize, there is a clear correspondence between the decision times from the two methods, where the decision time of the EAMs corresponded to the last three stages of the HsMM-MVPA. Furthermore, the nondecision time for the EAMs corresponded to the first two stages.

Functional Interpretation of HsMM-MVPA Stages

In all four conditions, we found evidence for the existence of five cognitive stages. The next step is to interpret those stages in terms of the underlying “cognitive processes.” The EAMs suggest that Stages 1–2 are perceptual, whereas Stages 3–5 concern decision-making. Here, we will review the literature to obtain independent evidence for such a functional interpretation of the discovered stages.

The first bump of all conditions has an average onset of around 100 msec and is characterized by central–parietal negativity (Figure 6). These findings resemble a standard ERP N1 component that has been largely associated with attention (Luck, 2005; Luck, Woodman, & Vogel, 2000). Stage 1 is, therefore, most likely a “preattention” stage. Although Bump 2 was found to be different between Easy Shapes and Easy Characters, on close inspection, it appears that this bump has very similar topological distributions in all conditions, with different amplitudes. With its average onset of around 200 msec and prominent frontal positivity (most salient for the Difficult conditions), the bump resembles a P2 component. Since its discovery, this component has been largely associated with attention (e.g., Miltner, Johnson, Braun, & Larbig, 1989; Rugg, Milner, Lines, & Phalp, 1987) and more recently with initial word processing (Lee, Liu, & Tsai, 2012; Mangels, Picton, & Craik, 2001). This provides evidence for the interpretation of Stage 2 as the stimulus processing stage. Taken together, this fits the interpretation of these stages as nondecision stages of the EAMs.

In its frontocentral negativity and average onset, Bump 3 is similar to the FN400 component that has been widely accepted to refer to familiarity-driven recognition (Paller, Voss, & Boehm, 2007; Curran, 2000; Rugg et al., 1998). In an alternative approach, this frontal negativity effect was linked to a decisional account (Hayama, Johnson, & Rugg, 2008; Dobbins & Han, 2006). The frontal negativity and average onset of Stage 3 are compatible with both interpretations. Thus, if we accept the familiarity interpretation of the frontal negative component, we would assume that, during this stage, the participants were recognizing the patterns in the stimuli (e.g., shape or color). If we accept the decisional interpretation, we would assume that this activity arises from a number of internal decisions participants had to make. The current findings cannot arbitrate between these different interpretations; however, given the current task, we tentatively propose the interpretation of this stage as familiarity stage.

In both the Shapes and Characters conditions, the last two stages (Stage 4 and Stage 5) were found to vary in their duration by decision difficulty. The largest difference was represented in Stage 4—being significantly longer when the decision was more difficult. In addition, Bump 4 that terminated this stage had a different topology with a more positive-going waveform across the scalp when the decision was easier. We, therefore, interpret Stage 4 as the core decision-making stage in which the details of the stimulus are recognized and mapped onto a response. The last stage then represents response execution. The differences in the duration of this stage were also statistically significant, albeit in the opposite direction: being significantly shorter when the decision was more difficult. As response mapping for Easy and Difficult conditions was identical, we assume that participants started preparing the response execution while making decision in Difficult conditions (Stage 4), leading to a shorter response execution stage. Taken together, this fits the interpretation of the last three stages (Stages 3, 4, and 5) as decision stages of the Wald model, where we should note that decision-making includes motor execution.

Although our functional interpretation of the underlying cognitive processes was based on the ERP literature, there is a striking similarity between the discovered stages and stages reported in previous HsMM-MVPA papers, providing a form of cross-validation. For example, the first two stages were similar to the first two stages in Anderson et al. (2016) and Zhang, Walsh, et al. (2017, 2018). In these studies, the cognitive tasks required more complex decision-making, suggesting that these two initial stages represent relatively low-level stimulus processing, in agreement with the current interpretation as nondecision stages. In addition, in Zhang, Borst, et al. (2017), the interpretation of the stages was consistent with a computational cognitive model developed in the ACT-R cognitive architecture (Anderson, 2007). Similar to perceptual stages, the decision stages (Stages 3–5) were comparable in brain topologies and temporal onsets to results of earlier HsMM-MVPA studies. For example, Stage 3 in our study—interpreted as a familiarity stage—resembles the familiarity stage discovered in an associative recognition task (Borst & Anderson, 2015). The last two stages labeled “decision” and “response” are similar to the corresponding stages from Anderson et al. (2016). Altogether, this suggests robustness of the findings and cognitive stage interpretation in the current study.

DISCUSSION

The goal of the current paper was to establish evidence for HsMM-MVPA (Anderson et al., 2016) as a method capable of parsing a cognitive task into meaningful stages. This question was addressed by designing four conditions that varied in perceptual processing and decision difficulty and complementing the HsMM-MVPA with EAMs. EAMs provide insights into cognitive processing by decomposing RTs into decision and nondecision processes. In turn, the HsMM-MVPA method allows us to zoom into these processes by finding a range of underlying processing stages. The simplicity of the designed tasks, the experimental manipulations, and the information obtained employing EAMs allowed us to test whether the detection of the stages and their cognitive interpretation are valid.

First, we demonstrated that the manipulations in stimulus type and decision difficulty led to significant differences in behavior, with the largest differences in RTs when a decision was more difficult. In addition, in the Easy conditions, participants were slower to respond to Characters than to Shapes. Second, to quantify the duration of decision and nondecision components, RTs were decomposed with EAMs. We found two main effects: Nondecision time was longer for Characters than for Shapes, and drift rate was higher for Easy decisions than for Difficult decisions. This matches previous studies that demonstrated that lower drift rate values express difficulty in the decision (e.g., Turner, van Maanen, & Forstmann, 2015; Donkin & van Maanen, 2014; Basten et al., 2010). Specifically, some studies that manipulate decision difficulty by increasing the set of response alternatives found that this led to lower drift rates (Anders, Riès, van Maanen, & Alario, 2015; van Maanen et al., 2012). This manipulation seems comparable to our experimental design, where the number of stimulus–response mappings varied between easy and difficult choices.

To discover the processing stages underlying these components, the HsMM-MVPA was applied. For all conditions, a model with five stages accounted best for the data. We found two main effects: In Easy conditions, the topology of Bump 2 and the duration of the consecutive stage varied by stimulus type, and the topology and duration of the last two stages varied by decision difficulty. The largest difference in decision difficulty was represented in Stage 4, which was significantly longer when decisions were more difficult.

The experimental manipulations in stimulus type and decision difficulty were reflected in both HsMM-MVPA and EAMs. Whereas EAMs allowed us to quantitatively assess the effects of these manipulations by decomposing RTs, HsMM-MVPA uncovered the cognitive stages that cause these differences. To explore the relationship between the results of the HsMM-MVPA method and the EAMs in more detail, we compared their respective decision and nondecision durations. There was a clear consistency between the methods: We found an almost perfect correspondence between the duration of Stages 3–5 of the HsMM-MVPA method and the decision time of the EAMs. Not only was this effect present on the means, but both methods captured the variability in decision durations for each condition and participant in a similar manner. On the basis of the literature, these HsMM-MVPA stages were interpreted as familiarity, decision, and response. Note that this means that motor execution is part of the decision process estimated by the EAMs. Although motor processes are typically thought to be part of the nondecision time, there is increasing evidence that motor (preparation) is part of the decision process itself (e.g., Spieser, Servant, Hasbroucq, & Burle, 2017; Thura & Cisek, 2014, 2016).

Consequently, the nondecision component in EAMs was represented in the first two HsMM-MVPA stages. However, here the connection was less strong: The EAMs reported a much higher variance of these stages than the HsMM-MVPA method, and although the EAMs showed a difference in nondecision time between stimulus types, this difference was not present in the HsMM-MVPA stages. If one assumes that the conclusions of the HsMM-MVPA approach are correct, one would conclude that EAMs overestimate the variability in nondecision time. One may suspect that this is the case because the RTs that are being modeled not only reflect underlying cognitive processes but also are influenced by alternative processes such as motor planning (Walsh et al., 2017; Haith, Pakpoor, & Krakauer, 2016). In addition, recent studies demonstrated that RT measures are biased and context dependent (Wong, Goldsmith, Forrence, Haith, & Krakauer, 2017; Wong, Goldsmith, & Krakauer, 2016). Because the EAMs that we used here are fully dependent on the RT measures, this variation is spread to all stages. However, the HsMM-MVPA method uses the ongoing EEG and can attribute this variation to the actual stages that are influenced. In terms of cognitive interpretation, this might mean that the early nondecision processes such as attention and stimulus encoding are biologically defined mechanisms that are very stable across different tasks and participants—early ERP components are also very stable typically—whereas the decision stages are more context dependent.

Alternatively, it might also be that the HsMM-MVPA underestimates the variation in nondecision time duration and attributes all variability to the stages that are linked strongest to overall RT (cf. Walsh et al., 2017). However, given the very small standard errors on the duration of the early perceptual stages, we think this explanation is less likely. Another possible explanation could be that HsMM-MVPA does not capture low-level visual processing because of limitations in its spatial resolution and the fact that EEG only measures the top levels of the cortex. However, we believe this to be unlikely for two reasons: First, a similar analysis on MEG data—which provides a higher spatial resolution—resulted in equally stable perceptual stages with low variability (Anderson et al., 2018). Second, recent studies reported a high correlation between EEG and functional magnetic resonance imaging activations in primary visual cortex (V1) concluding that EEG can detect changes in V1 (Im, Gururajan, Zhang, Chen, & He, 2007; Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002).

Thus far, we have presented functional interpretation of the HsMM-MVPA stages based on the existing ERP literature. Although there is a striking similarity in the discovered stages across different HsMM-MVPA studies, it is a continuing challenge to establish a robust way for interpreting these findings. One of the suggested solutions is to develop a process model that can guide the functional interpretation of the stages (Anderson et al., 2016). Moreover, in the past, intracranial EEG and MEG were used to obtain better spatial resolution (Anderson et al., 2018; Zhang, van Vugt, et al., 2018). This allowed for observing distinct brain regions that were activated during a particular cognitive stage and, consequently, a more precise interpretation. Another approach was to associate the discovered processing stages with connectivity networks (Portoles et al., 2018), providing additional information on the processes in the stage.

Conclusion

With this study, we have demonstrated the value of the HsMM-MVPA as a method to detect detailed cognitive stages in a task. On a more general level, the combination of the HsMM-MVPA models and EAMs allowed us to robustly answer the question “What are the stages people go through when they perform simple discrimination tasks?” The EAMs helped us to quantify the decision and nondecision components underlying the task performance, whereas the HsMM-MVPA method provided us with a detailed division of these components into processing stages. Numerous theories have modeled the performance on different tasks starting from perceptual processing to response execution with a top–down strategy. We suggest that HsMM-MVPA can inform the development of such process models as it derives the stages directly from EEG data in the opposite, bottom–up manner.

Furthermore, we would like to suggest that the analysis of simple tasks can be potentially used to interpret more complex tasks that require the involvement of higher cognitive processes. At present, a range of complex tasks was analyzed with the HsMM-MVPA method applied to EEG. These tasks included oddball paradigm tasks (Walsh et al., 2017), associative recognition (Anderson et al., 2016), visual working memory task (Zhang, van Vugt, et al., 2018), and others. Although in each of these tasks certain cognitive stages were detected, it remains difficult to integrate these findings and to create basic profiles of cognitive stages. Although the stages we discovered might be specific to our tasks, given how similar these stage definitions are to previously published stages (e.g., Anderson et al., 2016), it seems likely that they can be used as the starting point for creating a library of neural signatures associated with certain cognitive processes.

Author Contributions

Hermine S. Berberyan: Conceptualization; Formal analysis; Investigation; Methodology; Visualization; Writing—original draft; Writing—review & editing. Leendert van Maanen: Methodology; Writing—review & editing. Hedderik van Rijn: Conceptualization; Funding acquisition; Methodology; Supervision; Writing—review & editing. Jelmer Borst: Conceptualization; Funding acquisition; Investigation; Methodology; Project administration; Supervision; Writing—review & editing.

Funding Information

Nederlandse Organisatie voor Wetenschappelijk Onderzoek (http://dx.doi.org/10.13039/501100003246), grant numbers: 451-15-040, 453-16-005.

Acknowledgments

This research was supported by Netherlands Organisation for Scientific Research (NWO) Veni Grant 451-15-040 awarded to Jelmer Borst. Van Rijn was supported by the research programme “Interval Timing in the Real World: A Functional, Computational and Neuroscience Approach,” with Project Number 453-16-005, financed by the Netherlands Organisation for Scientific Research (NWO). The authors thank Daan Sijbring for helping in data collection for Experiment 1, Andrew Heathcote and Steven Miletic for their help and feedback on Wald modeling during model-based BrainHack in Amsterdam, and Jacolien van Rij for her feedback on linear mixed-effects analysis.

Reprint requests should be sent to Hermine S. Berberyan, Bernoulli Institute, University of Groningen, Nijenborgh 9, Groningen, The Netherlands 9700 AK, or via e-mail: h.berberyan@rug.nl.

Note

REFERENCES

Anders
,
R.
,
Alario
,
F.-X.
, &
van Maanen
,
L.
(
2016
).
The shifted Wald distribution for response time data analysis
.
Psychological Methods
,
21
,
309
327
.
Anders
,
R.
,
Riès
,
S.
,
van Maanen
,
L.
, &
Alario
,
F. X.
(
2015
).
Evidence accumulation as a model for lexical selection
.
Cognitive Psychology
,
82
,
57
73
.
Anderson
,
J. R.
(
2007
).
How can the human mind occur in the physical universe?
New York
:
Oxford University Press
.
Anderson
,
J. R.
,
Borst
,
J. P.
,
Fincham
,
J. M.
,
Ghuman
,
A. S.
,
Tenison
,
C.
, &
Zhang
,
Q.
(
2018
).
The common time course of memory processes revealed
.
Psychological Science
,
29
,
1463
1474
.
Anderson
,
J. R.
, &
Fincham
,
J. M.
(
2014
).
Extending problem-solving procedures through reflection
.
Cognitive Psychology
,
74
,
1
34
.
Anderson
,
J. R.
,
Zhang
,
Q.
,
Borst
,
J. P.
, &
Walsh
,
M. M.
(
2016
).
The discovery of processing stages: Extension of Sternberg's method
.
Psychological Review
,
123
,
481
509
.
Barr
,
D. J.
,
Levy
,
R.
,
Scheepers
,
C.
, &
Tily
,
H. J.
(
2013
).
Random effects structure for confirmatory hypothesis testing: Keep it maximal
.
Journal of Memory and Language
,
68
.
Basten
,
U.
,
Biele
,
G.
,
Heekeren
,
H. R.
, &
Fiebach
,
C. J.
(
2010
).
How the brain integrates costs and benefits during decision making
.
Proceedings of the National Academy of Sciences, U.S.A.
,
107
,
21767
21772
.
Bates
,
D. M.
, &
DebRoy
,
S.
(
2004
).
Linear mixed models and penalized least squares
.
Journal of Multivariate Analysis
,
91
,
1
17
.
Bates
,
D. M.
,
Kliegl
,
R.
,
Vasishth
,
S.
, &
Baayen
,
H.
(
2015
).
Parsimonious mixed models
.
ArXiv preprint arXiv:1506.04967
.
Bell
,
A. J.
, &
Sejnowski
,
T. J.
(
1995
).
An information–maximization approach to blind separation and blind deconvolution
.
Neural Computation
,
7
,
1129
1159
.
Bode
,
S.
,
Bennett
,
D.
,
Sewell
,
D. K.
,
Paton
,
B.
,
Egan
,
G. F.
,
Smith
,
P. L.
, et al
(
2018
).
Dissociating neural variability related to stimulus quality and response times in perceptual decision-making
.
Neuropsychologia
,
111
,
190
200
.
Boehm
,
U.
,
van Maanen
,
L.
,
Forstmann
,
B.
, &
van Rijn
,
H.
(
2014
).
Trial-by-trial fluctuations in CNV amplitude reflect anticipatory adjustment of response caution
.
Neuroimage
,
96
,
95
105
.
Bogacz
,
R.
,
Wagenmakers
,
E.-J.
,
Forstmann
,
B. U.
, &
Nieuwenhuis
,
S.
(
2010
).
The neural basis of the speed–accuracy tradeoff
.
Trends in Neurosciences
,
33
,
10
16
.
Borst
,
J. P.
, &
Anderson
,
J. R.
(
2015
).
The discovery of processing stages: Analyzing EEG data with hidden semi-Markov models
.
Neuroimage
,
108
,
60
73
.
Cohen
,
X. M.
(
2014
).
Analyzing neural time series data: Theory and practice
.
Cambridge, MA
:
MIT Press
.
Curran
,
T.
(
2000
).
Brain potentials of recollection and familiarity
.
Memory & Cognition
,
28
,
923
938
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
.
Journal of Neuroscience Methods
,
134
,
9
21
.
Di Russo
,
F.
,
Martínez
,
A.
,
Sereno
,
M. I.
,
Pitzalis
,
S.
, &
Hillyard
,
S. A.
(
2002
).
Cortical sources of the early components of the visual evoked potential
.
Human Brain Mapping
,
15
,
95
111
.
Dobbins
,
I. G.
, &
Han
,
S.
(
2006
).
Isolating rule- versus evidence-based prefrontal activity during episodic and lexical discrimination: A functional magnetic resonance imaging investigation of detection theory distinctions
.
Cerebral Cortex
,
16
,
1614
1622
.
Donders
,
F. C.
(
1969
).
On the speed of mental processes
.
Acta Psychologica
,
30
,
412
431
.
Donkin
,
C.
, &
van Maanen
,
L.
(
2014
).
Piéron's Law is not just an artifact of the response mechanism
.
Journal of Mathematical Psychology
,
62–63
,
22
32
.
Evans
,
N. J.
, &
Wagenmakers
,
E.-J.
(
2020
).
Evidence accumulation models: Current limitations and future directions
.
Quantitative Methods for Psychology
,
16
,
73
90
.
Gelman
,
A.
, &
Rubin
,
D. B.
(
1992
).
Inference from iterative simulation using multiple sequences
.
Statistical Science
,
7
,
457
472
.
Goldstein
,
E. B.
,
Humphreys
,
G. W.
,
Shiffrar
,
M.
, &
Yost
,
W. A.
(
2008
).
Blackwell handbook of sensation and perception
.
Oxford
:
Blackwell
.
Gomez
,
P.
,
Ratcliff
,
R.
, &
Perea
,
M.
(
2007
).
A model of the go/no-go task
.
Journal of Experimental Psychology: General
,
136
,
389
413
.
Haith
,
A. M.
,
Pakpoor
,
J.
, &
Krakauer
,
J. W.
(
2016
).
Independence of movement preparation and movement initiation
.
Journal of Neuroscience
,
36
,
3007
3015
.
Hayama
,
H. R.
,
Johnson
,
J. D.
, &
Rugg
,
M. D.
(
2008
).
The relationship between the right frontal old/new ERP effect and post-retrieval monitoring: Specific or non-specific?
Neuropsychologia
,
46
,
1211
1223
.
Heathcote
,
A.
(
2004
).
Fitting Wald and ex-Wald distributions to esponse time data: An example using functions for the S-PLUS package
.
Behavior Research Methods, Instruments, & Computers
,
36
,
678
694
.
Heathcote
,
A.
,
Lin
,
Y.-S.
,
Reynolds
,
A.
,
Strickland
,
L.
,
Gretton
,
M.
, &
Matzke
,
D.
(
2019
).
Dynamic models of choice
.
Behavior Research Methods
,
51
,
961
985
.
Henson
,
R. N.
(
2011
).
How to discover modules in mind and brain: The curse of nonlinearity, and blessing of neuroimaging. A comment on Sternberg (2011)
.
Cognitive Neuropsychology
,
28
,
209
223
.
Hoeting
,
J. A.
,
Madigan
,
D.
,
Raftery
,
A. E.
, &
Volinsky
,
C. T.
(
1999
).
Bayesian model averaging: A tutorial
.
Statistical Science
,
14
,
382
417
.
Hurvich
,
L. M.
, &
Jameson
,
D.
(
1957
).
An opponent-process theory of color vision
.
Psychological Review
,
64
,
384
404
.
Im
,
C.-H.
,
Gururajan
,
A.
,
Zhang
,
N.
,
Chen
,
W.
, &
He
,
B.
(
2007
).
Spatial resolution of EEG cortical source imaging revealed by localization of retinotopic organization in human primary visual cortex
.
Journal of Neuroscience Methods
,
161
,
142
154
.
Imani
,
E.
,
Harati
,
A.
,
Pourreza
,
H.
, &
Goudarzi
,
M. M.
(
2020
).
Brain–behaviour relationships in the perceptual decision-making process through cognitive processing stages
.
bioRxiv
.
Jepma
,
M.
,
Wagenmakers
,
E.-J.
, &
Nieuwenhuis
,
S.
(
2012
).
Temporal expectation and information processing: A model-based analysis
.
Cognition
,
122
,
426
441
.
Kass
,
R. E.
, &
Raftery
,
A. E.
(
1995
).
Bayes factors
.
Journal of the American Statistical Association
,
90
,
773
795
.
Klimesch
,
W.
,
Sauseng
,
P.
, &
Hanslmayr
,
S.
(
2007
).
EEG alpha oscillations: The inhibition-timing hypothesis
.
Brain Research Reviews
,
53
,
63
88
.
Kuznetsova
,
A.
,
Brockhoff
,
P. B.
, &
Christensen
,
R. H. B.
(
2018
).
lmerTest Package: Tests in linear mixed effects models
.
Journal of Statistical Software
,
82
,
1
26
.
Lee
,
C.-Y.
,
Liu
,
Y.-N.
, &
Tsai
,
J.-L.
(
2012
).
The time course of contextual effects on visual word recognition
.
Frontiers in Psychology
,
3
,
285
.
Luck
,
S. J.
(
2005
).
An introduction to the event-related potential technique
.
Cambridge, MA
:
MIT Press
.
Luck
,
S. J.
,
Woodman
,
G. F.
, &
Vogel
,
E. K.
(
2000
).
Event-related potential studies of attention
.
Trends in Cognitive Sciences
,
4
,
432
440
.
Makeig
,
S.
,
Westerfield
,
M.
,
Jung
,
T. P.
,
Enghoff
,
S.
,
Townsend
,
J.
,
Courchesne
,
E.
, et al
(
2002
).
Dynamic brain sources of visual evoked responses
.
Science
,
295
,
690
694
.
Mangels
,
J. A.
,
Picton
,
T. W.
, &
Craik
,
F. I. M.
(
2001
).
Attention and successful episodic encoding: An event-related potential study
.
Cognitive Brain Research
,
11
,
77
95
.
Matzke
,
D.
, &
Wagenmakers
,
E.-J.
(
2009
).
Psychological interpretation of the ex-Gaussian and shifted Wald parameters: A diffusion model analysis
.
Psychonomic Bulletin & Review
,
16
,
798
817
.
Miletić
,
S.
, &
van Maanen
,
L.
(
2019
).
Caution in decision-making under time pressure is mediated by timing ability
.
Cognitive Psychology
,
110
,
16
29
.
Milosavljevic
,
M.
,
Malmaud
,
J.
,
Huth
,
A.
,
Koch
,
C.
, &
Rangel
,
A.
(
2010
).
The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure
.
Judgment and Decision Making
,
5
,
437
449
.
Miltner
,
W.
,
Johnson
,
R.
, Jr.
,
Braun
,
C.
, &
Larbig
,
W.
(
1989
).
Somatosensory event-related potentials to painful and non-painful stimuli: Effects of attention
.
Pain
,
38
,
303
312
.
Morey
,
R. D.
(
2008
).
Confidence intervals from normalized data: A correction to Cousineau (2005)
.
Quantitative Methods for Psychology
,
4
,
61
64
.
Morey
,
R. D.
,
Rouder
,
J. N.
,
Jamil
,
T.
,
Urbanek
,
S.
,
Forner
,
K.
, &
Ly
,
A.
(
2015
).
BayesFactor: Computation of Bayes factors for common designs (version 0.9.12-2)
. .
Mulder
,
M. J.
,
van Maanen
,
L.
, &
Forstmann
,
B. U.
(
2014
).
Perceptual decision neurosciences—A model-based review
.
Neuroscience
,
277
,
872
884
.
Paller
,
K. A.
,
Voss
,
J. L.
, &
Boehm
,
S. G.
(
2007
).
Validating neural correlates of familiarity
.
Trends in Cognitive Sciences
,
11
,
243
250
.
Portoles
,
O.
,
Borst
,
J. P.
, &
van Vugt
,
M. K.
(
2018
).
Characterizing synchrony patterns across cognitive task stages of associative recognition memory
.
European Journal of Neuroscience
,
48
,
2759
2769
.
Posner
,
M. I.
(
2005
).
Timing the brain: Mental chronometry as a tool in neuroscience
.
PLoS Biology
,
3
,
e51
.
Ratcliff
,
R.
(
1978
).
A theory of memory retrieval
.
Psychological Review
,
85
,
59
108
.
Ratcliff
,
R.
, &
McKoon
,
G.
(
2008
).
The diffusion decision model: Theory and data for two-choice decision tasks
.
Neural Computation
,
20
,
873
922
.
Rektor
,
I.
,
Bareš
,
M.
,
Kaňovský
,
P.
,
Brázdil
,
M.
,
Klajblová
,
I.
,
Streitová
,
H.
, et al
(
2004
).
Cognitive potentials in the basal ganglia–frontocortical circuits. An intracerebral recording study
.
Experimental Brain Research
,
158
,
289
301
.
Rektor
,
I.
,
Kaňovský
,
P.
,
Bareš
,
M.
,
Brázdil
,
M.
,
Streitová
,
H.
,
Klajblová
,
H.
, et al
(
2003
).
A SEEG study of ERP in motor and premotor cortices and in the basal ganglia
.
Clinical Neurophysiology
,
114
,
463
471
.
Rugg
,
M. D.
,
Mark
,
R. E.
,
Walla
,
P.
,
Schloerscheidt
,
A. M.
,
Birch
,
C. S.
, &
Allan
,
K.
(
1998
).
Dissociation of the neural correlates of implicit and explicit memory
.
Nature
,
392
,
595
598
.
Rugg
,
M. D.
,
Milner
,
A. D.
,
Lines
,
C. R.
, &
Phalp
,
R.
(
1987
).
Modulation of visual event-related potentials by spatial and non-spatial visual selective attention
.
Neuropsychologia
,
25
,
85
96
.
Schroeder
,
C. E.
,
Steinschneider
,
M.
,
Javitt
,
D. C.
,
Tenke
,
C. E.
,
Givre
,
S. J.
,
Mehta
,
A. D.
, et al
(
1995
).
Localization of ERP generators and identification of underlying neural processes
.
Electroencephalography and Clinical Neurophysiology
,
44(Suppl.)
,
55
75
.
PMID:7649056
Shah
,
A. S.
,
Bressler
,
S. L.
,
Knuth
,
K. H.
,
Ding
,
M.
,
Mehta
,
A. D.
,
Ulbert
,
I.
, et al
(
2004
).
Neural dynamics and the fundamental mechanisms of event-related brain potentials
.
Cerebral Cortex
,
14
,
476
483
.
Sherri
,
M.
,
Boulkaibet
,
I.
,
Marwala
,
T.
, &
Friswell
,
M. I.
(
2019
).
A differential evolution Markov chain Monte Carlo algorithm for Bayesian model updating
. In
N.
Dervilis
(Ed.),
Special topics in structural dynamics
(
Vol. 5
).
Cham, Switzerland
:
Springer
.
Smith
,
P. L.
, &
Ratcliff
,
R.
(
2004
).
Psychology and neurobiology of simple decisions
.
Trends in Neurosciences
,
27
,
161
168
.
Spieser
,
L.
,
Servant
,
M.
,
Hasbroucq
,
T.
, &
Burle
,
B.
(
2017
).
Beyond decision! Motor contribution to speed–accuracy trade-off in decision-making
.
Psychonomic Bulletin & Review
,
24
,
950
956
.
Sternberg
,
S.
(
1969
).
The discovery of processing stages: Extensions of Donders' method
.
Acta Psychologica
,
30
,
276
315
.
Stewart
,
T. C.
,
Bekolay
,
T.
, &
Eliasmith
,
C.
(
2012
).
Learning to select actions with spiking neurons in the basal ganglia
.
Frontiers in Neuroscience
,
6
,
2
.
Stocco
,
A.
,
Murray
,
N. L.
,
Yamasaki
,
B. L.
,
Renno
,
T. J.
,
Nguyen
,
J.
, &
Prat
,
C. S.
(
2017
).
Individual differences in the Simon effect are underpinned by differences in the competitive dynamics in the basal ganglia: An experimental verification and a computational model
.
Cognition
,
164
,
31
45
.
Thura
,
D.
, &
Cisek
,
P.
(
2014
).
Deliberation and commitment in the premotor and primary motor cortex during dynamic decision making
.
Neuron
,
81
,
1401
1416
.
Thura
,
D.
, &
Cisek
,
P.
(
2016
).
Modulation of premotor and primary motor cortical activity during volitional adjustments of speed–accuracy trade-offs
.
Journal of Neuroscience
,
36
,
938
956
.
Turner
,
B. M.
,
Sederberg
,
P. B.
,
Brown
,
S. D.
, &
Steyvers
,
M.
(
2013
).
A method for efficiently sampling from distributions with correlated dimensions
.
Psychological Methods
,
18
,
368
384
.
Turner
,
B. M.
,
van Maanen
,
L.
, &
Forstmann
,
B. U.
(
2015
).
Informing cognitive abstractions through neuroimaging: The neural drift diffusion model
.
Psychological Review
,
122
,
312
336
.
van Maanen
,
L.
,
Grasman
,
R. P. P. P.
,
Forstmann
,
B. U.
,
Keuken
,
M. C.
,
Brown
,
S. D.
, &
Wagenmakers
,
E.-J.
(
2012
).
Similarity and number of alternatives in the random-dot motion paradigm
.
Attention, Perception, & Psychophysics
,
74
,
739
753
.
van Maanen
,
L.
,
van der Mijn
,
R.
,
van Beurden
,
M. H. P. H.
,
Roijendijk
,
L. M. M.
,
Kingma
,
B. R. M.
,
Miletić
,
S.
, et al
(
2019
).
Core body temperature speeds up temporal processing and choice behavior under deadlines
.
Scientific Reports
,
9
,
10053
.
Vehtari
,
A.
,
Gelman
,
A.
, &
Gabry
,
J.
(
2017
).
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
.
Statistics and Computing
,
27
,
1413
1432
.
Wagenmakers
,
E.-J.
, &
Farrell
,
S.
(
2004
).
AIC model selection using Akaike weights
.
Psychonomic Bulletin & Review
,
11
,
192
196
.
Walsh
,
M. M.
,
Gunzelmann
,
G.
, &
Anderson
,
J. R.
(
2017
).
Relationship of P3b single-trial latencies and response times in one, two, and three-stimulus oddball tasks
.
Biological Psychology
,
123
,
47
61
.
Wong
,
A. L.
,
Goldsmith
,
J.
,
Forrence
,
A. D.
,
Haith
,
A. M.
, &
Krakauer
,
J. W.
(
2017
).
Reaction times can reflect habits rather than computations
.
eLife
,
6
,
e28075
.
Wong
,
A. L.
,
Goldsmith
,
J.
, &
Krakauer
,
J. W.
(
2016
).
A motor planning stage represents the shape of upcoming movement trajectories
.
Journal of Neurophysiology
,
116
,
296
305
.
Wu
,
C. F. J.
(
1983
).
On the convergence properties of the EM algorithm
.
Annals of Statistics
,
11
,
95
103
.
Yeung
,
N.
,
Bogacz
,
R.
,
Holroyd
,
C. B.
, &
Cohen
,
J. D.
(
2004
).
Detection of synchronized oscillations in the electroencephalogram: An evaluation of methods
.
Psychophysiology
,
41
,
822
832
.
Yu
,
S.-Z.
(
2010
).
Hidden semi-Markov models
.
Artificial Intelligence
,
174
,
215
243
.
Zhang
,
Q.
,
Borst
,
J. P.
,
Kass
,
R. E.
, &
Anderson
,
J. R.
(
2017
).
Inter-subject alignment of MEG datasets in a common representational space
.
Human Brain Mapping
,
38
,
4287
4301
.
Zhang
,
Q.
,
van Vugt
,
M.
,
Borst
,
J. P.
, &
Anderson
,
J. R.
(
2018
).
Mapping working memory retrieval in space and in time: A combined electroencephalography and electrocorticography approach
.
Neuroimage
,
174
,
472
484
.
Zhang
,
Q.
,
Walsh
,
M. M.
, &
Anderson
,
J. R.
(
2017
).
The effects of probe similarity on retrieval and comparison processes in associative recognition
.
Journal of Cognitive Neuroscience
,
29
,
352
367
.
Zhang
,
Q.
,
Walsh
,
M. M.
, &
Anderson
,
J. R.
(
2018
).
The impact of inserting an additional mental process
.
Computational Brain & Behavior
,
1
,
22
35
.