Artificial grammar learning (Reber, 1967) is a form of implicit learning in which cognitive, rather than motor, implicit learning has been found. After viewing a series of letter strings formed according to a finite state rule system, people are able to classify new letter strings as to whether or not they are formed according to these grammatical rules despite little conscious insight into the rule structure. Previous research has shown that these classification judgments are based on knowledge of abstract rules as well as superficial similarity (“chunk strength”) to training strings. Here we used event-related fMRI to identify neural regions involved in using both sources of information as test stimuli were designed to unconfound chunk strength from rule use. Using functional connectivity analyses, the extent to which the sources of information are complementary or competitive was also assessed. Activation in the right caudate was associated with rule adherence, whereas medial temporal lobe activations were associated with chunk strength. Additionally, functional connectivity analyses revealed caudate and medial temporal lobe activations to be strongly negatively correlated (r = −88) with one another during the performance of this task.