Prior research has reported that the medial temporal, parietal, and frontal brain regions are associated with visual statistical learning (VSL). However, the neural mechanisms involved in both memory enhancement and impairment induced by VSL remain unknown. In this study, we examined this issue using event-related fMRI. fMRI data from the familiarization scan showed a difference in the activation level of the superior frontal gyrus (SFG) between structured triplets, where three objects appeared in the same order, and pseudorandom triplets. More importantly, the precentral gyrus and paracentral lobule responded more strongly to Old Turkic letters inserted into the structured triplets than to those inserted into the random triplets, at the end of the familiarization scan. Furthermore, fMRI data from the recognition memory test scan, where participants were asked to decide whether the objects or letters shown were old (presented during familiarization scan) or new, indicated that the middle frontal gyrus and SFG responded more strongly to objects from the structured triplets than to those from the random triplets, which overlapped with the brain regions associated with VSL. In contrast, the response of the lingual gyrus, superior temporal gyrus, and cuneus was weaker to letters inserted into the structured triplets than to those inserted into the random triplets, which did not overlap with the brain regions associated with observing the letters during the familiarization scan. These findings suggest that different brain regions are involved in memory enhancement and impairment induced by VSL.