This study is the first to assess lesion–symptom relations for subitizing and counting impairments in a large sample of neuropsychological patients (41 patients) using an observer-independent voxel-based approach. We tested for differential effects of enumerating small versus large numbers of items while controlling for hemianopia and visual attention deficits. Overall impairments in the enumeration of any numbers (small or large) were associated with an extended network, including bilateral occipital and fronto-parietal regions. Within this network, severe impairments in accuracy when enumerating small sets of items (in the subitizing range) were associated with damage to the left posterior occipital cortex, bilateral lateral occipital and right superior frontal cortices. Lesions to the right calcarine extending to the precuneus led to patients serially counting even small numbers of items (indicated by a steep response slope), again demonstrating an impaired subitizing ability. In contrast, impairments in counting large numerosities were associated with damage to the left intraparietal sulcus. The data support the argument for some distinctive processes and neural areas necessary to support subitization and counting with subitizing relying on processes of posterior occipital cortex and with counting associated with processing in the parietal cortex.