Elementary arithmetic requires a complex interplay between several brain regions. The classical view, arising from fMRI, is that the intraparietal sulcus (IPS) and the superior parietal lobe (SPL) are the main hubs for arithmetic calculations. However, recent studies using intracranial electroencephalography have discovered a specific site, within the posterior inferior temporal cortex (pITG), that activates during visual perception of numerals, with widespread adjacent responses when numerals are used in calculation. Here, we reexamined the contribution of the IPS, SPL, and pITG to arithmetic by recording intracranial electroencephalography signals while participants solved addition problems. Behavioral results showed a classical problem size effect: RTs increased with the size of the operands. We then examined how high-frequency broadband (HFB) activity is modulated by problem size. As expected from previous fMRI findings, we showed that the total HFB activity in IPS and SPL sites increased with problem size. More surprisingly, pITG sites showed an initial burst of HFB activity that decreased as the operands got larger, yet with a constant integral over the whole trial, thus making these signals invisible to slow fMRI. Although parietal sites appear to have a more sustained function in arithmetic computations, the pITG may have a role of early identification of the problem difficulty, beyond merely digit recognition. Our results ask for a reevaluation of the current models of numerical cognition and reveal that the ventral temporal cortex contains regions specifically engaged in mathematical processing.