In this study, we combined functional magnetic resonance imaging (fMRI) and dynamic causal modeling (DCM) to investigate whether object category effects in the occipital and temporal cortex are mediated by inputs from early visual cortex or parietal regions. Resolving this issue may provide anatomical constraints on theories of category specificity— which make different assumptions about the underlying neurophysiology. The data were acquired by Ishai, Ungerleider, Martin, Schouten, and Haxby (1999, 2000) and provided by the National fMRI Data Center (http://www.fmridc.org). The original authors used a conventional analysis to estimate differential effects in the occipital and temporal cortex in response to pictures of chairs, faces, and houses. We extended this approach by estimating neuronal interactions that mediate category effects using DCM. DCM uses a Bayesian framework to estimate and make inferences about the influence that one region exerts over another and how this is affected by experimental changes. DCM differs from previous approaches to brain connectivity, such as multivariate autoregressive models and structural equation modeling, as it assumes that the observed hemodynamic responses are driven by experimental changes rather than endogenous noise. DCM therefore brings the analysis of brain connectivity much closer to the analysis of regionally specific effects usually applied to functional imaging data. We used DCM to estimate the influence that V3 and the superior/inferior parietal cortex exerted over category-responsive regions and how this was affected by the presentation of houses, faces, and chairs. We found that category effects in occipital and temporal cortex were mediated by inputs from early visual cortex. In contrast, the connectivity from the superior/inferior parietal area to the category-responsive areas was unaffected by the presentation of chairs, faces, or houses. These findings indicate that category effects in the occipital and temporal cortex can be mediated by bottom–up mechanisms—a finding that needs to be embraced by models of category specificity.