The contrast sensitivity function (CSF) characterizes visual function, and is widely used in research on visual perception and ophthalmological disorders. The CSF describes the lowest contrast level that participants can perceive as a function of spatial frequency. Here, we present a new method to estimate the neural equivalent of the CSF that describes how a population of neurons responds to contrast as a function of spatial frequency. Using functional magnetic resonance imaging (fMRI) at 7 Tesla, we measured neural responses while participants viewed gratings that varied systematically in contrast and spatial frequency. We modeled the neural CSF (nCSF) using an asymmetric parabolic function, and we modeled the transition from no response to full response using a contrast response function (CRF). We estimated the nCSF parameters for every cortical location by minimizing the residual variance between the model predictions and the fMRI data. We validated the method using simulations and parameter recovery. We show that our nCSF model explains a significant amount of the variance in the fMRI time series. Moreover, the properties of the nCSF vary according to known systematic differences across the visual cortex. Specifically, the peak spatial frequency that a cortical location responds to decreases with eccentricity and across the visual hierarchy. This new method will provide valuable insights into the properties of the visual cortex and how they are altered in both healthy and clinical conditions.