Children with language-based learning problems often exhibit pronounced speech perception difficulties. Specifically, these children have increased difficulty separating brief sounds occurring in rapid succession (temporal resolution). The purpose of this study was to better understand the consequences of auditory temporal resolution deficits from the perspective of the neural encoding of speech. The findings provide evidence that sensory processes relevant to cognition take place at much earlier levels than traditionally believed. Thresholds from a psychophysical backward masking task were used to divide children into groups with good and poor temporal resolution. Speech-evoked brainstem responses were analyzed across groups to measure the neural integrity of stimulus-time mechanisms. Results suggest that children with poor temporal resolution do not have an overall neural processing deficit, but rather a deficit specific to the encoding of certain acoustic cues in speech. Speech understanding relies on the ability to attach meaning to rapidly fluctuating changes of both the temporal and spectral information found in consonants and vowels. For this to happen properly, the auditory system must first accurately encode these time-varying acoustic cues. Speech perception difficulties that often co-occur in children with poor temporal resolution may originate as a neural encoding deficit in structures as early as the auditory brainstem. Thus, speech-evoked brainstem responses are a biological marker for auditory temporal processing ability.