Durational patterns provide cues to linguistic structure, thus so variations in rhythm skills may have consequences for language development. Understanding individual differences in rhythm skills, therefore, could help explain variability in language abilities across the population. We investigated the neural foundations of rhythmic proficiency and its relation to language skills in young adults. We hypothesized that rhythmic abilities can be characterized by at least two constructs, which are tied to independent language abilities and neural profiles. Specifically, we hypothesized that rhythm skills that require integration of information across time rely upon the consistency of slow, low-frequency auditory processing, which we measured using the evoked cortical response. On the other hand, we hypothesized that rhythm skills that require fine temporal precision rely upon the consistency of fast, higher-frequency auditory processing, which we measured using the frequency-following response. Performance on rhythm tests aligned with two constructs: rhythm sequencing and synchronization. Rhythm sequencing and synchronization were linked to the consistency of slow cortical and fast frequency-following responses, respectively. Furthermore, whereas rhythm sequencing ability was linked to verbal memory and reading, synchronization ability was linked only to nonverbal auditory temporal processing. Thus, rhythm perception at different time scales reflects distinct abilities, which rely on distinct auditory neural resources. In young adults, slow rhythmic processing makes the more extensive contribution to language skills.