Leftward language production and rightward spatial attention are salient features of functional organization in most humans, but their anatomical basis remains unclear. Interhemispheric connections and intrahemispheric white matter asymmetries have been proposed as important factors underlying functional lateralization. To investigate the role of white matter connectivity in functional lateralization, we first identified 96 left-handers using visual half field naming tasks. They were then divided into atypical and typical functional dominance based on the lateralization of brain activation in a word generation task (for language production) and a landmark task (for spatial attention). Using a novel fixel-based framework, we obtained fiber-specific properties of white matter pathways. Results showed, first, that differences between two language dominance groups occurred in the asymmetry of the superior longitudinal fasciculus-III (SLF-III), whereas differences between two spatial attention dominance groups occurred in the rostrum and rostral body of the corpus callosum. However, the directions of functional lateralization were not associated with the directions of white matter asymmetries. Second, the degree of language lateralization was predicted by SLF-III asymmetry and the rostral body of the corpus callosum, whereas the degree of spatial attention lateralization was predicted by the rostrum of the corpus callosum. Notably, the degree of each functional lateralization was negatively correlated with the anterior and middle callosal connections, supporting the excitatory model of the corpus callosum. The results suggest that language lateralization is shaped by a combined effect of intra- and interhemispheric connections, whereas spatial attention lateralization relies more on interhemispheric connections.