Functional connectivity (FC) reflects brain-wide communication essential for cognition, yet the role of underlying biophysical factors in shaping FC remains unclear. We quantify the influence of physical factors—structural connectivity (SC) and Euclidean distance (DC), which capture anatomical wiring and regional distance—and molecular factors—gene expression similarity (GC), and neuroreceptor congruence (RC), representing neurobiological similarity—on resting-state FC. We assess how these factors impact graph-theoretic and gradient features, capturing pairwise and higher-order interactions. By generating remnant functional networks after selectively removing connections tied to specific factors, we show that molecular factors, particularly RC, dominate graph-theoretic features, while gradient features are shaped by a mix of molecular and physical factors, especially GC and DC. SC has a surprisingly minor role. We also link FC alterations to biophysical factors in schizophrenia, bipolar disorder, and attention deficit/hyperactivity disorder (ADHD), with physical factors differentiating these groups. These insights are key for understanding FC across various applications, including task performance, development, and clinical conditions. Author Summary We introduce and utilize a remnant functional networks-based framework to investigate how the brain’s functional connectivity (FC), representative of the brain-wide communication network essential for cognition, is shaped by various biophysical —physical and molecular—factors, including structural connectivity, physical distance, and neuroreceptor similarities. Our findings reveal that molecular factors, particularly neuroreceptor congruence, play a dominant role in shaping FC network features, while structural connectivity has a surprisingly minor influence. Applying this framework, we identify distinct biophysical drivers of FC alterations in schizophrenia, bipolar disorder, and ADHD, offering insights into the unique characteristics of these disorders. This study emphasizes the significance of molecular factors in shaping FC and provides a tool for exploring these associations across various contexts, including task performance, development, and clinical conditions.