The dopamine (DA) system plays an important role in cognition. Accordingly, normal variation in DA genes has been found to predict individual differences in cognitive performance. However, little is known of the impact of genetic differences on the link between empirical indicators of the DA system and cognition in humans. The present work used PET with 11C-raclopride to assess DA D2-receptor binding potential (BP) and links to episodic memory, working memory, and perceptual speed in 179 healthy adults aged 64–68 years. Previously, the T-allele of a DA D2-receptor single-nucleotide polymorphism, C957T, was associated with increased apparent affinity of 11C-raclopride, giving rise to higher BP values despite similar receptor density values between allelic groups. Consequently, we hypothesized that 11C-raclopride BP measures inflated by affinity rather than D2-receptor density in T-allele carriers would not be predictive of DA integrity and therefore prevent finding an association between 11C-raclopride BP and cognitive performance. In accordance with previous findings, we show that 11C-raclopride BP was increased in T-homozygotes. Importantly, 11C-raclopride BP was only associated with cognitive performance in groups with low or average ligand affinity (C-allele carriers of C957T, n = 124), but not in the high-affinity group (T-homozygotes, n = 55). The strongest 11C-raclopride BP–cognition associations and the highest level of performance were found in C-homozygotes. These findings show that genetic differences modulate the link between BP and cognition and thus have important implications for the interpretation of DA assessments with PET and 11C-raclopride in multiple disciplines ranging from cognitive neuroscience to psychiatry and neurology.