In our daily activities, we encounter numerous objects that we successfully distinguish and recognize within a fraction of a second. This holds for coarse distinctions (e.g., cat vs. hammer) but also for more challenging distinctions that require fine-grain analysis (e.g., cat vs. dog). The efficiency of this recognition depends on how the brain organizes object-related information. While several attempts have focused on unraveling large-scale organization principles, research on fine-grained knowledge organization is rather limited. Here, we explored the fine-grain organization of object knowledge and investigated whether manipulable objects are organized and represented in terms of their similarity. To accomplish this, different groups of individuals participated in a behavioral and functional magnetic resonance imaging (fMRI) release from adaptation experiment. Adaptation was induced by presenting different exemplars of a particular object, and release from adaptation was elicited by the presentation of a deviant object. The relationship between adaptation and deviant objects was manipulated into four levels of similarity, measured by feature overlap between these objects. Our findings revealed that increasing object similarity provoked slower reaction times and weaker fMRI release from adaptation. Specifically, we identified similarity-driven tuning curves for the release from adaptation in the medial fusiform, collateral sulcus, parahippocampal gyri, lingual gyri, lateral occipital complex, and occipito-parietal cortex. These results suggest that the processing and representation of objects in the brain and our ability to perform fine discriminations between objects reflect real-world object similarity in a relatively parametric manner.