Ventral occipito-temporal cortex is known to play a major role in visual object recognition. Still unknown is whether object familiarity and semantic domain are critical factors in its functional organization. Most models assume a functional locus where exemplars of familiar categories are represented: the structural description system. On the assumption that familiarity should modulate the effect of visual noise on form recognition, we attempted to individualize the structural description system by scanning healthy subjects while they looked at familiar (living and nonliving things) and novel 3-D objects, either with increasing or decreasing visual noise. Familiarity modulated the visual noise effect (particularly when familiar items were living things), revealing a substrate for the structural description system in right occipito-temporal cortex. These regions also responded preferentially to living as compared to nonliving items. Overall, these results suggest that living items are particularly reliant on the structural description system.