The format of high-level object representations in temporal-occipital cortex is a fundamental and as yet unresolved issue. Here we use fMRI to show that human lateral occipital cortex (LOC) encodes novel 3-D objects in a multisensory and part-based format. We show that visual and haptic exploration of objects leads to similar patterns of neural activity in human LOC and that the shared variance between visually and haptically induced patterns of BOLD contrast in LOC reflects the part structure of the objects. We also show that linear classifiers trained on neural data from LOC on a subset of the objects successfully predict a novel object based on its component part structure. These data demonstrate a multisensory code for object representations in LOC that specifies the part structure of objects.

A fundamental observation in the neurosciences is that the brain is a modular system in which different regions perform different tasks. Recent evidence, however, raises questions about the accuracy of this characterization with respect to neo-nates. One possible interpretation of this evidence is that certain aspects of the modular organization of the adult brain arise developmentally. To explore this hypothesis we wish to characterize the computational principles that underlie the development of modular systems. In previous work we have considered computational schemes that allow a learning system to discover the modular structure that is present in the environment (Jacobs, Jordan, & Barto, 1991). In the current paper we present a complementary approach in which the development of modularity is due to an architectural bias in the learner. In particular, we examine the computational consequences of a simple architectural bias toward short-range connections. We present simulations that show that systems that learn under the influence of such a bias have a number of desirable properties, including a tendency to decompose tasks into subtasks, to decouple the dynamics of recurrent subsystems, and to develop location-sensitive internal representations. Furthermore, the system's units develop local receptive and projective fields, and the system develops characteristics that are typically associated with topographic maps.