The study of brain-damaged patients and advancements in neuroimaging have lead to the discovery of discrete brain regions that process visual image categories, such as objects and scenes. However, how these visual image categories interact remains unclear. For example, is scene perception simply an extension of object perception, or can global scene “gist” be processed independently of its component objects? Specifically, when recognizing a scene such as an “office,” does one need to first recognize its individual objects, such as the desk, chair, lamp, pens, and paper to build up the representation of an “office” scene? Here, we show that temporary interruption of object processing through repetitive TMS to the left lateral occipital cortex (LO), an area known to selectively process objects, impairs object categorization but surprisingly facilitates scene categorization. This result was replicated in a second experiment, which assessed the temporal dynamics of this disruption and facilitation. We further showed that repetitive TMS to left LO significantly disrupted object processing but facilitated scene processing when stimulation was administered during the first 180 msec of the task. This demonstrates that the visual system retains the ability to process scenes during disruption to object processing. Moreover, the facilitation of scene processing indicates disinhibition of areas involved in global scene processing, likely caused by disrupting inhibitory contributions from the LO. These findings indicate separate but interactive pathways for object and scene processing and further reveal a network of inhibitory connections between these visual brain regions.