Object interaction requires knowledge of the weight of an object, as well as its shape. The lateral occipital complex (LOC), an area within the ventral visual pathway, is well known to be critically involved in processing visual shape information. Recently, however, LOC has also been implicated in coding object weight before grasping—a result that is surprising because weight is a nonvisual object property that is more relevant for motor interaction than visual perception. Here, we examined the causal role of LOC in perceiving heaviness and in determining appropriate fingertip forces during object lifting. We studied perceptions of heaviness and lifting behavior in a neuropsychological patient (M.C.) who has large bilateral occipitotemporal lesions that include LOC. We compared the patient's performance to a group of 18 neurologically healthy age-matched controls. Participants were asked to lift and report the perceived heaviness of a set of equally weighted spherical objects of various sizes—stimuli which typically induce the size–weight illusion, in which the smaller objects feel heavier than the larger objects despite having identical mass. Despite her ventral stream lesions, M.C. experienced a robust size–weight illusion induced by visual cues to object volume, and the magnitude of the illusion in M.C. was comparable to age-matched controls. Similarly, M.C. evinced predictive fingertip force scaling to visual size cues during her initial lifts of the objects that were well within the normal range. These single-case neuropsychological findings suggest that LOC is unlikely to play a causal role in computing object weight.
Successful physical interaction with everyday objects requires an adaptive understanding of their weight. Motor-related regions of frontal and parietal cortex, as well as the cerebellum, are known to be directly involved in controlling object lifting (Chouinard, Large, Chang, & Goodale, 2009; Jenmalm, Schmitz, Forssberg, & Ehrsson, 2006; Chouinard, Leonard, & Paus, 2005). Currently, however, little is known about the cognitive and neural mechanisms that guide a priori weight estimates before object lifting. Although a number of visual cues can be used to estimate an object's weight (Buckingham, Cant, & Goodale, 2009), physical size is one of the most salient and reliable visual indicators of mass (Buckingham & Goodale, 2013). In the context of the well-reported anatomical distinction and functional dissociation between the dorsal and ventral visual processing streams (Goodale & Milner, 2018), a natural prediction is that object volume may be computed initially within the ventral perceptual pathway and relayed to dorsal areas to facilitate the planning and online control of grasping actions.
In a recent test of this idea, Gallivan, Cant, Goodale, and Flanagan (2014) combined a slow event-related fMRI design with multivoxel pattern decoding techniques to identify brain regions that code for object weight during the planning and execution phases of manual object lifting. Participants repeatedly lifted identical-looking cylinders with different weights, and cortical responses to the objects were examined during the distinct plan and lift phases of each trial. As expected, voxel-based activation patterns predicted the weight of the to-be-lifted object in dorsal somatosensory and motor areas. However, Gallivan et al. (2014) also used an ROI approach to functionally identify the lateral occipital complex (LOC), an object-selective region of occipitotemporal cortex (Grill-Spector & Malach, 2004; Kourtzi & Kanwisher, 2001), which is composed of two subregions: area LO and the posterior fusiform gyrus (Weiner, Natu, & Grill-Spector, 2018). Gallivan et al. (2014) found that fMRI activation patterns within LO and posterior fusiform gyruss predicted the weight of the object before the lift itself, suggesting, surprisingly, that shape-selective areas within the ventral visual processing stream represent the nonvisual property of object weight. These findings raise the critical question of whether LOC plays a functional role in processing object weight or whether it receives feedback information about object size from other areas, such as dorsal somatosensory and motor regions, that are critical for computing object weight.
Case studies in neuropsychological patients can provide important insights into how brain structure relates to specific cognitive processes. With respect to shape processing, neuropsychological studies have demonstrated that the LOC plays a critical functional role in visual, but not haptic, shape perception. Whereas small unilateral infarcts to area LO lead to severe behavioral deficits in shape recognition and bilateral reductions in fMRI responses to visually presented objects (Konen, Behrmann, Nishimura, & Kastner, 2011), haptic shape perception in patients with bilateral LOC lesions remains strikingly preserved (Snow, Goodale, & Culham, 2015). By extension, if LOC plays a functional role in computing object weight, a clear prediction is that LOC damage should lead to severe deficits in anticipating object weight, based on visual size cues.
Here, we investigated whether LOC is essential for coding object weight by examining weight perception and fingertip force control in a neuropsychological patient (M.C.) who has extensive bilateral lesions of occipitotemporal cortex that completely encompass LOC (Figure 1). Critically, although M.C.'s visual function is severely compromised by her lesion, she can, in certain circumstances, detect moving stimuli (Riddoch, 1917), as well as spared visual sensitivity for static targets that fall within her peripheral visual field (Snow et al., 2015). We examined M.C.'s capacity for weight perception in the context of the size–weight illusion (SWI), where small objects are reported as feeling heavier than identically weighted large objects (Charpentier, 1891). Critically, this illusory difference in perceived weight can be induced by visual cues alone, when lifting the objects via a string and pulley system (Buckingham, Milne, Byrne, & Goodale, 2015; Masin & Crestoni, 1988) or with a handle (Buckingham & Goodale, 2010a; Flanagan & Beltzner, 2000).
In addition to these reliable perceptual effects, volume cues have also been shown to affect the way objects are lifted over successive trials. Because of the predictive application of fingertip forces, lifters initially use forces that reflect an object's relative size and, therefore, the expected weight. In the context of SWI-inducing objects, participants will typically use a significantly greater rate of force to grip and lift a large object than they would to grip and lift an identically weighted smaller object (Gordon, Forssberg, Johansson, & Westling, 1991; Davis & Roberts, 1976).
We predicted that if ventral brain regions such as LOC are critical for representing the expected weight of an object (Gallivan et al., 2014), then bilateral LOC lesions should lead to (1) an inability to experience the SWI and (2) a failure to use size cues to guide sensorimotor prediction during initial lifts of the weight illusion-inducing stimuli. Conversely, if LOC does not play a causal role in processing object weight, M.C. should readily experience the SWI (i.e., report that the small objects feel heavier than the large objects) and apply fingertip forces in a predictive way that reflects the apparent weight of the object on initial lifting trials (i.e., gripping and lifting larger objects with greater force than smaller objects).
M.C. is a right-handed woman who was 43 years old at the time of testing. At the age of 30, M.C. fell into a coma for 59 days following hypotension and respiratory infection. During the period of coma, M.C. suffered a stroke. Initial CT revealed bilateral occipital lobe infarctions. When M.C. emerged from coma, she reported having no vision, and static perimetry testing confirmed her to be totally blind. Over subsequent months poststroke, M.C. was found to have residual sensitivity to moving visual stimuli—a phenomenon known as Riddoch's phenomenon (Riddoch, 1917), and this sensitivity has continued to improve since her stroke. More recent ophthalmological assessments report that M.C. has coarse visual sensitivity to stimuli positioned within the periphery of the upper left visual quadrant and the lower right quadrant. Informal testing indicates that M.C.'s ability to visually discriminate high-spatial frequency information, such as visual textures, is minimal (although this has not been tested psychophysically), and therefore, we focused our examinations on M.C.'s ability to perceive weight cued by visual size. High-resolution structural MRI scans of M.C.'s brain around the time of testing (Figure 1) reveal extensive bilateral infarctions of the occipital and temporal lobes that extend dorsally into the right posterior parietal cortex. Although most of M.C.'s visual cortex was destroyed by the stroke, she has a small region of tissue remaining at the rostral end of the calcarine sulcus, corresponding to the peripheral visual field. Critically, however, M.C. has no residual activation in the region corresponding to LOC for visually or haptically presented objects (Snow et al., 2015). Detailed information about M.C.'s clinical and neurological case history and her neural selectivity to a range of natural and artificial visually presented objects can also be found in Snow et al. (2015).
M.C.'s perceptual and sensorimotor performance was compared with a group of 18 neurologically healthy age-matched controls who were tested in Scotland and North America. The control group comprised 12 women and 6 men, ranging in age from 38 to 46 years (mean = 42.4 years). All control participants provided written informed consent before testing, and M.C. provided verbal informed consent. All procedures were approved by the research ethics committee at Heriot-Watt University, UK, and the University of Nevada, Reno.
Patient M.C. and the age-matched controls sat at a height-adjustable chair in front of a large table to lift and judge the weight of six 3-D-printed hollow black plastic spheres (Figure 2A). The spheres had diameters of 5, 7, 9, 11, 13, and 15 cm and are denoted as Objects 1 (5 cm) through 6 (15 cm), respectively. The spheres were filled with different amounts of lead shot around their center of mass so that they each weighed precisely 266 g. The objects sat on the table surface on small concave circular stands to keep them stable before the lifts. Each object had a small plastic mount on the top surface to facilitate the rapid attachment and removal of a plastic and aluminium handle. The handle contained a single ATI Nano17 force transducer (Figure 2B; for further details, see Buckingham et al., 2009). The grasping pads on the handle were overlaid with a textured surface to prevent slippage during grasping. The force transducer recorded the force vectors tangential and orthogonal to the grasp handles at 1000 Hz. Participants were encouraged to adjust the height of the chair before the start of the experiment to ensure it was at a comfortable height for them to perform the object lifting trials (without the use of a chinrest).
Before lifting any of the illusion-inducing objects, M.C. and the control participants undertook a series of 15 practice lifts with three identically sized nonexperimental cylinders weighing 246, 266, and 286 g. Upon completion of the practice trials, each experimental sphere was placed in front of M.C. (and the control participants) in ascending order of size. Participants were then asked to rate how heavy they expected each object to be, without touching them, on a scale of 0–100.
At the start of each experimental lifting trial, participants were asked to close their eyes, at which point one of the six spheres was placed in front of the participant on the table. The stimulus was placed in line with the body midline within a comfortable reaching distance from the participant's body. An auditory tone then signaled to the participant to open their eyes and reach out to pick up and hold the object via the handle using the thumb and the index finger only. After 5 sec, a second auditory tone signaled the participant to replace the object on the table and to give a numerical rating of how heavy the object felt. As our analysis of fingertip forces focused on the initial lifts, it is critical to report the order in which the objects were presented. Following a case–control design, M.C. and the controls first lifted Object 3, followed by Objects 1, 2, 6, 4, and finally Object 5. All six objects were lifted 10 times in 10 similarly pseudorandomized sets, for a total of 60 lifts. The experimental trials took between 40 and 60 min to complete, in a single session without breaks, for M.C. and the controls.
Data Reduction and Analyses
The reported heaviness ratings for each participant were normalized to a Z distribution to account for individual variability in the scores. Given that the magnitude of the SWI does not change across repeated trials (Buckingham & Goodale, 2010b; Grandy & Westwood, 2006), the normalized ratings were averaged across all 10 lifts of each object to determine the presence and magnitude of the experienced SWI.
Grip force was defined as the force orthogonal to the grasp handle; the vector sum of the remaining forces was designated as the load force. These forces were smoothed with a 14-Hz Butterworth filter and differentiated with a 5-point central difference equation to yield their rates of change. The peak values of grip force rate (pGFR) and load force rate (pLFR) before lift-off were taken as the primary indices of sensorimotor prediction. By contrast, the way in which illusion-inducing objects are lifted has been shown to adapt rapidly to the actual mass of the object (Grandy & Westwood, 2006; Flanagan & Beltzner, 2000). Therefore, the pGFR and pLFR from only the first lift of each object was analyzed to determine whether object volume affected sensorimotor prediction.
Perception of Weight
Before lifting the experimental stimuli, but after lifting the practice stimuli, M.C. reported that she expected the larger objects to be heavier than the smaller objects in a roughly linear fashion. Qualitatively, M.C. also showed a clear sensitivity to the size of the objects: When asked to describe how each object appeared, she replied, “A bit of height and width; round at the bottom; looks quite small” for the smallest object (Object 1); “Oh, that's much bigger; looks round down the bottom” for one of the mid-sized objects (Object 3); and “Oh! That's huge! Looks like the circle is getting broader…that's really big” for the largest sphere (Object 6). All control participants reported that they expected the larger objects to outweigh the smaller objects in a roughly linear fashion.
In terms of how heavy the identically weighted spheres felt immediately after each lift, M.C. reported that the smaller objects felt increasingly heavier than the larger objects (Figure 3A). Indeed, M.C. showed a strong linear relationship between object size and perceived heaviness (R = .45). In a bivariate linear regression equation, we found that object (1–6) in our model provided a good prediction of reported heaviness (F = 15.02, p < .001). The slope of the model was −0.263, which was significantly different from 0 (t = 3.88, p < .001), providing strong statistical evidence that M.C. experienced a robust SWI.
To quantify the magnitude of M.C.'s SWI, we calculated the slope of the relationship between the radius of the object and the average rating given for that object (because objects feel heavier as they get smaller, experiencing the SWI would result in a negative slope). We then compared M.C.'s slope with that of the control group using Crawford's test for comparing an individual's slope to those from a normative sample (Crawford & Garthwaite, 2004). This test found no difference between the slope of M.C. and the control, t(17) 1.17, p = .26 (Figure 3B). In summary, not only did M.C. experience an SWI, but her SWI was indistinguishable from that of age-matched controls.
Fingertip Forces during Lifting
Next, we examined M.C.'s fingertip forces on the initial lift of each of the six spheres. In the context of SWI-inducing objects, healthy participants will typically use a significantly greater rate of force to grip and lift a large object than they would to grip and lift an identically weighted smaller object (Gordon et al., 1991; Davis & Roberts, 1976). We first confirmed that M.C. gripped the objects in an approximately normal fashion by comparing the average pGFR she applied over the first six lifts with that of controls with Crawford and Howell's modified t test (Crawford, Garthwaite, & Porter, 2010; Crawford & Howell, 1998). The force rates M.C. employed over these lifts were not significantly different from the control sample (p = .38), indicating that she did not employ a more cautious “probing” strategy than neurologically healthy individuals. In terms of her ability to use visual size cues to modulate grasping, M.C. showed a clear tendency to grip the spheres with fingertip force rates that broadly reflected their apparent weight, such that she gripped the largest objects with higher rates of force than the smallest objects (Figure 4A). It is worth noting that this pattern is quite variable and far from linear, presumably because of the complex interactions between trial order effects and object size on fingertip force control (Cashaback, McGregor, Pun, Buckingham, & Gribble, 2017; Loh, Kirsch, Rothwell, Lemon, & Davare, 2010). Indeed, degree of variability within the age-matched control group on this measure (presented in the Supplementary Material, available at https://osf.io/udqm8/) was sufficiently high that we were unable to conduct the Crawford and Howell's procedure to compare the M.C.'s slopes with those of the controls (Bartlett's test p < .05). Instead we calculated a simple metric of sensorimotor prediction by subtracting the force used to grip the smallest object for the first time from the force used to grip the largest object for the first time (e.g., Buckingham, Michelakakis, & Rajendran, 2016). These difference scores were then compared with Crawford's test for comparing scores from a single case and normative sample (Crawford et al., 2010). This analysis found that M.C.'s pGFR fell well within the normal range shown by the control participants, t(17) = 0.07, p = .94, with an estimated 47.2% chance of the normal population falling below M.C.'s score (Figure 4B). In other words, M.C.'s use of visual size cues to scale her grip force rates fell broadly within normal limits, despite her extensive occipitotemporal lesions.
Finally, we undertook the same analysis on M.C.'s pLFR applied during the initial lift of each object. As with the grip forces, M.C.'s mean pLFR on the initial interactions of each sphere was indistinguishable from the controls' (p = .51), indicating she did not employ a probing strategy when lifting. Qualitatively and consistent with her grip forces, M.C. showed a clear tendency to lift the spheres with force rates that broadly reflected their apparent weight, such that she lifted the largest object with a higher rate of force than any of the other objects (Figure 5A). To quantify her sensorimotor prediction based on visual volume cues, we again calculated the first-trial different score (force rate used for the smallest sphere subtracted from the force used to lift the largest sphere). As the pGFR, this metric of M.C.'s sensorimotor prediction was indistinguishable from that of the control participants, t(17) = 0.20, p = .84, with an estimated 42.1% chance of the normal population falling below M.C.'s score (Figure 5B). Thus, in line with the other reported metrics, M.C. appears to be able to use visual size cues to guide her lifting behavior, despite her extensive occipitotemporal lesions.
In this study, we investigated how visual size influences weight perception and sensorimotor prediction in M.C., a neuropsychological patient with extensive occipitotemporal lesions that encompass LOC bilaterally (Snow et al., 2015). We examined M.C.'s capacity for sensorimotor prediction and perceiving illusory weight differences in the context of the SWI—in which small objects typically feel heavier than equally weighted large objects (Buckingham, 2014; Nicolas, Ross, & Murray, 2012). A recent neuroimaging study implicated LOC in computing object weight (Gallivan et al., 2014). Here we used a neuropsychological lesion approach to determine whether the LOC plays a causal role in weight perception and/or sensorimotor prediction (i.e., expected heaviness). We hypothesized that if LOC is critical for computing object weight, then M.C., who lacks LOC bilaterally, should experience no SWI whatsoever and show no size-based sensorimotor prediction during her initial object lifts.
In fact, we found that M.C. experienced a clear and robust SWI, showing a significant positive relationship between object size and felt weight. Comparisons of the SWI in M.C. versus an age-matched neurologically healthy control group revealed, surprisingly, that the illusion experienced by M.C. was indistinguishable from that experienced by controls. Before testing, M.C. was able to judge the objects' relative sizes by correctly rank-ordering them in terms of how heavy they looked, and her qualitative remarks during the testing phase underscored further a sensitivity to visual size cues. These findings are particularly surprising given the extensive lesions to M.C.'s ventral visual system and the suggestions from previous studies that the strength of the SWI is related to the reliability of size as a cue to weight (Buckingham, 2014). We were also able to determine the extent to which M.C. was able to use size cues to guide her fingertip forces. Here, M.C. showed a broadly “normal” pattern of fingertip force rates, gripping and lifting the largest sphere of the set at a higher rate of force than the smallest sphere. Indeed, in terms of the most extreme objects, M.C. showed similar levels of sensorimotor prediction to the controls, suggesting that M.C. used visual size cues to guide the way she initially gripped and lifted the illusion-inducing objects. This fingertip force data must, however, be interpreted with caution because of the potential for trial order effects that are unavoidable in single-case designs (see Methods) to influence lifting behavior, which could account for the heterogeneous patterns of data seen particularly in Figure 4A.
Taken together, given that visual size cues induced a robust SWI and sensorimotor prediction in patient M.C., who has no LOC (nor intermediate ventral visual areas, such as V4, that can provide shape-related inputs to LOC), the data from the current work suggests that object-selective areas in the ventral visual pathway do not play a causal role in computing object weight when preparing to lift an object. These neuropsychological findings provide critical new causal insights into the role of LOC in coding action-relevant object properties. Gallivan et al. (2014) observed pattern-based decoding of fMRI signals corresponding to object weight, not only in dorsal premotor and contralateral primary motor cortex but also in the left and right LOC in the ventral visual pathway These results were surprising because action-relevant object attributes (i.e., weight) that are typically considered to be within the purview of dorsal cortex were nevertheless represented in the ventral “perceptual” pathway. Gallivan et al. (2014) surmised that ventral shape-processing areas are involved in integrating object information acquired through vision and sensorimotor experience in the service of guiding goal-directed actions. Our neuropsychological results, however, suggest that, although LOC receives information about object weight, the neural architecture responsible for computing weight lie outside LOC. Although Gallivan et al. (2014) did not conduct whole-brain searchlight analyses to determine whether other brain areas (outside the selected ROIs in PMd, M1, and somatosensory cortex) code object weight, all of these areas are intact in patient M.C.
If LOC carries information about object weight, how is this achieved and what purpose does it serve? In humans, the lateral surface of the dorsal and ventral pathways are anatomically connected via the posterior arcuate fasciculus and the vertical occipital fasciculus (Weiner, Yeatman, & Wandell, 2017; Yeatman et al., 2014), which presumably underpins the strong functional connectivity shown between the dorsal and ventral visual streams (Chen, Snow, Culham, & Goodale, 2018; Sim, Helbig, Graf, & Kiefer, 2015). Indeed, early visuomotor responses in the dorsal pathway functionally contribute to subsequent object-related processing in the ventral stream, in both humans (Sim et al., 2015) and monkeys (van Dromme, Premereur, Verhoef, Vanduffel, & Janssen, 2016). Although the functional significance of object-selective regions in the dorsal stream is currently the focus of intense investigation (Freud, Plaut, & Behrmann, 2016), accumulating evidence suggests that dorsal object areas can operate independent of action planning and execution (Kourtzi & Kanwisher, 2000; Faillenot, Decety, & Jeannerod, 1999; Grill-Spector et al., 1999; Sereno & Maunsell, 1998). Together, the available evidence suggests that, although LOC receives information about object weight from dorsal areas, this information is not necessary to experience or predict object weight. One possibility is that dorsal cortex relays information about likely object weight, based on prior manual interactions, to LOC (e.g., “motor memories”—see Loh et al., 2010; Chouinard et al., 2005), and this information could be relevant for determining object identity (Sim et al., 2015). Alternatively, LOC may represent cue information that is relevant in a given context for distinguishing one object from another (Lacey & Sathian, 2014). One final point to consider is that, as with all single-case studies, it is likely that there has been a significant degree of cortical reorganization of M.C.'s brain, and it is thus possible that the regions that were previously involved in representing object properties have been reconfigured anatomically. Similarly, it is quite possible that neural connections between brain regions have been reweighted to support her (albeit severely impaired) object vision. Furthermore, even with a large control sample, our statistical design and method would be unable to detect subtle deficits in either the magnitude of the perceptual effect or sensorimotor prediction. Nevertheless, it is important to note that previous (albeit univariate) whole-brain neuroimaging studies of object lifting and weight perception have not reported LOC activation (Chouinard et al., 2009; Jenmalm et al., 2006). An important avenue for future research will be to use noninvasive approaches, such as TMS, to determine whether LOC plays a causal role in weight perception (where there is insufficient time for large-scale cortical reorganization to take place).
In summary, we describe a patient with extensive ventrotemporal lesions completely encompassing LOC bilaterally who experiences the SWI, judging small spheres as feeling heavier than larger spheres of identical mass, and who lifts objects in a predictive way such that initial fingertip forces are influenced by the visual size of objects. These findings suggest that LOC is not causally involved in weight perception or fingertip force parameterization in the context of object lifting, but rather that ventral object-selective areas, such as LOC, are downstream recipients of weight-related information that is computed elsewhere in the brain. These neuropsychological data will serve as a catalyst for future convergent studies using neuroimaging and brain stimulation approaches to determine the brain regions that are critically involved in weight perception.
We would like to thank M.C. for her continued enthusiasm for taking part in our studies. We would also like to thank our control participants for volunteering their time, as well as two anonymous reviewers for providing comments on earlier drafts of this manuscript. Finally, we would also like to thank Rob McIntosh for engaging in helpful discussions about effect size and experimental power in the context of single-case work.
This research was supported by grants to J. C. S. from the National Eye Institute of the National Institutes of Health under Award Number R01EY026701 and an EPSCoR grant from the National Science Foundation (Grant Number 1632849). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or National Science Foundation.
Reprint requests should be sent to Gavin Buckingham, Department of Sport and Health Sciences, Richards Building, St. Luke's Campus, University of Exeter, Exeter, EX1 2LU, UK, or via e-mail: firstname.lastname@example.org.