“Location” Incommensurability and “Replication” Indeterminacy: Clarifying an Entrenched Conflation by Using an Involved Approach

Reproducible results and repeatable measurements at the same location are fundamental to science, yet of grave concern to scientists. Involvement in biological re-surveys under MVZ-Berkeley, Harvard-LTER and Hamaarag elucidated “replication” and “location” and untangled “incommensurability” from “no fact of the matter” and “indeterminacy.” All cases revealed incommensurability without indeterminacy on the smallest scale and indeterminacy without incommensurability on higher scales, with communication failure in the former and successful workarounds in the latter. I argue that an involved philosophy helps clarify fundamental concepts in this case, and since it could also help other cases it therefore should be examined in every case before being ruled out.

This paper emerged from a decade of involvement in long-term biodiversity surveys in places such as Yosemite Valley, Lassen National Park, New England Harvard Forest, and the Israeli Negev Desert. While actively engaged in the scientific work it was impossible not to notice the reoccurring discussions over how to replicate a survey to the same location. These two terms—“replication” and “location”—were necessary and basic for the scientists yet marginal (almost non-existent) in philosophical literature. The first two sections of this paper analyze the different practical meanings of replication and location through describing the survey fieldwork, and I argue for an unavoidable and basic uncertainty for re-visiting a location outdoors. The third section utilizes the re-survey description for clarifying Kuhn’s “incommensurability” and for differentiating this concept from “empirical equivalence” and two types of “indeterminacy.” The fourth section reflects on how the philosophical analysis was done and recommends an involved approach to philosophy of science instead of the detached mode common today. At least in this case an involved approach was a valuable tool, and I argue for its general, replicable, relevance to science and philosophy of science.

Replication: “the set of technologies which transforms what counts as belief into what counts as knowledge” (Shapin and Schaffer 1985, p. 225), is fundamental to science. Repeatability of a scientific experiment and reproducibility of its results are common scientific practices ever since Boyle ([1660] 1999)¹ and Redi ([1668] 1909), and it is widely accepted that one cannot fully explain a biological process nor empirically confirm a generalization without them (Shavit and Griesemer 2009; Shavit 2013).

Philosophers of science were traditionally more skeptical of the possibility and relevance of replication. Problems concerning replication were initially presented as epistemic absurdities, from Wittgenstein’s 1953 rule-following paradox: “No course of action could be determined by a rule because every course of action could be made out to accord with the rule” (Wittgenstein 1953, p. 201); to Popper’s note of the relativity of similarity, “But if repetition is thus based upon similarity…[it] means that anything can be made to a repetition of anything, as long as we adopt the appropriate point of view” (Popper [1959] 1992, p. 422), to Collins’s experimental regress: “The problem is that, since experimentation is a skillful practice, it can never be clear whether a second experiment was conducted sufficiently enough to be considered as check on the results of a first. Some further test is needed to test the quality of the experiment” (Collins 1992, p. 2) and so forth. Hacking (1983) concludes that the concern with replication is a philosophical pseudo-problem “…because, roughly speaking, no one ever repeats an experiment” (Hacking 1983, p. 231).

Given this long tradition of skepticism, it is apparently surprising to learn about scientists’ widespread and genuine concern for what Nature editors referred to as “the plague of non-reproducibility in science” (Hayden 2013): the fact that widely-published research in many scientific fields is never replicated, and may not even be replicable nor become generalizable (for example, in Nature see: Editorials 2014a & b; Bissell 2013; Russell 2013; Sanderson 2013; Gunn 2014). The bulk of attention is focused on biomedical research, but owing to the overwhelming variability in scope, scale, data structure, and semantics for studying the dynamics of our environments, the problem of reliable experimental replication is clearly applicable to ecological and biodiversity research (Michener and Jones 2012), as well as to agriculture, molecular biology, bioinformatics and other biological disciplines (Shavit and Ellison Forthcoming). Furthermore, in biological research, the spatial and temporal contexts—the location of a genome, cell, organism, population, habitat or ecosystem—as well as the researcher’s questions, methods, and available means of funding are constantly changing. Since biological research is contingent on the historical and social context in which it is being conducted, biologists are confronted with this key challenge: how do we both conceptualize and implement (operationalize) replication?

The term replication refers to wide-ranging practices: a) repeating the same measurement process (sampling, experimentation, natural experiment and so on), and obtaining comparable—not necessarily similar—results between both measurements; b) reproducing the same result from the same computational analysis on the same data without a new measurement process (Cassey and Blackburn 2006); and c) retrieving the individual entity, a physical item (specimen, photo, blood tissue, etc.) or a record (stored in a field-journal, excel spreadsheet, SQL database and so forth) for aggregating, comparing, and interoperating² the data. Different operational meanings of replication, along with their different goals and motivations, typically dominate different stages of the research, e.g. field measurement, data synthesis, metadata description and many more (Shavit 2013). In this paper I focus on the first meaning of replication: how to repeat an empirical process of measurement—species presence in these cases—at the same location. For that end we first need a clearer understanding of “location.”

Ever since scientific natural history became engulfed in a range of scientific disciplines (Kohler 2006; Strasser 2008; Nyhart 2009), revisiting the same location became necessary for the causal study of biological systems (Latour 1987, ch. 2). Moreover, it seems any explanation in ecology, biogeography or biodiversity requires, at a minimum, an identified location, a description of the distribution patterns of a population or species, and a comparison of location patterns for one or more spatial scales. Variables that correspond to changes in pattern, such as the location’s average temperature, may point to the process or processes that caused such a change in the distribution of organisms and groups of organisms (populations, species, etc.). In response to global climatic change (Lloyd 2010; Frigg et al. 2013) and global attention to a crisis of species’ extinction (Wilson 1992, ch. 12), the biological community became intensely engaged in meticulously tracing a species’ location back to its geographical point/line (Tingley and Beissinger 2009) on various scales and for various purposes (Kohler 2012). Scientists worked hard to ensure that their information would be available and interoperable for others to use in the future (Bowker 2005; Ellison et al. 2006; Karasti et al. 2010; Leonelli 2013).

The inherent vagueness of location is discussed in depth, although in very different contexts, in the philosophy of quantum physics (Barad 2007), in science and technology studies with regard to geo-political maps (Black 1997; Gugerli 1998), and in eco-feminist studies of the politics of inscribing social places and situated knowledge (Shiva 2000; Code 2006). However, a study of the various non-metaphorical meanings of location on multiple scales is relatively new to the philosophy of biology (Shavit and Griesemer 2009, 2011a; 2011b; Kohler 2012).

In order to clarify the concepts of location and replication, in addition to literary analysis, I was actively involved in three biodiversity case studies, participating over the course of three to eight years in fieldwork, lab meetings, and workshops. The case studies involved the research institutions of Harvard-LTER, at Harvard University, the Museum of Vertebrate Zoology (MVZ), University of California at Berkeley, and the Hamaarag at the Israeli Academia of Sciences. The research organizations conducted rigorous, repeated surveys of designated locations across New England, California, and Israel respectively. Their resulting models, concepts, protocols, and data set national and international standards in biodiversity research and policy.³ Hence, the analysis of their use of location and repeated sample is expected to be relevant for other scientists and philosophers of science alike.

In all these cases location meant two very different ways to operationalize one’s concept of space—exogenous to one’s system of study or interacting with it. Moreover, each operational location was committed to a very different set of best practices emerging from different epistemic values for conducting scientific replication: describing a location in a way that will allow it to be representative of many other locations or comprehensively accurate of that particular place. An exogenous location assumes that organisms’ impact on their environmental space, through their physiology, metabolism, behavior, or sheer existence, can be safely ignored for successfully predicting their distribution (Hutchinson 1978, pp. 159–60; Guisan and Thuiller 2005). An alternative, interactionist location assumes that organisms and their environments are mutually co-determined hence the above impacts should not be disregarded (Levins and Lewontin 1985, pp. 1–2).⁴ The problem is that both operational concepts of location, along with their conflicting and sometimes mutually exclusive practices, are necessary for rigorously repeating a survey at the same location.

Operating according to a specific concept of location in a specific working context signifies a commitment: an actual expenditure of resources (Gerson 1998) based upon specific constitutive and contextual values (Longino 2004), cognitive and social constraints (Longino 2004), and generatively entrenched (Wimsatt 2007, ch. 7) work procedures for coordinating the scientific work (Gerson 2007). An exogenous concept of space is committed to revealing general distribution patterns, and hence values representative data. On the other hand, an interactionist concept of location presumes that one cannot typically ignore the current and historical contexts of organism-environmental interactions, as sometimes they create a relevant casual chain that impacts upon a species’ location. An interactionist location values, therefore, comprehensive data on that particular location and species.

Given the goal of representativeness, an exogenous partition of space strives to locate a measuring device (e.g., climatic chamber, trap, camera, etc.) at a preselected random point that defines the longitude, latitude and angle of a regular shape (e.g., rectangle, hexagon, transact line, etc.). It deliberately attempts to ignore any hypothetical prior knowledge of the historical and biological context of the species, location, and studied field. On the other hand, an interactionist partition of space seeks to set that device in a place that is biologically meaningful to the organism studied. The device is set according to a preselected, non-random environmental stratification, typically by irregular polygons (e.g., constructed run-ways, preferred microhabitats, modulated patch-type, etc.) hypothesized to be relevant for understanding the dynamics of a particular biological system. When arriving at the field one must decide what to measure first; one type of measurement precludes the other measurement from taking place in the smallest spatial location and affects the possibility of repeating a visit to that same location.

In all three case studies, at the MVZ, Harvard-LTER and Hamaarag, scientists who jointly wrote the research grant and agreed on the sampling method only days before venturing outdoors, were somewhat surprised to learn that these practices became mutually exclusive on the smallest spatial scale outdoors. The issue of location/locality did not disappear over time, but rather brought about explicit conceptual discussions among the scientists regarding their technology, method and evaluation criteria (Shavit and Griesemer 2011a). These discussions, termed philosophical by the scientists, expanded and boosted the cooperative research in certain contexts and in other contexts stopped it altogether.

The problem was not a mere technicality. In all three cases a more precise GPS actually deepened the dilemma while working outdoors. Simply put, the problem was what to do first. The golden standard for an exogenous location requires a pre-chosen random⁵ set of independent spatial points, each with precise longitude, latitude, and elevation/depth coordinates. Only after a point-location was identified by the researcher outdoors and a measurement device was placed at that point and the GPS’s precision and extent were recorded was one supposed to record that point’s surroundings and ecological and geographical context. Unfortunately, an interactionist protocol requires the exact opposite: to first identify in the field, not the location of an abstract point but the location of a geographical and ecological context suspected of being causally relevant to the organism of study (e.g., its burrow, relevant micro-habitat, patch etc.). After identifying such a micro-environmental location, you set up the measuring device within/outside that locality, and only then record its grid coordinates with GPS.

Both exogenous and interactionist concepts of location are necessary for any rigorous biodiversity survey. Each operation of location however binds the researcher to different work practices for maintaining its standard. Since one cannot utilize both procedures for the same set of location data collected on the same spatial scale, location uncertainty is fundamental and inevitable (Shavit and Griesemer 2009). Furthermore, when performing an individual measurement on the smallest spatial scale of the studies here, in most cases I observed that a) the measurement device had to be relocated to different longitudinal and latitudinal coordinates when positioned according to different concepts of location (even in the uncommon incidents when the device maintained its lat./long. coordinates, its location was empirically different as its description as a location)⁶ and b) a barrier for communication between scientists holding different concepts was evident, translation was lost, and decisions were based upon hierarchy⁷ or separation of the data records.⁸ Upon later reflection of this incident, when interviewing the scientists involved, they did not refer to heuristic considerations as their cause of disagreement. For example, they did not say that the procedures used by their colleagues were more complex or required more time and effort.⁹ They also did not say that the statements delivered by their collaborators were false, but rather: “it did not make any sense,”¹⁰ or “he is smart, I simply could not understand why he was so stubborn on this issue,”¹¹ and often they only smiled gently and said: “I’m sorry, I could not do it the way they [or: he] wanted it.”¹²

This clear-cut empirical gap and communication breakdown in each of these biodiversity studies did not lead, however, to any disagreements on the overall answer to the question of species location and species distribution. Why? Since that answer was provided statistically, by aggregating results across higher spatial scales that encompass multiple individual measurements, as it was easily accepted that there are incompatible ways to analyze that aggregated data. Acknowledging the barrier at the individual trap made scientists frequently alternate between spatial scales as the relevant scale.¹³ They juxtaposed different concepts of location rather than seek a single concept for all levels. This facilitated the emergence of a productive scientific discourse on different operational meanings of location (Shavit and Griesemer 2011a) to consider alternative ecological and evolutionary models (Shavit and Griesemer 2011b). It also facilitated an ongoing public discussion on replication via symposium, international workshop and a monograph (Shavit 2013;¹⁴ Shavit and Ellison, forthcoming).

The philosophical concepts of incommensurability (Kuhn [1962] 1970; Feyerabend 1962), underdetermination (Duhem 1969; Quine 1953), and indeterminacy of translation (Quine 1960, 1990) seem especially relevant in this case. For both incommensurability and indeterminacy of translation “the paradoxical situation stems from meaning variance—the same terms have different meanings in the seemingly incompatible theories” (Ben Menahem 2006, p. 11), yet only “incommensurability implies that from the perspective of one paradigm (theory), the alternative is not simply false, but makes no sense at all” (Ben Menahem 2006, p. 11). Listening to the biologists discuss adequate location and replication while participating in the re-survey created the impression that this is indeed a case of incommensurability.

In the next section, I will employ these concepts for describing the small details of the scientific practice with which I was involved. Such a description will render the scientists’ disagreement more sensible than bizarre, which is presumably a better description.¹⁵ Paying close attention to the scientific routine clarified a common philosophical conflation between incommensurability and empirical equivalence (Ben Menahem 1990) and between indeterminacy of translation and of reference, and should therefore assist in avoiding it. Noticing and reflecting upon the conceptual aspects attached to a scientific disagreement also facilitated a discussion among the scientists that clarified their operational concepts and working procedures. That is, involvement in the routine scientific work not only can help to better describe and understand science—the standard role for the history and philosophy of science—but may also illustrate the benefits of philosophy for science (Griesemer 2011), at least for some scientists and HPS scholars.

Philosophical discourse is rich with discussions over incommensurability, yet is still replete with conflation of incommensurability with empirical equivalence (Ben Menahem 1990). I first met Thomas Kuhn’s Structure as a teenager and “like many philosophers of my generation and the one preceding it, I was first attracted to the philosophy of science by The Structure of Scientific Revolution. But like many of my peers, I found that once I started actually doing philosophy of science, I was far less of a Kuhnian than I had thought” (Levine 2010, p. 376). In this article, I will show that a Kuhnian view of incommensurability can be sensibly held if the philosopher of science participates, notices and cares about routine scientific work.

Elucidating what a Kuhnian view amounts to is beyond the scope of this paper. Given space constraints, I will not discuss here the interesting debates over this notion’s history (Agassi 2002; Oberheim 2005), sociological context (Demir 2008) or philosophical compatibility with realism and scientific progress (Hoyningen-Huene 1993; Sankey 2009; Hoyningen-Huene and Oberheim 2009; Davies 2013). Instead, I will depend on the common understanding that incommensurability refers to a translation breakdown (Ben Menahem 2006; Levine 2010) and utilize Davies’ 2013 explicit definition of the term:

Davies links incommensurability specifically to its empirical extension, and also defines “quasi-analytic” principles as principles formed while conducing empirical research that function not as truth-evaluable descriptions of the world but as meta-linguistic statements (Davies 2013). I will rely on this and on Ben Menahem’s argument against the association of incommensurability with empirical equivalence between semantically non-equivalent theories (1990, 2006); and, as a result, against the conflation of incommensurability with indeterminacy of translation and the common phrase no fact of the matter. As Davies’ definition points out, and as seen in our case studies, the traps were placed at different places on the ground given different operational concepts of location, so indeed incommensurability held a clear and pressing fact of the matter.

Most history of philosophy of science scholars nonetheless conflate these two. According to Ben Menahem, the blame for conflating incommensurability with ‘no fact of the matter’ could be placed on Kuhn’s [1962] 1970 explicit claim that paradigms are incommensurable and are therefore equivalent in the sense that there is no fact of the matter as to which paradigm to adopt (Ben Menahem 2006, p. 94). Exactly who and what brought about this conflation is the topic of a larger historical project, and in our context suffice it to note that Kuhn himself clearly does not connect incommensurable paradigms with different facts: “Just because it is a transition between incommensurabilities, the transition between competing paradigms cannot be… forced by logic and natural experience” [italics mine] (Kuhn 1970, p. 149). I agree with Ben Menahem that a claim for no fact of the matter does not follow from incommensurability, nor does it conform to well-known examples of empirical equivalence (for example Poincaré’s argument for the empirical equivalence of different geometries). Briefly stated, incommensurability and indeterminacy of translation are not closely related.

Then what is indeterminacy of translation? “The thesis is then this: manuals for translating one language into another can be set up in divergent ways, all compatible with the totality of speech dispositions, yet incompatible with one another” (Quine 2004, p. 120). There is no barrier of communication. However, due to the lack of logical inference from observational to theoretical sentences, i.e., the underdetermination of theories, very different theoretical sentences can fit the same observation sentence rather than a one-to-one relationship between theory and data. That is, the meaning of the data is not a determined entity that is somehow captured in our minds (the museum myth) and is independent from its translation.

In our case studies, a repeatable survey of the same location, i.e., a detailed protocol for re-measuring the same location in the same way, was constantly translated while moving between working protocols. Such explicit discussions between adherents of the different operational concepts of location, alternating between concepts across spatial scales, occurred when one was committed to choosing a statistical package for analyzing one’s data. It was accepted as common knowledge that different statistical packages are based upon different theoretical constraints and on different idealized premises of how best to aggregate results from several spatial scales. Although the exogenous and interactionist concepts of space were evaluated differently by various researchers and cultural-research bodies, and unlike the breakdown of communication outdoors at the single trap scale, at the lab, in all three cases, researchers could agree on a manual for translating the aggregated operational concept of location between different statistical packages. They agreed even though they knew very well that these statistical packages employed different computational rules and constraints for analyzing their collected data, many of these rules completely unrealistic.

Quine distinguishes such an indeterminacy of translation, which is manifested when different (and incompatible) sentences correlate to the same empirical data from the indeterminacy of reference (what he terms “ontological relativity”). A reference is indeterminate when the terms of the same sentence (or theory) can be correlated with the world in different ways (Quine 1990). In this instance, different empirical content may fit the same sentence, and there is no fact of the matter. In all of our case studies, such indeterminacy of reference occurred. Researchers easily agreed on the truth-value of sentences describing the survey results, either observational sentences summarizing the fieldwork or prediction sentences resulting from their SDMs (species distribution model), while alternating between incompatible causal interpretations of these sentences regarding the world.

For example, researchers agreed on the number of organisms and species detected on a transact line, e.g., thirty Peromyscus maniculatus (deer mouse) collected from Yosemite Valley in 1913 and in 2013, yet were hesitant to agree on the causal meaning of this empirical result. That is, even if it is true that the same number of deer mouse were taken from the same location, whether or not the survey process itself was repeated will determine, at least in part, if the findings regarding the same deer mouse’s population, e.g., its size or structure, were in fact reproduced across a century of research. Perhaps the same number of rodents occupied the same number of traps in 1913 and 2013, yet past collectors were much more experienced and efficient in collecting animals than today’s collectors? Different causal dynamics, such as where the rodent population grows or declines, can still correlate with the same number of mice caught,¹⁶ so researchers firmly maintained their skepticism as to the ontological interpretation of their observation sentences. Such indeterminacy of reference clearly differs from indeterminacy of translation, as the former denotes causally interpreting empirical results and the latter to choosing a suitable model to analyze these results. However, both concepts help address the pressing problem of replication mentioned at the beginning of this article (Shavit and Ellison forthcoming) and both clearly differ from the incommensurability of placing a measurement device in what scientists call the same location.

In this study, a clarification of concepts was gained by an involved philosophical approach. I use involved here in the everyday meaning of care and active engagement. That is, both caring that the analysis will be true to the scientists’ actual speech-act, accurate from the scientists’ point of view and relevant to their interests, and active engagement in that the scholarly account will emerge from an active participation in routine scientific work. Such an involved approach, which is typically expected from an anthropologist rather than a philosopher, in effect enriched and improved the analysis. I argue that since this approach clarified very basic scientific concepts and philosophical conflations in three major case studies, and, since it could be relevant to other concepts and case studies as well, it therefore should be tested in these other cases before being ruled out.

To recapture, the main benefits of noticing that biologists have very different concerns and goals and use very different procedures when repeating a survey at the same location are: a) the unpacking of a common and long-lasting conflation between incommensurability, empirical equivalence and no fact of the matter; b) distinguishing incommensurability from indeterminacy of translation and both from indeterminacy of reference; c) exposing the philosopher to a basic scientific problem (before it became celebrated in Nature) although it was often perceived by her colleagues as a mere philosophical pseudo-problem; and d) making the invention of new pseudo-problems more difficult since the philosopher could easily notice if what she calls a problem is relevant to her scientist colleagues.

To be sure, no fancy method was employed here nor did the philosopher help or guide the scientists. Instead, the scholar merely described to the scientists what she saw and heard and what she read in their historic archives, consequently inviting a reflected response. It was the scientists’ commitment to rigor that conceptually distinguished a basic uncertainty at the smallest spatial scale outdoors from a worked-around ambiguity when analyzing the aggregated location data back at the laboratory. Together, the scholar and scientists explicated the benefits of containing this communication breakdown in the former case, of working around the translation differences in the latter case, and noticed the benefits of not conflating the two.

In that respect, an involved philosopher’s findings are deeply and systematically contingent upon her specific scientific partners. She cannot make a claim for necessary truths about science or the history and philosophy of science, yet her dependence on contingent and localized contexts may also liberate her from the confinement of inevitable results and thus increase her autonomy (Ben Menahem 2009). Finally a lack of universal results by an involved philosopher does not imply a lack of importance. For example, since an explication of replication is critical for regulating and standardizing science (Shavit and Ellison forthcoming) and since our society is immersed and often regulated by science, then the philosopher’s findings in this case can clearly impact society at large.

The benefits of this approach should not surprise us. Nor is it completely new. After all, a wise man already said that “‘to give a new concept’ can only mean to introduce a new deployment of a concept, a new practice” (Wittgenstein [1956] 1983, p. 432). The aim of such an involved dialogue between a practically oriented philosopher and a reflective scientist is not to transform either of them, but to systematically build interdisciplinary bridges while minding the gaps between them.