Chromatophony: A Potential Application of Living Images in the Pixel Era

Cephalopods such as squids are known to rapidly change their body color to display patterns for intimidation or camouflage. Using this phenomenon, we created Chromatophony, an artwork in which chromatophores, organelles composed of living cells that allow the squid to change color, are converted into an audio visualizer by electrical stimulation.

In this paper, we use Chromatophony as a means to reconsider bioart from the perspective of visual art. We report on the significance of an idea called Living Images in the practices of bioart, as a means of proposing a critical perspective countering the dominance of the unit of the pixel in contemporary visual display.

We first describe the idea of Living Images and then discuss the historical background of the pixel to compare digital images and Living Images. We explain our work and the significance of Living Images within biomedia art as well as within contemporary visual culture.

We define Living Images simply as aesthetic images generated from living cells, exemplified in the history of scientific practices and the recent rise of bioart: from Alexander Fleming’s microbial paintings [1] to recent trends in bioart through experimentation with biological networks using slime mold [2]. Although Chromatophony (Color Plate C) can be placed within this category, our work notably attempts to embody the idea of Living Images literally, as we explain below.

The idea of Living Images is derived from the hypothesis, declared recently in the domains of art history and visual culture studies, that images are alive [3,4]. Hans Belting, for example, is one of the leading theorists of Bild-Anthropologie, which examines how images have been intertwined with bodies and media across cultures throughout human history. According to Belting, mental or physical images are never fully controlled by humans but rather are nomadic entities embodied by our bodies and media [5]. From ancient burial objects to contemporary digital photography, Belting emphasizes the functions and vitality of the image independent of humans that enable it to inhabit the human body and technological media.

While Belting describes “living images” from an anthropological perspective, we have appropriated this idea and applied it to bioart. Bioart is a practice of using the knowledge of biology as an artistic medium, or of advancing the changing nature of life through artistic output [6]. Along with the rise of bioart, the development of molecular biology and genetic engineering also continued during the latter half of the twentieth century. The recent discovery of horizontal gene transfer [7] and the way viruses and pathogens proliferate overlap with the nomadic images proposed by Belting. Therefore, we focus on bioartworks composed of cells or tissues that are literal living images.

We also compare the pixel, the elementary unit of digital images, context of Living Images. We refer to the studies of Alexander Galloway [8] in which he detaches the pixel from its computer context and places it on a historical horizon with reference to Neo-Impressionist pointillism and eighteenth-century calculators. From this perspective, we evaluate the analogical functions of chromatophores and pixels as units of living/digital images.

In 1965, the term pixel appeared in a technical report titled “Digital Video Processing at JPL” [9], derived from the words “picture element” and “picture cell.” For a time, it was used alongside the word pel, also derived from “picture element.” Gradually, the word pixel came to mean an “element in an image sensor,” or the smallest unit of a digital image, a lightemitting element composing a digital display [10].

In contrast to the historiography of this invention, Galloway extended the implications of this term epistemologically by focusing on its functions. Initially, he noted two modes of the pixel: the square as the smallest geometric point and the numerical value as an algebraic number without substance. He then argued that even before the computer, pixels could be found in the technical and aesthetic practices of the modern era, starting in the eighteenth century. If we understand the pixel as a small square, it is possible to view these geometric points as a descendant of artistic expressions such as De Stijl, color field paintings and Neo-Impressionist pointillism. Conversely, given that pixels function as a numerical value, they had been incorporated by Jacquard looms as well as Nicholas Saunderson’s eighteenth-century calculator. The latter, a manual calculator, had a procedural algorithm and mechanism that involved the interlocking of a grid of pins to facilitate computations.

Based on these two aspects, Galloway reconsidered the features of a pixel as a given coordinate or luminance value that could be processed mathematically: “The pixel is to the digital image as the frame is to the cinematic image” [11]. In other words, a single pixel does not make sense unless it is assembled with other pixels in an image, just as a frame in a film only has meaning if projected continuously with other frames. Further, pixels divide images spatially, just as frames divide motion temporally.

Galloway is not the only one to trace the genealogy of pixels back to film and Neo-Impressionism. Among studies in aesthetics and art history, Meredith Hoy states in her book From Point to Pixel, “Pointillism contains enough digital elements from the beginning to the end of production to warrant consideration as a digital system, but it is not computational” [12]. From this perspective, the little square pixel can be treated as the descendant of modern paintings, although they exclude its numerical function.

Before Galloway and Hoy, Sean Cubitt stated in his book The Cinema Effect that the equivalent of the pixel could be found in the pointillism of Neo-Impressionists such as Camille Pissarro, Georges Seurat and Paul Signac, who translated light into pigments. Further, Cubitt extended this relationship to the contemporary products of the Lumière brothers: the cinematograph and Autochrome [13].

Although the emphasis differs from art history to media theory, these discussions are significant when examining the context of Chromatophony, which uses the pigmented colors that are embedded in the skin of a squid to form a visual display. To compare the chromatophores constituting Living Images with the pixels in digital images, we adopted claims that focus on the visual and spatial as well as the algebraic and temporal aspects of pixels.

Rapid color change in squids is made possible by a set of organelles called chromatophores, comprising multiple muscle and nerve cells, and pigment-containing pouches [14]. Each chromatophore contains only one type of pigment; during color changes, only the pouches contract or expand, changing them in size [15]. This makes chromatophores an elementary unit of Living Images just as the pixel is in digital images as some biologists implied [16].

These “dots” come together to form larger patterns, whose density of colors evokes the impression of pointillist paintings of the Neo-Impressionists; zoologist Andrew Parker compares the way these pigment vesicles come together to create colorful images to pixels on a screen [17].

This analogy is not confined to visual appearance but extends to function. Each squid has chromatophores with unique relative positional relationships [18]. Further, their ability to change color is based on the physical coordinate values of the chromatophores stimulated by nerve impulses. This feature is similar to the algebraic aspects of pixels that represent numerical values, as Galloway has described.

Therefore, we might summarize the correspondence between pixels and chromatophores as follows. First, from a geometric viewpoint, both are small dots in a larger pattern. Second, just as pixels in a digital image connect to an electrical circuit to emit light in response to an assigned luminance value, the chromatophores on the body of a cephalopod are connected to neural circuits, resulting in the contraction of muscles in response to an electrophysiological signal.

Given that their visual appearance and their functional system resemble pixels, it might be no coincidence that one of the etymologies of the word pixel is picture “cell.” Therefore, if a pixel is a “cell” in a digital image, so too can a chromatophore be a “pixel” in a Living Image. Our aim is not just to appreciate the beauty of chromatophores but also to see the living cell as a medium for composition of visual artwork. The significance of this work is discussed below, where we explain the works we created in detail and discuss the possibilities of expressions created using chromatophores.

In work similar to ours, Backyard Brains (BB), a company that sells educational neuroscience laboratory kits, conducted an experiment wherein music was used as the electrical stimulation to manipulate a squid’s chromatophores [19]. In the video, they stimulate chromatophores by connecting electrodes to the fins of a euthanized longfin squid. The video shows the chromatophores opening and closing, responding mainly to the low bass and kick sounds of the song “Insane in the Brain” by Cypress Hill. Greg Gage, cofounder of BB, explained that this choice was made because low sounds tend to generate action potentials in motor neurons [20].

In contrast, we created music tailored to the characteristics of the chromatophores and made them audible to people through speakers to achieve audiovisual unity when observed. To determine the signals most likely to stimulate chromatophores, we selected a sine wave as the electrical stimulus and investigated the relationship between the response of chromatophores and the signal while adjusting its frequency.

For this project, fresh squid was ordered from Yobuko port, Saga Prefecture, near our laboratory. The squid was filleted before the experiment and a clipped sample was used. The stimulus was applied to the epidermis of the squid through iron electrodes, which were attached to an audio cable. Figure 1 shows the setup used for the experiment. The voltage of the stimulus was increased until the chromatophores exhibited the desired reaction. The distance between the electrodes was 15 mm, and the voltage ranged from 0.4 to 0.8 V. The action potentials of the squid nerve ranged from 5 mV to 10 mV, which differed significantly from the voltages used in the experiments. This difference may have been due to the high resistance of the electrodes and the sample. The results indicated that the squid’s chromatophores responded best to stimuli of approximately 90 Hz and barely responded to stimuli exceeding 800 Hz (Fig. 2). When similar successive stimuli were given, the response of the chromatophores could be erratic. The time required for chromatophores to open in response to the stimuli differed based on their color (Fig. 3).

Based on these results, we created music to stimulate the chromatophore by editing a low-frequency stimulus (around 90 Hz) as sound material on Ableton Live. We thereby created the video work Chromatophony. The reaction of the chromatophores was recorded using a digital microscope (VHX-5000). The result was a computer graphic–like video in which the geometric movements were coordinated with our custom music, although the system used was simple and the responding material was organic (Fig. 4).

Considering the chromatophore as a unit of Living Images, the possibilities we can derive from this work should be further explored. Although the placement and color (RGB) of each pixel can be specified in advance, stimulating chromatophores using electrodes is less accurate, and we do not have the ability to move individual chromatophores. However, our aim was not to control their organs but to modulate the audible thresholds as an aesthetic condition between living things. In fact, the result becomes apparent through the fluidity and dynamic movement in the images, which achieves a display that cannot be replicated by software. In realization Chromatophony ceases to be a mere video artwork but becomes a site-specific one, considering that we had to acquire fresh squid for each performance. Although it had to be recorded digitally and shared on a pixel-based screen, we consider these specific features of Chromatophony an embodiment of Living Images.

We discussed Chromatophony by comparing chromatophores and pixels from the perspective of media theory and demonstrated the distinctive characteristics of images formed by chromatophores. Our aim for the future is not to replace the squid’s chromatophores with pixels but to show more diverse ways of understanding visual expression. Therefore, below we expand the discussion and compare Living Images with digital images.

Above, we define Living Images as aesthetic images generated from living cells. Johanna Rotko’s yeast-based photographic project (Fig. 5) [21], Diana Scherer’s root-sculpture project (Fig. 6) and experiments of natural computation visualized in slime mold (Fig. 7) [22] are examples of recent works. Alexander Fleming’s microbial painting is an early example of this kind of work. Not only are his images bioart, they were also created in the process of scientific experimentation using living organisms. As these works are also the corporealization of living images, we can formulate their features specifically.

First, their textures are not restricted to displays or printing limited by arbitrary signals—RGB or CMYK. Traditional paintings use natural pigments whose colors could not be quantified easily, and printed materials have unique textures resulting from the materials used, including the pigment and pulp. Furthermore, the Autochrome photographic technique used potato starch as a filter to realize vivid color development. Living Images make us aware of the texture possibilities that natural components combined with digital technologies can create. For example, Chromatophony combines complex color texture with computational audio signals. Thus, Living Images can change how we perceive the historical role that living things played in art, such as carmine refined from scale insects.

Moreover, Living Images present unpredictable features we cannot control. Although the cells themselves can be analyzed in chemical and physical terms [23], living materials can also produce spontaneous and contingent movements. Those reactions sometimes produce novel results in visual images exceeding our intentions and expectations, and this is one of the reasons bioart is receiving increasing attention.

Our point is not to praise the beauty of nature in contrast to artifacts created from it, for the examples we discussed did not use “pure” natural products, but to claim that living things are inherently hybrids of the natural and the artificial. Therefore, we apply the term aesthetics to Living Images, as it concerns works of art as well as implying an attempt to critically examine the senses themselves, which are embodied in specific technocultural conditions. Thus, Chromatophony is not a “re-presentation” of concrete pictures or figures but an attempt to “present” the dynamic process of a Living Image, which becomes perceivable through cells.

We have discussed the idea of Living Images and the historical background of pixels as a prelude to the discussion of their characteristics. We then compared chromatophores and pixels in the context of media studies and proposed Chromatophony as an example of a Living Image. Finally, we discussed the characteristics of this work compared to other bio-art and the possibilities of using Living Images as visual expressions.

However, the scope of Living Images is still debatable. It is necessary to discuss whether the smallest unit of Living Images needs to be a cell and whether the cell needs to be alive. Inevitably, this leads to discussions regarding the ethical considerations of life and death.

The possibilities of various forms of Living Images that are not mere replications of photographs or paintings can also be discussed. In this respect, it is important to represent images that are recognizable to humans while focusing on the dynamic process through which images emerge across species; this is what distinguishes Chromatophony from other bio-art.

As images flood networked media today, the number of shares and clicks seems to measure their value. In contrast, Living Images cannot be edited or shared as easily. This may seem inconvenient at first; however, in today’s world overflowing with images, Chromatophony gives us an opportunity to reconsider the way in which we interact with images. By analyzing the units of Living Images and comparing them with pixels in digital images, we can observe the material conditions of their vitality, providing us with a fresh perspective on the theory and history of images.

This project was supported in part by JSPS KAKENHI (grant nos. JP17H04772, JP19H01225, JP20H01203 and JP21H00495).