Abstract

We introduce three of our interactive artworks that translate text into artificial creatures or creatures into text by means of user interaction. These installations make use of experimental literature, media archaeology, surrealism, artificial life, and algorithmic methods.

1 Introduction

The principles of artificial life and its evolutionary computation have been important components of several of our interactive art installations since the early 1990s [24, 23, 31]. Our main motivation is to create links that enable the users' interaction data to influence the evolution of artificial systems. From 1997 on we also started to use text that can be produced by users in the installations. In our systems text functions as a genetic code for artificial creatures. With the help of an algorithmic text creation process, users metaphorically give life to their thoughts, expressing them in the form of text. These text fragments are then transformed into creatures that can evolve. Conceptually these interactive artworks are based on experimental literature, artificial life, meme theory, and open artwork ideas from process-based art.

2 Algorithmic Text Creation Techniques and Gene-Culture Coevolution

In 1920 Tristan Tzara, who was a central figure in Dadaism (a movement that influenced art, music, film, and literature), thought of creating a system in which newspaper clippings were poured out of a bag to produce new forms of random poems and experimental typographies. The Surrealist movement also took up several of the techniques of Dada automatic literature and pinpointed the unconscious as a source of inspiration. Methods for making cut-ups and collages were extended through new forms such as exquisite corpse techniques, automatic poetry, indecipherable writing, and surautomatism (a theory or process that involved taking automatism to its most absurd limits), as well as collaboratively written texts such as were used in Les Champs Magnétiques by André Breton and Philippe Soupault [5] or texts produced under hypnosis by Robert Desnos [10].

Inspired by the Dadaists and Surrealists and by the Letterist [8], the Beat poets William S. Burroughs and Brion Gysin rediscovered the cut-up technique in the early 1960s. Developing it further, they took existing newspaper texts and images, cut them up into pieces, and recombined them to form collages, thereby adding drawings and photographs. Burroughs also photographed the resulting collages, cut these up again and created new collages. He “established a new form of writing: the cut-up method—whereby text and image fragments are intuitively pieced together to form open associative narrative structures in order to expand the boundaries of language and describe human consciousness” [12]. A driving force behind Burroughs' work was the striving to eliminate control. This also becomes clear in his 1959 novel Naked Lunch [6]. The book is written in a nonlinear fashion and can be read in any order. Under the influence of various mind-altering substances, the protagonist of the novel sees his typewriter, a Clark Nova, transform into a giant talking insect. In an interesting statement Burroughs draws a concrete connection between words and living organisms, regarding the typewriter as a living organism: “I suggest that the word is just such a virus […] I advance the theory that a virus is a very small unit of word and image” [7].

This idea can also be connected to the meme theory, which was propagated by Richard Dawkins in 1976. He asserts that the spread of ideas and cultural phenomena can be explained with the help of biological and evolutionary principles. “Cultural transmission is analogous to genetic transmission in that, although basically conservative, it can give rise to a form of evolution” [9]. While he admits that “Language seems to ‘evolve’ by non-genetic means, and at a rate which is orders of magnitude faster than genetic evolution” [9], he still sees a strong connection between natural evolution and the manner in which cultural ideas, symbols, or various cultural practices such as melodies, catch-phrases, fashion, know-how, or fads proliferate within a particular culture.

The link between evolution and human culture was also elaborated by Charles Lumsden and E. O. Wilson, who developed a general theory of gene-culture coevolution (and introduced the term) in 1981 [19]. E. O. Wilson even goes as far as to link evolution to an epic, stating that “[t]he true evolutionary epic, [when it is] retold as poetry, is as intrinsically ennobling as any religious epic” [34]. The British philosopher John Dewey in 1929 said, “The mind has a special relationship to language and communication. Only a communicating system in a social environment can literally think, because thinking is symbolic and symbolism is social” [14].

3 Living Poetry by Christa Sommerer and Laurent Mignonneau

Our interest in combining interactive art, language, and artificial life goes back to our interactive artificial life installation Life Spacies created in 1997 (Figure 1). In that artwork users could write text on a keyboard, either on site in the NTT-ICC museum in Tokyo or through the Internet [25]. When they wrote an e-mail, an artificial life creature was created that lived in the virtual environment of the NTT-ICC museum. In return for each e-mail they sent, the users received an image of and information about the specific life characteristics of their creature by means of e-mail. On-site users in Tokyo could play and interact with these three-dimensional creatures, which were projected on the screen, and with their gestures capture them and cause them to be reproduced on the screen.

Figure 1. 

Life Spacies artificial life environment at NTT-ICC Tokyo in 1997.

Figure 1. 

Life Spacies artificial life environment at NTT-ICC Tokyo in 1997.

The written text input of the users' messages was used as a genetic code for the artificial creatures' appearance, behavior, and (ultimately) survival. An in-house text-to-form editor linked the characters and the syntax of the written text to design parameters of the creature's body. The composition, length, and arrangement of the text determined the creature's shape, number of limbs, color, texture, and ability to move. When the users on site at the NTT-ICC museum interacted with the system, the artificial Life Spacies creatures moved around on the projection screen, eager to survive and propagate. On-site users could enhance the breeding process by activating and touching the creatures with their hands. A chroma-keying technique was developed that allowed us to integrate the user's image into the 3D graphical environment. When the user touched two creatures with his hands, they exchanged their genetic code and created a child creature. All of the creatures in Life Spacies were also able to move around the user's projected image in a simulated three-dimensional space. As the life span of each creature had to be limited (based on computing resources), an e-mail was sent back to each creator informing him/her about the popularity of his/her creature, the number of offspring it had, how many times it had been “touched” by the visitors, and the time of its death.

In a later version called Life Spacies II (1999), creatures chose their mating partners by themselves. The creatures were fed with text provided by the user, written on an interface that enabled them to specifically feed their creatures. This enabled them to acquire energy and eventually reproduce. In this version the integration of the images of the user into the 3D environment was omitted. A detailed description of the text-to-form encoding and algorithms behind Life Spacies and Life Spacies II appears in the literature [26, 27].

4 Life Writer, 2006

In 2006 we continued to use artificial life concepts, evolutionary programming, and the translation of text into three-dimensional forms. This time, however, instead of employing a keyboard, we transformed a vintage typewriter from the 1940s into a computer interface for the interactive work Life Writer [28, 29]. We kept all the usual functions of the typewriter's keys intact and functional and connected them to our computer software via an in-house-built interface. The Life Writer stands on an old table, and a projection shines directly from above onto the paper in the typewriter. This gives rise to the impression that the paper is becoming the computer screen.

When a user writes text on the typewriter, it first appears on the paper. It is real paper, but the text that appears on it is projected. The movement of the paper is seamlessly linked to the movement of the typewriter's paper roll. An in-built sensor tracks the position of the carriage return and matches it to the position of the projection screen. This creates the illusion that the text is emerging from the machine, as if it were a normal printed text from the typewriter. Once the user activates the carriage return handle of the Life Writer, however, the text that initially appeared on the paper begins to transform into black-and-white spiderlike creatures (Figure 2).

Figure 2. 

A user typing text to create artificial life creatures on the Life Writer.

Figure 2. 

A user typing text to create artificial life creatures on the Life Writer.

The transformation of text into form is done in a similar way to that in Life Spacies [26]. Depending on the complexity of the text, the bodies and limbs of the creature become increasingly differentiated, modulated, and varied. As there are usually considerable variations in the texts and characters typed by different people, the creatures also vary greatly in appearance. Two identical texts produce creatures that look and behave identically.

A new creature immediately moves around and starts to crawl about on the paper. It tries to find text to eat. When the user types more letters, creatures quickly snap them up, but they can only use those letters that are part of their own genetic code (= their own text). Once a creature has succeeded in eating enough text, it can also look for a partner and reproduce. The creatures' children look similar to their parents and inherit their genetic codes with mutations. Eventually the paper fills up with more and more hungry spiders that end up trying to snap up all the text that the users write. The user can also kill creatures by pushing them off the edge of the paper or squeezing them back into the machine. This will make them disappear, thereby creating space for new creatures.

The use of obsolete technology such as an old-fashioned typewriter is a reference to media archaeology, an emerging field of study that examines digital culture and memory in connection with previous technological and historical developments. Erkki Huhtamo states, “Media archaeology travels not only across time but also across media. […] Whereas the convergence of media has often been seen as a defining characteristic of the digital era, media archaeologies broaden the analysis of the relationship of media devices, formats, genres, and aesthetics far beyond the digital, tracking convergences, divergences, crossing paths, and overlapping within a much wider cultural space” [16].

In summary, we can say that Life Writer is an artificial life interactive artwork that combines old and new technologies with the help of a media-archaeological interface; it is also an example of an art form in which the artwork evolves towards a living art, an idea we proposed in 1996 [18]. It extends algorithmic poetry with the help of evolutionary programming, and constitutes an open-ended system in which participants engage in the act of creating artificial life by means of writing.

5 Escape, 2012

In 2012 we developed another media-archaeological interface for the interactive installation Escape [30]. This work of art was developed as a site-specific installation for the civilian air raid shelter in Salenstein Switzerland, as part of our solo show at The View Contemporary Art Space.

The project consists of an antique film projector and projection screen from the 1930s. The projector was modified so as to contain a small LCD video projector. Through an integrated sensor we measured the movement and speed of the crank with which the 8-mm film was originally scrolled forward. The film was replaced by in-house computer software; the images are projected through the projector's lens.

When visitors enter the room, they see a fat fly sitting on the antique projection screen. Once they start turning the crank of the old projector, the fly starts to move around frenetically, as if it were trapped and trying to escape. If the user continues to turn the handle, more and more flies appear and come together as if they had discovered something to eat.

The movements of the flies are programmed so that when they encounter a hidden text pixel below their bodies, they stop moving and call more flies to come to them. All of the flies fly in random directions in the virtual 3D space between the projector and the screen. Once they bump against the surface of the screen, they start walking about the screen in a random fashion and sit down at the place where they encounter a hidden text pixel.

This process results in the rapid elucidation of different parts of the text as the number of flies on the screen surface increases. Since every movement of the flies is based on random walking patterns, the text appears quite evenly over the entire surface of the screen after a short time. If the user keeps turning the handle, the text becomes more and more legible, as more and more flies adhere to it and thereby make it increasingly visible. The text is the first chapter of The Metamorphosis by Franz Kafka.

When the user stops turning the projector crank, however, all the flies start to move around again, and as soon as they reach the edge of the screen, they disappear and do not return. Ultimately only a single fly remains on the screen; it will never be able to escape. If the user decides to turn the crank again, the next page of Kafka's text is loaded and more and more flies come back and stick to it, starting to make the new text appear (Figure 3). If the user is patient enough, he can in this manner read the entire chapter of The Metamorphosis by Franz Kafka. In this well-known story, the protagonist Gregor Samsa is transformed into a gigantic insect overnight.

Figure 3. 

A user as she creates the text of Kafka's The Metamorphosis, which is composed of artificial flies.

Figure 3. 

A user as she creates the text of Kafka's The Metamorphosis, which is composed of artificial flies.

In an updated version of Escape we are currently adding two more features. A hidden camera tracks the user's face and, by means of the previously described process, uses the flies to recompose it on the screen. The randomly swarming flies adhere alternately to the written text and to the users' facial features, metaphorically bringing the user into the story. When the user stops turning the crank, the dead flies do not just disappear; they fall onto the bottom of the screen, pile up there, and slowly obscure the images and texts. Only if nobody interacts with the system do the flies slowly fade away, except for one fly that never escapes. In this updated version, a surrealistic impression is created in which the user becomes increasingly covered by insects and at the same time merges into Kafka's story, eliminating the distance between the author and the reader (user) of the story.

The installation also draws a connection to the famous Clark Nova scene in the Naked Lunch movie [20] (based on William S. Borroughs' novel), in which a typewriter turns into an insect; here this surrealistic idea undergoes further development and the user is transformed into a mass of insects while continuing to be fused with Kafka's text. The user here becomes the author as well as the reader; he co-produces the text and is simultaneously part of the story.

6 Summary

We have introduced three interactive art works in which users can write text that becomes alive and evolves. These installations are inspired by algorithmic text creation techniques such as were introduced by Surrealism [1, 5], Fluxus [22], Concept Art [13], and Art&Language [3]. William S. Burroughs' cut-up methods [12], The Vienna Group's ideas on Expanded Poetry [2, 33], and Peter Weibel's text-based media art works [32, 15] were also important sources of inspiration.

In “The Death of the Author” [4], Roland Barthes gave rise to the notion that the reader could replace the author and the text itself could become more prominent. The idea of assigning the author (of a text) an important role in the development of an artwork was also proposed by Umberto Eco in his The Open Work [11]. According to him, the recipient of the artwork organizes the work and, in real collaboration with the author, gives it its structure. Ryszard Kluszcyński mentions that in addition to Roland Barthes and Umberto Eco, Richard Rorty [21] and Jacques Derrida have also had an important impact on the discussion of the liberation of the author and, ultimately, of the user in interactive art. For him “Derrida's proposal can be considered as one of the methodological paradigms for the reflection of interactive art. This concept liberates the work/text from its dependence on or derivation from any communicated a priori sense, as it acknowledges the fact that the meanings are not communicated, but negotiated by the recipient” [17].

Our concept of Living Poetry can be seen as a continuation of H. C. Artmann's concept of expanded poetry [2], a term he introduced in 1954. As a member of the Vienna Group, he claimed that everyone should become a poet, and that the poetic act and the poetic result should be completely opened up. In Life Spacies, Life Writer, and Escape we take this notion one step further: The text that is produced by the users is not only completely dependent on their decisions; it also becomes alive and can evolve further. By combining genetic programming techniques, interactivity, and experimental literature, a new form of interactive artificial life poetry has been created.

References

1
Apollinaire
,
G.
(
1995
).
Calligrammes, poèmes de la paix et de la guerre 1913–1916
.
Paris
:
Éditions Gallimard
.
2
Artman
,
H. C.
(
1997
).
Eight-point-proclamation of the poetical act/Acht-Punkte Proklamation des politischen Actes
. In
P.
Weibel
(Ed.),
Die Wiener Gruppe/The Vienna group. Ein Moment der Moderne, 1954–1960, Die visuellen Arbeiten und die Aktionen/A moment of modernity 1954–1960/The visual works and the actions
(pp.
54
55
).
Vienna/New York
:
Springer Verlag
.
3
Atkinson
,
T.
,
Bainbridge
,
D.
,
Baldwin
,
M.
, &
Hurrell
,
H.
(Eds.) (
1969
).
Art-language: The journal of conceptual art
,
1
(
1
).
Coventry
:
Art & Language Press
.
4
Barthes
,
R.
(
1977
).
The death of the author
. In
Image-music-text
.
New York
:
Hill and Wang
.
5
Breton
,
A.
, &
Soupault
,
P.
(
1920
).
Les champs magnétiques
.
Paris
:
Gallimard
.
6
Burroughs
,
W. S.
(
1959
).
Naked lunch
.
New York
:
Grove Press
.
7
Burroughs
,
W. S.
(
2001
).
Electronic revolution
.
Bonn
:
Pociao's Books
.
8
Curtay
,
J.-P.
(
1985
).
Letterism and hypergraphics: The unknown avant-garde 1945–1985
.
Franklin Furnace
.
9
Dawkins
,
R.
(
1976
).
The selfish gene
.
Oxford, UK
:
Oxford University Press
.
10
Desnos
,
R.
(
1924
).
Deuil pour deuil
.
Paris
:
Aux Éditions du Sagittaire
.
11
Eco
,
U.
(
1973
).
The open work
.
Cambridge, MA
:
Harvard University Press
.
12
Fallows
,
C.
,
Genzmer
,
S.
, &
Hühnel-Benischek
,
U.
(
2012
).
Foreword
. In
Cut-ups, cut-ins, cut-outs: The art of William S. Burroughs
.
Vienna
:
Verlag für Moderne Kunst
.
13
Flynt
,
H.
(
1963
).
Essay: Concept art
. In
An anthology of chance operations
.
New York
:
La Monte Yuong & Jackson Mac Law
.
14
Godfrey-Smith
,
P.
(
1996
).
Spencer and Dewey on life and mind
. In
M. A.
Boden
(Ed.),
The philosophy of artificial life
(p.
323
).
Oxford, UK
:
Oxford Reading of Philosophy
.
15
Huck
,
B.
(
2007
).
Peter Weibel, everything is poetry
. In
Peter Weibel, Das offenen Werk, 1964–1979
.
Ostfildern
:
Hantje Cantz Verlag
.
16
Huhtamo
,
E.
, &
Parikka
,
J.
(Eds.) (
2011
).
Media archaeology—approaches, applications, and implications
(p.
119
).
Oakland
:
University of California Press
.
17
Kluszcyński
,
R.
(
2009
).
Theoretical, philosophical and cultural contexts of interactive art
. In
A.
Porczak
(Ed.),
Interactive media arts
.
Krakow
:
Department of Intermedia, Academy of Fine Arts Krakow
.
18
Kusahara
,
M.
,
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1996
).
Art as living system
.
Systems, control and information journal
,
40
(
8
),
16
23
.
19
Lumsden
,
J. J.
, &
Wilson
,
E. O.
(
1981
).
Genes, mind, and culture: The coevolutionary process
.
London
:
World Scientific
.
20
Naked lunch
(
1991
).
Directed by David Cronenberg, written by William S. Burroughs, produced by Jeremy Thomas
.
21
Rorty
,
R.
(
1967
).
The linguistic turn, recent essays in philosophical methods
.
Chicago
:
University of Chicago Press
.
22
Schmit
,
T.
(
1990
). In
A. B.
Oliva
(Ed.),
Ubi fluxus ibi motus, 1990–1962, La Biennale di Venezia
(pp.
259
260
).
Milano
:
Mazzotta
.
23
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1995
).
Phototropy—interactive installation of artificial life
. In
Beyond the global village
(pp.
134
135
).
Milano
:
La Triennale di Milano
.
24
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1997
).
A-Volve—an evolutionary artificial life environment
. In
C.
Langton
&
K.
Shimohara
(Eds.),
Artificial Life V
(pp.
167
175
).
Cambridge, MA
:
MIT Press
.
25
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1997
).
Life Spacies
. In
ICC Concept Book
(pp.
96
101
).
Tokyo
:
NTT-ICC Tokyo
.
26
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1999
).
Life Spacies: A genetic text-to-form editor on the Internet
. In
AROB'99 International Symposium on Artificial Life and Robotics Conference Proceedings
(pp.
73
77
).
Oita
:
Oita University
.
27
Sommerer
,
C.
, &
Mignonneau
,
L.
(
1999
).
VERBARIUM and LIFE SPACIES: Creating a visual language by transcoding text into form on the Internet
. In
1999 IEEE Symposium on Visual Languages
(pp.
90
95
).
Tokyo
:
IEEE
.
28
Sommerer
,
C.
, &
Mignonneau
,
L.
(
2006
).
Life Spacies II & Life Writer
. In
M.
Crutchfield
(Ed.),
All digital exhibition catalog
.
Cleveland
:
MOCA Museum of Contemporary Art
.
29
Sommerer
,
C.
, &
Mignonneau
,
L.
(
2006
).
Life Writer—creating life through the act of writing
. In
The Gen[H]ome Project exhibition catalog
.
Los Angeles
:
MAK Museum of Contemporary Art
.
30
Sommerer
,
C.
, &
Mignonneau
,
L.
(
2012
).
Escape
. In
The View Contemporary Art Space exhibition catalog
(pp.
12
13
).
Salenstein
:
The View Contemporary Art Space
.
31
Stocker
,
G.
,
Sommerer
,
C.
, &
Mignonneau
,
L.
(Eds.) (
2009
).
Christa Sommerer and Laurent Mignonneau—Interactive Art Research
.
Vienna, New York
:
Springer Verlag
.
32
Weibel
,
P.
(
1982
).
Mediendichtung
. In
O.
Breicha
(Ed.),
Protokolle
,
2
(
82
),
42
.
33
Weibel
,
P.
(Ed.)
1997
.
Die Wiener Gruppe/The Vienna Group. Ein Moment der Moderne, 1954–1960. Die visuellen Arbeiten und die Aktionen/A moment of modernity 1954–1960/The visual works and the actions
.
Vienna/New York
:
Springer Verlag
.
34
Wilson
,
E. O.
(
1978
).
On Human Nature
.
Cambridge, MA
:
Harvard University Press
.

Author notes

*

Contact author.

**

Institute for Media, Department of Interface Cultures, University of Art and Design, Kollegiumgasse 2, 4010 Linz, Austria, and Aalborg University, Denmark. E-mail: christa.sommerer@ufg

Institute for Media, Department of Interface Cultures, University of Art and Design, Kollegiumgasse 2, 4010 Linz, Austria. E-mail: laurent.migonneau@ufg.at