Abstract

This article documents Harold Cohen’s last phase of creativity from 2010 until his death in 2016, a period that witnessed an accelerated coevolution of Cohen’s relationship with the artificial intelligence program AARON on the one hand and his technological and artistic innovations on the other, culminating in a new art form featuring the void. Implications for the human and machine interface are discussed.

AARON was an artificial intelligence program created by Harold Cohen (1928–2016), artist and pioneer in computer-generated art, more than four decades ago. As one of the world’s longest-running AI systems in daily use, AARON was meant to be god-like: “Why do I want AARON to be autonomous? To be god. To leave something behind that never existed before,” Cohen emailed me (23 February 2005). In another email to me, Cohen made reference to a joke: “I once made a joke about being the first artist in history to have a posthumous exhibition of new work” (11 February 2005).

But toward the end of 2011, Cohen seemed to have changed his mind. For instance, during the last exhibition of his lifetime [1], Cohen (HC) had the following exchange in an interview with Sheldon Brown (SB) at Calit2 [2]:

SB: Not to be morbid about it, but at a certain point, AARON might be continuing past his biological partner here. … So AARON could conceivably go on producing work for the future.

HC: Well, AARON could go on producing work indefinitely. … To be realistic, I rather suspect that AARON will end when I end, because why would anybody want to take up my other half? People should build up their own other selves.

Shortly after Cohen’s passing, AARON failed to recover from a thunderstorm-related power outage [3]. Of course there are multiple versions and replicas of AARON in various museums, but according to Thomas Machnik, Cohen’s assistant, the one operating in the capacity of “a development system running in the studio and someone is actively engaged with AARON in the same way or in a similar [way] as Harold was” is no more [4].

Science fiction as this may sound, AARON’s demise is arguably the logical conclusion of a machine that has become “the other self” [5] of a mortal. This article documents the metamorphosis of Cohen’s relationship with AARON in the last six years of his life between 2010 and 2016. Email correspondences with Cohen are reproduced in italics.

A Story of Collaboration

Cohen did not simply develop his program, he frequently talked and thought about his relationship with it; sustained reflections on the relationship stimulated further development of his relationship with AARON. It is at this juncture that my collaboration with Cohen played a catalytic role.

In 2004 I sought help from Cohen to pursue a philosophical investigation on creativity. In his usual iconoclastic fashion, however, Cohen rejected all received notions of creativity without offering any working definition of his own. Our online discourse folded within a few months, as it led us nowhere. After a hiatus of five years, I resumed my research project with Cohen. This time I decided to focus on theories that he was not familiar with in order to avoid his acid iconoclasm, and also to keep him engaged—given that Cohen was insatiably curious and always keen on learning something new. More specifically, I invited him to a dialogue on various models of the mind as explanations of his creativity [6]. Cohen accepted the invitation, saying that “important, difficult questions force me to think through what I’m doing in a way that’s hard to achieve in isolation” (20 August 2010). Our collaboration resulted in two publications of mine: an essay on his emerging new art form [7] and an academic paper on creativity [8].

One of the earliest email exchanges I (LS) had with Cohen (HC) went as follows (27 December 2004):

LS: What’s your relationship with AARON?

HC: well, I’m not sure there’s a term for it. I doubt Bush would let us get married, though if it was called Annabelle he probably wouldn’t notice.

LS: Would it be appropriate for me to wish him a happy new year?

HC: him?

The various philosophical theories I introduced to Cohen in subsequent years were attempts to help him better articulate this relationship that he did not have a ready-made term for.

Technical Innovations

In mid-2010, Cohen picked up the paintbrush again after leaving the coloring job to AARON for over a decade. One of the factors to which Cohen attributed to this major innovation was the need for dialogue with AARON. Indeed, dialogue was the key to creativity, he wrote:

Creativity … lay in neither the programmer alone nor in the program alone, but in the dialog between program and programmer; a dialog resting upon the special and peculiarly intimate relationship that had grown up between us over the years [9].

As a programmer, Cohen’s goal had always been program autonomy [10]. But in 2009, when a newly developed algorithm brought things very close to that goal—AARON could now handle color, forms and composition all on its own—Cohen suffered something of a crisis. He wrote [11]: “I felt that my dialog with the program, the very root of our creativity, had been abruptly terminated.” This crisis in relationship “led to a resumption of the dialog—having AARON provide an ‘underpainting’ to which I could then provide qualities the program couldn’t provide.” (9 March 2011)

From Cybernetics to Semiotics

After Cohen picked up the paintbrush again in mid-2010, cybernetic hierarchy gave way to a more equalitarian relationship with AARON. The cybernetic hierarchy was evident in an earlier statement of Cohen: “I can change the rules. AARON can’t. It IS the rules. Which defines it as the most remarkable artist’s assistant in history, not as an artist” (11 February 2005). But in the 2011 interview with Brown, Cohen stated that coloring things by hand “brought me back into dialog with the program, but in a somewhat different mode than the one I’d lost. I no longer think very much about the program’s autonomy. I think of the program as a collaborator rather than a talented assistant” [12].

Of all the philosophies I introduced to Cohen [13], the semiotics of Charles Sanders Peirce [14] deserves special mention. Peirce claims that thinking entails a self-to-self dialogue that has the triadic structure of self-other-self [15]. This triadic formulation of the internal dialogue suggests that the self needs to loop through an “other” in order to come home to itself, a process known as self-integration. The “other” here, I suggested, was AARON. In his interview with Brown, Cohen credited this theory of self-integration for the title he used for the Calit2 exhibit—Collaboration with My Other Self:

I’ve spent a large part of the last year in correspondence with a psychologist, Louise Sundararajan, who has been building a case that my involvement with the program has had less to do with productivity than it has had to do with creating an “other” that I could discourse with. … I think she’s right [16].

Cohen’s notion of image as stand-for-ness [17] also resonated well with Peircean semiotics [18], which posits that the mind functions as interpretant (that which interprets) of the sign, which refers to (stands for) something else [19]. The semiotic relationship between the mind (as interpretant) and its sign would support the following division of labor: AARON = Sign, which presents an image; Cohen’s mind = interpretant, which does the reading/interpreting. To wit, reading AARON’s intent had become a preoccupation for Cohen, to remain till his death [20], as evident in Cohen’s following statements:

I sometimes feel as though AARON is presenting me with a world behind a gauzy screen, and that my job is to remove the screen and show what’s really there. … The only times I “edit” is when AARON makes images in which some passages are difficult to read and I need to clarify them so that I know how to proceed. … I don’t add my own forms. Nothing purist here, AARON’s handwriting is too difficult to emulate [21].

During our philosophical discussions, Cohen was wont to remind me that “For me, ideas are the keys that open doors, and it’s what I can do on the far side of the door that’s central” (30 December 2010). Indeed, Cohen went very far with some of the ideas I introduced. A case in point is his new art form inspired by the notion of the void in Chinese art.

The Void

When Cohen picked up the paintbrush again in 2010, he was trying to fix yet another complication: The new algorithm developed in 2009 produced simpler and flatter images that exposed the gap between meaning and intention in the untouched-by-hand look of AARON’s prints. Cohen explained to me that intentionality in conventional art is usually associated with the manipulation of physical materials by the painter. When the image was complex, this gap between meaning and intentionality in computer art did not attract attention. Now it did when the image became simpler and flatter. To add intentionality, Cohen painted over the background of AARON’s drawings/prints (Fig. 1).

Harold Cohen, untitled, paint over print on canvas, December 2010, NO.101215.045.

Fig. 1.
Harold Cohen, untitled, paint over print on canvas, December 2010, NO.101215.045.
Fig. 1.
Harold Cohen, untitled, paint over print on canvas, December 2010, NO.101215.045.

I asked: But why cover up the gap between meaning and intention? I told Cohen about the void in classical Chinese paintings [22], in which the void “consists of a discontinuous presence—presence perforated by absence” [23].

In the 2011 Calit2 exhibit [24], Cohen left the unpainted canvas blank in three of his works. He subsequently referred to his paintings that feature the void as “new work.” Evidenced by unpainted canvas (see Color Plate D), the void made its appearance in some paintings toward the end of 2011 through 2012. From 2013 until his death in 2016, all of Cohen’s paintings feature the void [25].

Cohen wrote to me about his “new work” [26]:

I’ve been meaning to write for some weeks, to tell you about new work, evidently strongly influenced by your observations on the void in chinese [sic] art and our subsequent discourse. The new paintings distinguish between the void—the unpainted canvas—and the backgrounds of events that occur in the void. The events are the marks, lines, provided by AARON. The backgrounds are the areas of color which I use to determine for the viewer how the marks should be grouped—a bit like naming constellations in a random distribution of stars. Don’t have any pictures yet, but I’ll send you some in a few days. (4 April 2013)

And again: “The newest work rests heavily on your insights, apropos Chinese landscape, about the difference between background and void. For which I remain deeply grateful” (11 August 2013).

The Final Innovation

In 2015, due to increasing difficulties in mobility, Cohen shifted to the finger-painting mode. This technological innovation led to another novel approach—he moved into “AARON’s space” to do art. Previously, AARON’s drawings were printed on canvas—which is in the physical space of the programmer—for Cohen to make his selections for coloring. With finger painting, selection took place in the virtual space of the program itself such that the programmer and program were no longer domains apart.

Cohen first mentioned finger painting in the following email:

My new work* is proving difficult, and getting around even more so.

*I’m using a 7ʹ touch sensitive screen, going into AARON’s space (as it were) to color its drawings. Looks good on the screen, but trying to print what I see there is very hard. (2 March 2015)

He explained more fully his difficulties in his online paper on finger painting [27]: “Some of the colors I can display on my big screen are outside the gamut of the printer; it simply can’t produce them from the six ‘primaries’ it uses.” These difficulties resulted in a shift of focus.

On 23 January 2016, I visited his studio and took the following notes on finger painting, which were subsequently read and approved by Cohen on 4 February 2016:

Relationship of colors becomes more central than before.

Finger painting gets past the point where skill is important.

Conceptual problems become central. The biggest conceptual problem is: What kind of things can be colored? What color relationship goes with the drawing? How color fits in the drawing naturally?

Most of the drawings [by the program] are not colored, because I can’t figure out how to color them.

I am only allowed to color what the program tells me is there, leaving the rest to the void.

Before, computer makes a drawing, takes the output to the domain of physical reality, and physical color is my space, a domain apart from the drawing. Now the whole thing is done in the program domain. This is the most radical work I have done. Color becomes part of the program domain; everything is done in the program domain, resulting in a more intimate relationship between program and programmer.

After reading the notes I took, Cohen added: “The next phase will be more interactive with the program that I have been in daily contact with for 50 years” (4 February 2016). He repeated the same idea in his paper on finger painting, published online four days later:

It could give rise to a new level of intimacy between my collaborator and myself, our roles freed of the restrictions of drawing on its part and coloring on mine. Or it could give rise to something I can’t even conceive of at this moment. Have to wait (no, work) and see [28].

A Culminating Vision

For Cohen, his new paintings seemed to have an ontological dimension. Consider his space-making in the scenario of “naming the constellations” that he wrote earlier:

The new paintings distinguish between the void—the unpainted canvas—and the backgrounds of events that occur in the void. The events are the marks, lines, provided by AARON. The backgrounds are the areas of color which I use to determine for the viewer how the marks should be grouped—a bit like naming constellations in a random distribution of stars. (4 April 2013)

Here Cohen differentiated between two types of space (see Color Plate D):

Space 1: the colored background is a kind of narrative space, in which Cohen interpreted with color the marks/ lines of AARON. This interpreting task was alluded to as naming the constellations of stars.

Space 2: The unpainted canvas is the void, in which are consigned all the uninterpreted marks/lines of AARON that Cohen could not decipher and hence did not color.

While the narrative space is the mind’s playground for naming/interpreting, the void beyond naming is the wide, impassive universe. This point is driven home by Cohen’s next allusion to naming the constellations.

On my visit of 23 January 2016, Cohen and I left the dinner guests temporarily to go to his studio to take a look at his finger-painting machine. When we returned to the dinner table, the conversation was on the discovery of new planets. Joining at the end of the conversation, Cohen said, “A sprinkle of dust in space, you use a telescope that renders light years away a very short distance—just you and the dust you discovered.” This enigmatic statement kept turning in my mind, so I wrote to him on 4 February 2016 for clarification. In his response on 6 February 2016, Cohen elaborated further on this magnificent imagery:

Astronomical story in full: you scatter dust in some vast area in space, maybe one or two grains per cubic kilometre, Then you step back several light years, build a strong-enough telescope, and what you see is not dust but the Horse’s Head nebula or whatever. By analogy, the program generates clusters (like clusters of dust) and it’s my job to find the horse’s head.

While our existence is no more significant than specks of dust in the wide expanse of the universe, humans can nevertheless find consolation of intimacy in the cozy narrative space that is cocreated by the mind and its sign: “You use a telescope that renders light years away a very short distance—just you and the dust you discovered” (23 January 2016). Indeed, “just you and the dust you discovered” sums up the joy and essence of creativity in both arts and the sciences. A more gratifying homecoming in the universe cannot be found. In hindsight, might this imagery not be Cohen’s premonition of his final destination?

Two months later, Cohen passed away, on 27 April 2016. His partner Hiromi Ito wrote to me the next day: “We put his bed in the studio, so he died surrounded by the paintings” (Fig. 2).

“A little altar for Harold in his studio … where he died.” (Hiromi Ito, personal communication, 28 April 2016) [31].

Fig. 2.
“A little altar for Harold in his studio … where he died.” (Hiromi Ito, personal communication, 28 April 2016) [31].
Fig. 2.
“A little altar for Harold in his studio … where he died.” (Hiromi Ito, personal communication, 28 April 2016) [31].

Epilogue

What I have documented here is, first, a contrarian discourse on the human and machine interface. Instead of the usual focus of information technology such as reading-writing-editing codes, rich and complex modalities of relationship—such as collaboration, intimacy and interpretation of intent—loomed large.

Second, a clear pattern emerged from the amorphous interface between human and machine: Relationship between human and machine is shown to have both epistemological and ontological consequences—the former pertains to technological innovations; the latter, a new art form that at the same time entails a new way of being in the universe. Furthermore, all three developments—Cohen’s relationship with AARON, technological innovations, and new art form—were intimately connected in an interdigitation of coevolution.

Third, Cohen’s reflections on his relationship with AARON have brought us to the crossroads of the information age: Whether we continue to nonchalantly crack and hack codes, from artificial intelligence to biology—or pause to ponder how the mind and its sign are coconstitutive of each other [29]—will have far reaching ramifications for our future as a symbolic species [30].

References and Notes

1
The Gallery @ Calit2
,
Collaborations with My Other Self: Harold Cohen, October 27–December 9
, Exhibition Catalog No. 13 (La Jolla, CA: gallery@calit2,
2011
): http://gallery.calit2.net/portal/catalog/2011-2012-Collaborations-Self.pdf.
2
Sheldon
Brown
,
“Interview with the Artist: Harold Cohen,”
in [1] pp.
18
41
.
3
For details see
Louise
Sundararajan
,
“Harold Cohen (1928–2016): The Last Six Years of a Creative Life,”
Setu Bilingual Journal
4
(August
2019
): www.setumag.com/2019/08/harold-cohen-1928-2016-last-six-years.html. DOI: https://doi.org/10.1162/leon_a_01906
4
Thomas
Machnik
, personal communication (15 January
2018
).
5
From the titular words in [1].
6
Sundararajan [3].
7
Louise
Sundararajan
,
“Essay/Recent Paintings of Harold Cohen: A Mind’s Journey to Itself,”
in [1] pp.
6
15
.
8
Louise
Sundararajan
,
“Mind, Machine, and Creativity: An Artist’s Perspective,”
The Journal of Creative Behavior
48
,
No. 2
,
136
151
(
2013
). PMCID: PMC4265294
9
Harold
Cohen
,
“Driving the Creative Machine,”
Orcas Center, Crossroads Lecture Series
,
Orcas, WA
(
2010
), p.
9
: www.aaronshome.com/aaron/publications/orcastalk2s.pdf (accessed 27 July 2019).
10
Harold
Cohen
,
“The Art of Self-Assembly: The Self-Assembly of Art,”
Dagstuhl Seminar on Computational Creativity (July
2009
): www.aaronshome.com/aaron/publications/index.html (accessed 27 July 2019).
11
Cohen [9] p. 12.
12
Brown [2] p. 38.
13
Sundararajan [3].
14
Terrence W.
Deacon
,
The Symbolic Species: The Co-evolution of Language and the Brain
(
New York
:
Norton
,
1997
).
15
Norbert
Wiley
,
The Semiotic Self
(
Chicago
:
University of Chicago Press
,
1994
).
16
Brown [2] p. 36.
17
Harold
Cohen
,
“What Is an Image?”
(
1979
): www.aaronshome.com/aaron/publications/whatisanimage.pdf (accessed 27 July 2019).
18
For details see Sundararajan [7].
19
Deacon [14].
20
Sundararajan [3].
21
Sam
Cornish
,
“Harold Cohen,”
in
Harold Cohen: New Paintings
, exhibition catalog (
London
:
Bernard Jacobson Gallery
, 8 April–7 May
2011
) pp.
3
10
, p. 9.
22
George
Rowley
,
Principles of Chinese Painting
(
Princeton, NJ
:
Princeton Univ. Press
,
1959
).
23
Sundararajan [8] p. 8.
24
Ref. [1].
25
Some such paintings can be found in “Wall 4 AARON as Collaborator” at www.aaronshome.com.
26
Sundararajan [3].
27
Harold
Cohen
,
“Fingerpainting for the 21st Century”
(
2016
): www.aaronshome.com/aaron/publications/8Feb2016Fingerpainting-for-the-21st-Century-with-Figures.pdf (accessed 27 July 2019) p. 3.
28
Cohen [27] p. 3.
29
The behavioral, albeit not yet the ontological, aspect of the humanmachine feedback loop is gaining attention in science; see
I.
Rahwan
et al
,
“Machine Behaviour,”
Nature
568
(
2019
) pp.
477
486
.
30
Deacon [14].
31
Hiromi
Ito
, personal communication (28 April 2016).

Color Plate D: Harold Cohen and Aaron: Collaborations in the Last Six Years (2010–2016) of a Creative Life

Harold Cohen, First Sighting, oil over pigment ink on canvas, 48 × 86.5 in, 2012, detail showing the void (unpainted canvas) and the backgrounds (areas of color) of events (AARON’s digital print of marks and lines), with some “events” remaining indecipherable (marks on unpainted canvas).

Harold Cohen, First Sighting, oil over pigment ink on canvas, 48 × 86.5 in, 2012, detail showing the void (unpainted canvas) and the backgrounds (areas of color) of events (AARON’s digital print of marks and lines), with some “events” remaining indecipherable (marks on unpainted canvas).
Harold Cohen, First Sighting, oil over pigment ink on canvas, 48 × 86.5 in, 2012, detail showing the void (unpainted canvas) and the backgrounds (areas of color) of events (AARON’s digital print of marks and lines), with some “events” remaining indecipherable (marks on unpainted canvas).
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. For a full description of the license, please visit https://creativecommons.org/licenses/by/4.0/legalcode.