An alife game to teach evolution of antibiotic resistance

The emergence of antibiotic resistant bacteria is a major threat to public health and there is a constant need for education to limit dangerous practices. Here, we propose to use alife software to develop training media for the public and the physicians. On the basis of the Aevol model we have been developing for more than six years, we built a game in which players fight bacterial infections using antibiotics. In this game the bacteria can evolve resistance traits, making the infection more and more difficult to cure. The game has been tested with automatic treatment procedures, showing that it behaves correctly. It has been demonstrated during the French ”Nuit des Chercheurs” in October 2012.


Introduction
The rapid spread of antibiotic resistant bacteria is a growing threat to public health.Attempts to fight this threat include searching for new antibiotic molecules, understanding the evolutionary dynamics of resistance traits and organizing health care services to avoid dissemination of resistant bacteria.However, all specialists and authorities agree that the most important challenge to fight resistance is education.As Stuart B. Levy already argued in 2002: "Much work is needed on education of the consumer and the prescriber" Levy (2002).In many countries, large awareness campaigns were conducted, but even after ten years, (Bush, 2011) still claimed that "substantial increases in public education about bacteria and antibiotic importance are vitally important".Despite fundings from international and national agencies and the enrollment of non-profit organizations, the basic laws of antibiotic resistance are still very poorly known by the public, leading to maladapted usages that favor the spread of resistant strains.
One of the difficulties when teaching antibiotic resistance is that many factors are intertwined, leading to messages that might sound contradictory (e.g., limit antibiotic usage but systematically finish your treatment even though you feel you are cured!).To understand the resistance problem, one needs to grasp the entire evolutionary dynamics of antibiotic resistance, from the selection of resistant mutants to their spread in a bacterial population (MacCallum, 2007).In particular, one must understand the paths that can lead to higher resistance levels (Almahmoud et al., 2009;Weinreich et al., 2006).These dynamics are far from trivial and are influenced by many parameters: the mutation rate, the population size, population bottlenecks or the fitness cost of resistance.As a matter of fact, the very principles of evolution are poorly known by the public and even by physicians.
To facilitate the understanding of the microbial world and dynamics, computer games have been developed like e-Bug (Lecky, 2011;Farrell, 2011), Bait (Kerr, 2005) or the virtual infection control simulation (Pulman and Shufflebottom, 2009).These programs emphasize the population dynamics of the resistant and susceptible bacterial strains with or without different antibiotics but they do not include evolution per se and cannot account for resistance appearance and increase.Although attempts to include evolutionary algorithms in games are numerous, they mainly focus on the evolution of avatar behavior (Grand and Cliff, 1997;Stanley et al., 2005) and often take many freedoms with the real evolutionary phenomena (Bohannon, 2008).We argue that artificial life evolutionary models have a huge potential to develop educational games and that too little work has been done in this direction, except for few occasional attempts (Adami Lab. and Beacon center, 2013;Miglino et al., 2012).
Here, we present the "Aevol Serious Game", based on the Aevol model of bacterial evolution.The game idea is quite simple: the player fights a series of bacterial infections with five different antibiotics: prokarycin, microbicin, bactericin, bacillicin and aevolicin (all antibiotic names but the last one have been chosen to enable the teacher to explain what is a "microbe").The player controls which antibiotics are delivered and at what doses.The availability of five antibiotics allows to modulate the treatment strategies to fight the infections.Of course, at the beginning of the game, one can cure an infection very simply by setting one or more antibiotics to the maximum for a few generations.However, the artificial bacterial colony may evolve antibiotic resistance traits that make it more difficult to cure subsequent infections.Moreover, the acquired resistance traits directly depend on the way antibiotics have been used previously and these effects cannot be cancelled.Thus, as the game goes on, it becomes increasingly difficult to kill the bacteria and to fight infectious diseases.
We first present how the evolving bacterial population is modeled.We then illustrate the model behavior with a simulated player who delivers antibiotics in various automatic ways.Finally, we report how non-scientific players used the game during "La Nuit des Chercheurs" in October 2012.

Modeling the evolving bacterial population
The Aevol model (http://www.aevol.fr)was developed to study the evolution of genome structure and the influence of indirect selection pressures for robustness or evolvability, see Knibbe et al. (2007Knibbe et al. ( , 2008)).We present here an adapted version of the model, aimed at teaching non-scientific public about antibiotic resistance in the context of evolution.The model is organized as a generational evolutionary algorithm, each generation consisting in three main steps: genome decoding, selection and reproduction with both local mutations and chromosomal rearrangements.

From genotype to phenotype
The genotype-phenotype mapping in Aevol was inspired by the microbial transcription and translation processes.Each artificial organism owns a genome organized as a circular double-stranded binary string containing a variable number of genes separated by non-coding sequences (figure 1).A set of signaling sequences is used to identify the regions that will be transcribed into mRNAs and within those the ones that will be translated into proteins.
Transcription initiation and termination sites are directly inspired by bacterial genetics.In Aevol, we defined a promoter as a sequence close enough to a predefined consensus and a terminator as a short sequence able to form a stemloop structure.When a promoter is found, the transcription proceeds until a terminator is reached, thus producing an mRNA whose expression level directly depends on the promoter quality.
Translation occurs when a ribosome-binding site is present on an mRNA, followed by a START codon.Then, the following sequence is read three bases (one codon) at a time, until an in-frame STOP codon is found.Each codon is translated into an abstract "amino-acid" using an artificial genetic code (figure 1).
An artificial chemistry (Dittrich et al., 2001) was defined to model the protein activity and the resulting phenotype.We defined an abstract, one-dimensional space Ω = [0, 1] of possible cellular processes (therefore, in this model, a "cellular process" is a real number).Each protein can either realize or inhibit a particular set of these biological processes with a certain efficacy.For simplicity, we use piecewiselinear functions with a symmetric, triangular shape.Hence the activity of a protein can be fully characterized by the position m of the triangle on the axis, its half-width w and its height h.
A protein's primary sequence is viewed as three interlaced binary sequences that code for m, w and h values (see figure 1).Small mutations in the coding sequence will change these parameters, resulting in a modification of the protein activity.Once all proteins encoded by a given genotype have been identified and characterized, their activities are combined into a global fuzzy set representing the individual's phenotype P .The possibility distribution of P , called f P , indicates to what extent the individual can realize each abstract cellular process.
For the game, the interval Ω = [0, 1] of cellular processes is divided into 7 subintervals (figures 1 and 3).The first (Ω 0 ) and last ones (Ω 6 ) represent sets of abstract metabolic processes and they are used to determine the performance of the individual in the competition for reproduction (see below).The five intermediate zones (Ω 1 to Ω 5 ) correspond to resistance traits respectively against the five antibiotics.The proximity of these traits along the functional axis facilitates multi-drug resistance (a common feature in bacteria) through pleiotropy since a single triangle can span 2 contiguous zones.

Environment, fitness measure, death and competition
In Aevol, the environment is represented by a phenotypic target: a fuzzy set E defined on Ω whose possibility distribution f E indicates the optimal degree of possibility for each "metabolic process".To evaluate an individual, we compare its phenotype P to the optimal one E.The "gap with target" g is computed as the geometric area between these two sets on the "metabolic" subintervals (figure 1): The lower the gap, the fitter the individual.This penalizes both the under-and over-realization of each metabolic process.
Cells can die with a probability P death = 20 × g if g < 0.05, or P death = 1 if g ≥ 0.05, implying that some mutations are lethal.This mortality process, not present in the original Aevol model, was introduced in the game because antibiotics may cause severe population bottlenecks, which are known to make selection less efficient and allow mildly deleterious mutations to accumulate in the population (an effect used by microbiologists in "mutation accumulation" experiments (Korona, 2004)).Making highly deleterious mutations in the game actually lethal prevents them from propagating during the population bottlenecks.In the runs below, typically around 20% of the individuals die at each generation through this mortality process.
Cells can also die because of an antibiotic treatment.Although antibiotics can act on different bacterial subsystems (Normark and Normark, 2002), in this first version of the game, we simply simulated an effect on cell mortality de-  pending on antibiotic dosage.When a cell receives an antibiotic i with a dosage α i (0 ≤ α i ≤ 1), it has a probability P antibio.i= 0.9 × α i to die 1 except if it is protected by a resistance trait.The resistance traits to the antibiotics are deduced from the phenotypic function within the Ω 1 to Ω 5 zones (one zone per antibiotic, figure 1).The probability to survive a treatment with a given antibiotic i is: There is no direct cost to resistance.However, since resistance is generally acquired through mutation of existing genes, indirect costs are very likely to occur.If more than one antibiotics are delivered simultaneously, the same process is applied for all the antibiotics.Cells are placed on a 40 × 40 grid (Misevic et al., 2012).Each grid spot contains either a living or dead cell (figures 1 and 3).Local competition for reproduction takes place between living cells only.The population is entirely renewed at each generation.Specifically, each grid spot at generation t + 1 is filled with an offspring from one of the neighboring living cells at generation t (note that this offspring can die immediately, e.g. if it undergoes a lethal mutation).The cell that will produce the offspring is drawn according to a reproduction probability P reprod that decreases with g.If none of the 9 neighboring spots contained a living cell at generation t, the spot is left unchanged.

Reproduction, local mutations, rearrangements
During their replication, genomes can undergo both local mutations (single nucleotide substitutions, and insertions or deletions of 1 to 6 bp) and chromosomal rearrangements (duplications, deletions, translocations and inversions).Here, all local mutation occurred with probability µ mut = 1 × 10 −5 per bp per generation and all rearrangements with probability µ rear = 1 × 10 −6 per bp per generation.Not all mutational events have a phenotypic effect.For example, a mutation in a region that is not transcribed will most probably be neutral (except it it occurs in a promoter).Because Aevol allows for gene duplication and divergence, it can evolve new functions (e.g.antibiotic resistance traits) and not only modify existing ones.

Evolution of wild-type strains
Before the game itself, the software was used to evolve a "wild-type strain" in the same environment as used for the game without antibiotics.Here, we used a strain that evolved for 100,000 generations.Its genome is 80, 920 base-pair long and has 106 coding sequences present on 101 coding mRNAs.It is well adapted to its environment (g ≈ 0.006), ensuring that slightly deleterious mutations can accumulate without being immediately lethal. 1 The 0.9 factor prevents the population from being killed all at once: at maximum dosage the infection is cured in 8 to 10 generations if no resistance evolves.Note that, for pedagogical reasons, α is always displayed as percentages of the maximal dose.

Simulating infection
During the first steps of the game, driving the population to extinction is easy since the wild-type is highly susceptible to antibiotics.In such case, the game can go on with a reinfection (triggered by the player).Then the cell with the lowest g switches from dead to alive.The resistance traits are thus more difficult to maintain since they are not considered in the choice of the "resuscitated" cell, thereby mimicking an infection from an antibiotic-free environment.The resistance traits are actually eliminated unless they were carried by the individual with the best g, a situation that usually occurs when the resistance traits were useful for a very long period (long enough to enable fixation of these traits despite their mutational load).After reinfection, the germ progressively colonizes all the grid, showing circular patterns where central cells accumulate fewer mutations than peripheral ones (figure 2).This pattern reproduces a known property of population expansion: mutations accumulate on the expanding fronts (Excoffier and Ray, 2008).

Graphical outputs
The Aevol game is not designed to be used by the player alone but with a supervisor who can explain in real time the effects of the treatment and advise the player.The graphical outputs (figure 3) were designed accordingly.On the left panel, the player can see the whole population on the grid, each living cell being represented by a square, colored according to the gap to target g of the individual.This panel also displays the current antibiotic dosages that can be changed through the keyboard.It allows the player to directly perceive the effect of the antibiotic on the number of living cells.The right panel displays three different views of the current best living cell.Two of them represent its genome, with either the transcribed (top left) or translated sequences (top right).The third one (bottom) shows its phenotype (in green) including the resistance traits (for clarity ECAL -General Track ECAL 2013 Here, the fittest individual became resistant to bacillicin after less than 100 generations (but resistance is not high enough to be fixed yet).Finally, the phenotypes of all living cells are represented with blue lines, allowing an estimation of the population diversity.and esthetic reasons, Ω is displayed in polar coordinates).It also shows the environmental target (red curve), the optimal shapes for the five resistance traits (red triangles), and the phenotypes of all living cells (blue lines).This allows the player to estimate the resistance traits in both the best individual and the rest of the population and thus to detect clonal interference between the metabolic and the resistance genes.

Behavior with an automatic player
Although the Aevol game was not developed to study but to teach the emergence of resistant traits, we show here the behavior of the model with a simulated player that delivers antibiotics in various automatic ways.This study enables to verify that the behavior of the model is realistic enough to be used to educate people.It is also important for teachers that can use it to prepare himself to comment the players' actions.

Effects of antibiotic dose
For each antibiotic, the wild-type strain was given a constant antibiotic delivery α, varying between 50% and 100%.
Each time the antibiotic treatment resulted in eradicating the bacterial population, a new infection was automatically triggered and the antibiotic treatment was momentarily stopped until the population grew over 1000 living cells (the carrying capacity of the environment being 1600 individuals).For each antibiotic and dosage, we conducted the simulation until the resistance trait was considered to be fixed in the population (i.e. the resistance of the best living cell was sufficient to make it resistant to a maximum antibiotic dose).We measured both the number of successes of the antibiotic (number of infections driven to extinction) and the number of elapsed generations before the resistance criterion was met.
Figure 4 shows that antibiotics cannot drive the infections to extinction for doses below 80%.The effect of antibiotic dosage is intuitively obvious: the higher the dosage, the higher the probability to eradicate the infection.
Figure 5 shows whether the resistance criterion is met and if so, how fast, depending on the dose.Three ranges of dosages can be distinguished by comparing figures 4 and 5.For α < 60% (low dosage), the treatment fails and no resistance is acquired, most likely because the fitness cost associated with the resistance mutation is too high compared to the benefit it confers.For 60% ≥ α ≥ 80% (intermediate dosage), the treatment also fails but resistance can be acquired, which confirms the risk associated with sublethal dosages.For α ≥ 85% (high dosage), although the treatment succeeds several times, all populations eventually acquired resistance.Increasing dosage generally speeds up resistance acquisition, but the precise relationship is antibiotic-dependent.More experimental replicates with different random seeds and wild-types would be needed to test whether this effect is random, genome-dependent (i.e.dependent on the probability to find a favorable mutation in the genome) or antibiotic-dependent (i.e.dependent on the position of the resistance trait in Ω).
Figure 5: Number of generations elapsed before the resistance criterion was met, as a function of the antibiotics dosage α (dashed curve: mean).For α = 60%, only aevolicin led to resistance.For α = 65%, 3 of the 5 antibiotics led to a resistance.These first results validate the game as a realistic tool to educate people for a correct use of antibiotics.They show that, in the model, bacteria can acquire resistance traits and that the time to fixation of these traits depends on the way antibiotics are used.Moreover, the behavior of the five antibiotics is globally coherent, with slight differences that complexifies the game.

Effect of treatment timing
For each of the five antibiotics, the wild-type strain has been submitted to intermittent treatments (with a 100% dosage).The dose delivery randomly switches from treatment to nontreatment using a Poisson process with a switching probability ranging from 1/2 to 1/11 at each generation.In all cases, the mean antibiotic delivery over a long period of time is 50%.Figure 6 shows that the treatment timing has no effect on resistance acquisition.Similarly, there is no effect on the probability of success of the treatment (data not shown).
In contrast to the previous results, here the bacteria quickly acquire resistance in all cases (the maximum delay being 1318 generations) while, for the same mean dosage (50%), a constant treatment never leads to resistance acquisition.Besides, intermittent treatment leads to a mean of 4.3 defeated infections before the resistance is fixed, regardless of the frequency of switches2 , while such a low dose could not eradicate the infection with a constant treatment.This is probably an indirect effect of the fluctuation of population size when the infection is treated in an intermittent way.Indeed, a constant treatment with low doses leads to a small but constant population size (around 650 individuals for a 50% dose).In contrast, an intermittent treatment leads to huge variations in the population size (see figure 7 for an example).
Population genetics theory states that in case of variations in the population size, the effective population size is given by the harmonic mean over time of the real population size.
Here, for a mean switching probability of 1/3 and before any resistance trait acquisition, the population size varies between 1 and 1250 individuals (mean value around 350 individuals) but the harmonic mean of the population size (excluding the periods of infection when no antibiotic is delivered) is around 18 individuals!The high level of genetic drift can hence explain that resistant individuals (that generally also carry deleterious mutations) sometimes manage to reproduce, thereby favouring the spread of resistance.Moreover, when the treatment temporarily stops, the population can expand.As figure 2 shows, this leads to mutation accumulation in the population.Thus, intermittent treatments also increase the apparent mutation rate (although the spontaneous mutation rate remains constant), thus increasing the population diversity and hence favouring the emergence of resistant mutants.
Figure 6: Number of generations elapsed before the resistance criterion was met, as a function of the antibiotics treatment dynamics.The antibiotic treatment switches between on and off at each generation with a probability ranging from 1/2 to 1/11.See figure 4 or 5 for the legend.
These differences between intermittent high dose treatment and constant small dose treatment (for similar mean) shows the importance of taking into account evolutionary effects like population bottenecks and spatial expansion when dealing with antibiotic resistance.

Effect of pharmacodynamics and adherence
In the first two experiments, the antibiotic dosage was either constant or piecewise-constant over time.However, in a real situation, the antibiotic delivery depends on many physiological parameters such as means of delivery and half-life of the drug.The Aevol game can be used to teach the effects of these parameters and to introduce a patient's adherence to the prescribed treatment in the model, that is, the fraction of scheduled doses taken.Indeed, it has been shown that adherence can have a strong influence on resistance emergence, at least during antiviral therapy (Rosenbloom et al., 2012).Here, we show the behavior of an antibiotic treatment with bacillicin.The drug was given to the patient at 40% doses with one dose every 9 generations (the maximum drug concentration still being 100%).Five percent of the drug was degraded at each generation (drug half-life: 21.5 generations).
Figure 7, top panel, shows how drug concentration evolves during a treatment with perfect adherence, and the resulting effect on the bacterial load.Since the evolutionary process is included in the simulation, we can study the influence of drug dynamics on the emergence of a resistant mutant in relation to the population expansion that occurs regularly during the antibiotic delivery.The game can also be used to show the impact of patient drug-taking behavior on the dynamics of both the bacterial population and emergence of resistant strains.Figure 7, bottom panel, shows the drug concentration, bacterial population and resistance to bacillicin when the patient randomly misses one dose out of four.Here, after only two missed doses (doses 4 and 7), the infection duration (initially 68 generations) is substantially longer.This also increases the risk for resistance to emerge before the end of the treatment, which in the present case does happen.Note that the simulation presented here was specifically chosen to illustrate the emergence of resistance due to a bad adherence.However, systematic experiments have not been performed yet on this question and no conclusion can be taken at this stage.

Conclusion and future work
The Aevol game was used in real conditions during "La Nuit des Chercheurs 2012" (October 2012), one of the main public science events in France.The simulation ran for three hours using a beam projector, and around 40 people from 10 to 65 years old played with it (the game was also successfully tested with younger children but in a smaller audience).After playing with the game, visitors wandered around the place, discussed with other researchers, and came back later to see how the game evolved and possibly play again.The result was a clear success: during the whole event, the bacterial population progressively acquired resistance to the five antibiotics.By the end of the experiment, complex treatments had become mandatory to fight the infections.Visitors could witness that the misusage of antibiotics during their first attempts had strongly influenced the evolution of the population in the long term and created the subsequent difficulty for the next players.This shows the importance of a global tracing of the game, allowing the players to visualize the effects of previous treatments and detect a posteriori those having succeeded and those resulting in resistance acquisition.This experiment also showed that the game cannot be used without a supervisor to explain the behavior of the population and provide a minimal basis of evolution and genetics.
Although preliminary, the Aevol game provides a proof of concept that alife models can be used to teach people difficult but relevant scientific questions.It also opens the door to many new developments.First, although a game is not a model, it provides the opportunity to compare the simulated dynamics with what is observed in in vivo experimental evolution (Hindré et al., 2012).This will require a more precise model of antibiotic action on the cell and of antibiotic resistance traits.Second, there are ample opportunities for improvement of the game.An important direction would be to enable the player to browse the whole population (not only the best individual) and to visualize a posteriori the entire course of evolution, starting from the beginning of the antibiotic treatment.Antibiotic delivery can also be improved.
Here the antibiotic dose is directly fixed by the player (figure 3).One may ask the player for a prescription (i.e.dose and scheduling of antibiotic uptakes) and include pharmacodynamics in the model.Moreover, in the current version, antibiotics are harmless for the patient and can be used at maximum dosage without deleterious effects.Adding such effects (which are well documented for this kind of drugs) will add complexity to the game.Finally, an island model can be introduced in the game.Different islands could represent different patients as well as the global environment, thus enabling complex infection dynamics and multi-player gaming (e.g. on the Internet).

Figure 1 :
Figure 1: Graphical representation of the Aevol algorithm with antibiotics.The algorithm iterates three main steps: (1) genome decoding and evaluation, (2) selection of the best individuals and (3) reproduction with mutations and rearrangements.See the main text for details.The lightnings correspond to mutations and rearrangements undergone during reproduction.Cells on the grid are colored according to gap with target, red cells being those with lowest g and blue cells being the higher ones.The dead cells are the black cells.The violet zone on the cellular process axis (Ω 1 to Ω 5 ) correspond to resistance traits (one zone per antibiotic).

Figure 2 :
Figure 2: Population expansion during an infection process (here at t = 1, 5, 10, 15 and 20 generations after the reinfection).Cells are colored according to fitness, red cells being those with highest fitness (lowest g) and blue cells being the worst ones.The circular pattern is clearest at t = 15 and t = 20 where many blue cells are observed n in the periphery of the infection, while the central zone contains mainly orange and red ones.

Figure 3 :
Figure 3: Screen capture of the Aevol game.The left panel presents bacteria population with a color code indicating their fitness (red cells being the fittest and blue the least fit).It also presents the current concentration of the five antibiotics (here, bacillicin is given at a 80% dose).The right panel presents the fittest individual's transcriptome (top left), genome (top right) and phenotype (bottom).The phenotype is represented in polar coordinates (green surface) together with the target function (red curve) and the five resistance sub-functions (red triangles).Here, the fittest individual became resistant to bacillicin after less than 100 generations (but resistance is not high enough to be fixed yet).Finally, the phenotypes of all living cells are represented with blue lines, allowing an estimation of the population diversity.

Figure 4 :
Figure 4: Number of infections successfully cured, as a function of the antibiotics dosage α (dashed curve: mean).

Figure 7 :
Figure 7: Effect of pharmacodynamic (top panel) and adherence (bottom panel) on bacterial load and antibiotics resistance (see main text for details).