When Congestion Can Be Useful: Modelling Driver Diversion Behaviour in Road Traffic Networks

The ability to accurately predict driver route choices is an important part of traffic assignment, the process of forecasting traffic flows on roads across a region. Many assignment methods only consider the presence of recurrent forms of congestion , such as during rush hour periods, and fail to incorporate non-recurrent congestion effects caused by irregular events such as road traffic accidents. This paper proposes an agent based driver route choice model which includes driver reactions to the presence of non-recurrent congestion, supposing that drivers learn relationships between congestion locations and adjust their expectation of network travel times en-route, potentially choosing to divert. By simulating an example network with mixed populations consisting of agents capable of diverting and not, the result is found that initially increasing the proportion of diverting agents from zero is beneficial to the system as might be expected, reducing the number of vehicles navigating the incident affected area, but beyond a tipping point agents can no longer perceive the presence of congestion prior to diverting and network performance decreases. The model not only demonstrates the conflict between agents adopting travel time reducing behaviour and its impact on system performance, but it also highlights the importance of modelling driver knowledge appropriately to reproduce plausible phenomena in simulation.


Introduction
It is important to be able to predict the impacts on road traffic flows of potential interventions such as new road layouts or signal timings, construction of new homes and retail zones, or population growth over time.To assist with this prediction, which can suggest severity of congestion or levels of pollution, traffic assignment is the methodology which attempts to answer the question 'what is the likelihood that drivers will use this route to travel from an origin point to a destination point?'.After establishing an expected level of vehicular demand between each origin and destination pair, the congestion causing feedback effect of many driver's routing decisions is included in the assignment process.
In order to predict road traffic flows adequately a thorough understanding of how drivers make routing decisions is required.This can then be translated in to models of driver route choice behaviour which describe how relative measures of route attractiveness inform final routing decisions, typically including key preference factors such as travel time, distance travelled or toll costs (Bekhor et al., 2006).Capturing the important motivations and other elements of driver choice, such as reactions toward the presence of congestion, is crucial in order to produce accurate models and subsequent traffic flow predictions.
Two types of congestion are generally considered to exist in road traffic networks.The first is congestion caused by vehicular demand exceeding capacity (in the form of maximum traffic flow) on a regular basis, such as during rush hour and other peak periods, known as 'recurrent' congestion, and is present (in some form) in most assignment methods.The second 'non-recurrent' form of congestion is caused by unexpected capacity reducing incidents such as accidents, road maintenance works or local surges in demand reducing excess capacity, and rarely features in practical traffic assignment applications.This work examines the extent to which the possible presence of non-recurrent congestion could influence traffic flows and accordingly the importance of considering it in traffic assignment.
Traditional assignment approaches adopt an aggregate approach to analysis, splitting overall demand according to the relative attractiveness or utility of whole route options with 'usual' road network properties and so ignoring any possibility of non-recurrent congestion.The recent rise of agent based models of route choice instead allow modelled drivers to make individual decisions based on their unique experience of the transportation network, allowing for learning strategies and the evolution of choice over time to be explored (Nagel and Marchal, 2006).This paper examines a mechanism whereby agents, each representing single drivers, are able to react to new information regarding the presence of non-recurrent congestion en-route.If this ability provides agents with a different impression of road network characteristics causing them to act in a different but plausible manner to what would usually be predicted, then it can be considered an important aspect of driver choice which should be included in mainstream traffic assignment models.
The paper is structured as follows: firstly the current state of, and assumptions behind, traffic assignment methods and driver route choice models are provided.Then the underlying traffic interaction model used here is described followed by the novel behaviour model presented in this work.The behaviour model is implemented on an simple example road network and the outcomes and insight provided by this model are finally discussed.

Modelling traffic flows at user equilibrium
Traffic flows between each origin and destination pair under consideration are generally assumed in predictions to be in a configuration known as 'user equilibrium', originally defined by Wardrop (1952).Here the generalised travel costs (disutility) to travellers of each used route is minimal and equal to any other, thus providing zero incentive for a driver to use a different route.Extensions to the definition of user equilibrium have been made since including 'Dynamic User Equilibrium', where travel times are equal and minimal in each departure time period (Peeta and Ziliaskopoulos, 2001), and 'Stochastic User Equilibrium', where a portion of travellers choose sub-optimal routes representing driver perception errors (Bekhor et al., 2006).
An aspect of route flow dynamics which has received relatively little attention from modelling research is the process by which transportation networks move to an equilibrium flow set from any initial flow configuration, such as when a network change occurs and route flows pass through disequilibrium states.Some models have been developed, such as a class of 'day to day' route choice models derived from the work of Horowitz (1984) which assume that daily traffic flow configuration is a function of flows on the previous day(s) which can tend towards an asymptotic equilibrium flow distribution as the number of simulated days tends to infinity.These models have been extended to consider the basins of attraction of multiple flow equilibria (Bie and Lo, 2010) and long term system behaviour (Smith et al., 2013).Crucially for this work, day to day route choice models have also been developed incorporating multi agent systems where each agent, representing a single driver, holds a unique, evolving impression of network attributes which guide future route choice decisions (Liu and Huang, 2007;Tiang et al., 2010).These methods have been found to give identical predicted traffic flow configurations to other assignment approaches (Snowdon, 2013).

Driver reaction to non-recurrent congestion
Where non-recurrent congestion is considered in traffic assignment, it is generally represented as a capacity reduction occurring along a section of road which increases the travel time for a constant level of vehicular demand (Gao et al., 2008).To model drivers diverting to avoid congestion, Unnikrishnan and Waller (2009) introduce the concept of 'recourse' where drivers are modelled receiving up to date net-work state information as they traverse the network, which is a technique also adopted by other traffic simulation tools (Sykes, 2010).In reality however many drivers will not be privy to such information and must rely on previous experience to guide network travel time expectations.
Modelling driver re-routing and its influence on route choice has previously been attempted in network representations which impose a spatial congestion structure externally to the model (Gao et al., 2008).In reality the location of queues move within-day as a result of driver routing decisions (Long et al., 2008), for example the consequence of drivers diverting to avoid non-recurrent congestion may introduce more non-recurrent congestion elsewhere in the network and clear the area initially afflicted.It is important for accurately modelling traffic flows that the outcome of drivers adopting diversion behaviour, and any impacts of potential diverting opportunities on initial route choices, is understood.
This work incorporates two main features in the traffic assignment process in order to capture both the effect of incomplete network information and of emergent spatial congestion structure as has been described.The first is the use of a cell transmission model, which models vehicle movements along road links and the build up of queues in parts of the network, and the second is a novel application of a coupled hidden markov model representation of driver knowledge given to agents traversing the simulated network.

A cell transmission model of road traffic interactions and congestion propagation
In this work vehicle movements and interactions are captured using a Cell Transmission Model (CTM).The CTM used here is a reimplementation of previous works (Long et al., 2008(Long et al., , 2011)), extended from Daganzo's original CTM (Daganzo, 1994).In this formulation a network G = (N, A, C) features a set of nodes N connected by a set of links A and includes a set of centroids C which are each attached to a single node n ∈ N and can represent origins and/ or destinations.In the CTM each link is discretized in to homogeneous cells and time is partitioned in to intervals such that the cell length is equal to the distance travelled by free-flow traffic in one time interval δ.For example, here δ = 5s and free flow vehicle speed (v) is 15m/s so cell length is 75m.A time variable, t, advances by 5s at each simulation time step.Each cell also has a fixed capacity of vehicles which can reside inside it at each time step.The units traversing the cell transmission model are implemented as agents which each hold their own driver behaviour model.
At each time step, as well as advancing agents on to their next cells and links, a number of agents drawn from the population may be entered in to the simulation at centroids specified by the agents.Should an agent be unable to join the link connected to n they will join a centroid waiting queue and attempt to enter the network at each time step onwards.Each simulated day agents are injected in to the simulation in the same order but the number of agents entering at each time step is randomly chosen (up to a limit of 5 per origin).
Unlike in previous implementations of the CTM, in this system each modelled agent holds a unique identity and attached behaviour model.Prior to arriving at a node, agents are interrogated for their turning intention and next link choice.The simulated day only comes to an end when all vehicles have left the network.
Equilibrium is hard to define in a stochastic traffic system since a degree of route choice 'noise' may ensure that flows never converge to a single stable set.When describing deterministic route choice systems, Bie and Lo (2010) define equilibrium as a route flow configuration which re-generates itself indefinitely.Here a heuristic method is adopted that the system is left to evolve for a period longer than is required to visually reach an equilibrium flow distribution.

Modelling a driver agent's network knowledge using a coupled hidden markov model
Previous work has reported positively on describing a period of observed road link conditions as belonging to a set of states including 'free flow', 'mildly congested' and 'highly congested' travel time distributions (Kwon and Murphy, 2000;He et al., 2006).In reality link states exhibit an often predictable spatial structure around the road network as queues propagate from a single starting point such as a busy junction or incident location and affect other regions.These correlations have been used as a basis for developing reliable travel time predicting algorithms (Min and Wynter, 2011).
Choosing the right number of states to represent road link performance and capture travel time variation sufficiently is not a trivial task.The appropriate set of states may vary by location and time of day under consideration.In their work, Kwon and Murphy (2000) use two states, free flow and congested, but this work uses three states to emphasise the distinction between minor and severe congestion.Drivers may experience any state regardless of its cause, but non-recurrent congestion may result in an unusual state set being experienced compared to under 'usual' recurrent congestion.States as used here only depend upon travel times alone: The driver behaviour model proposed here focusses on allowing driver agents to learn link state structures through the use of a Coupled Hidden Markov Model (CHMM).By learning link state structures drivers can re-evaluate the expectation of congestion elsewhere in the network based on that day's experience.Traffic systems have previously been represented as CHMMs for predictive purposes (Kwon and Murphy, 2000;Herring et al., 2010) but have yet to be explored as the basis of an agent based driver knowledge representation.

Model overview
A single hidden markov model (HMM) considers time as discrete and at each step can be in one of a number of unobservable (hidden) states, S. At each time step the model emits one of a number of externally observable symbols, V , with a given probability in each internal state, B = {b j (k)} j ∈ S, k ∈ V .A transition probability distribution describes the state which the model will be in at the next step given the current state, C = {c ij } i, j ∈ S. The probability of the model being in any initial state is given by a distribution, This work uses an extension to HMMs proposed by Zhong and Ghosh (2001) which allows for the consideration a network of coupled hidden markov models.The state of each HMM in the next time step is influenced not only by itself but also by the (hidden) states of other connected HMMs.Here the CHMM is achieved by extending both the system transition matrix to describe the influencing effect of model a on model a, C = {c (a ,a) ij }, initial probability distributions, π = {π a i } , and also introducing a coupling matrix Θ = {θ a ,a } which defines how the set of HMMs are connected.

Implementation details
Each driver agent is equipped with a single CHMM as described, with each HMM representing a single road link in the network.The goal of the model is to determine an expectation of travel time for a link a, ϕ a , both at the start of each simulated day and en-route once the state of other links elsewhere in the network has been observed.For this implementation the CHMM belonging to each driver agent is considered fully connected (Θ = J |L| ) although in their work Kwon and Murphy (2000) only connect HMMs considered connected in the road network.The relationships between nearby link states are not fully understood yet limiting the interconnectedness of the CHMM saves on computations and memory use required by the simulation.
Driver agents also store an associated expected travel time (in simulation steps, i.e. multiples of 5 seconds) for each state of each link, Γ = {γ a j }, j ∈ S, a ∈ A, which is adjusted by daily experience using the exponentially weighted moving average model where r a is the experienced travel time on link a and α is an externally set learning parameter (0.01 in this implementation): Once a driver agent has traversed a link a they observe the link's state exactly (i.e. each state is tied to only one observation, B = I |S| ) so only the state in question's expected travel time is updated.The initial state distribution, π a , is then updated directly as the average experienced proportion of occasions that the link was in state s.At the start of the simulation the probability of any link being in any state is equal and each state's expected travel time, γ a j =free flow travel time on link.
The expectation of travel time on network links at the start of any simulated day can then be simply found as ϕ = Γπ.
This model would be sufficient to ignore within day effects and determine an equilibrium set of route flows based on initial route choices alone.However this work sets out to incorporate the possibility of agents changing the expectation of a link's state en-route based on information relating to the state of other links obtained on the trip, within day.
The system transition matrix is updated at the end of each simulated trip, C = {c (a ,a) ij }, as the experienced proportion of occasions that link relationships occurred.That is, for the HMM associated with link a, the state probability distribution describes the probability of link a being in state j given that link a was in state i.
As an agent travels through the network, experience is accumulated in two sets which are re-initialised as empty at the start of each day; L = {a}, which stores the identifiers of experienced links, and O = {o a } which stores the corresponding set of observed link states.This information is used to update the expected probability that link a will be in state i, P (o a i ) as the average expected state of link a given its relationships with each of the traversed links in the set L: Thus the single expected travel time on link a can be reevaluated en-route as: To inform the choice model, the final utility of a route is given as ϕ • −0.1 (negative since travel time is a disutility).The discrete choice model used to determine the probability of an agent choosing a route is a path sized logit model (with calibrated parameter = 0.1) which takes in to account the overlap between route options as well as utility (Bekhor et al., 2006).
In this application of a CHMM the re-evaluation of HMM states occurs when the agent receives any new information about current link states, such as when leaving a link.The result of this is that the CHMM does not operate in fixed time steps which would create unnecessary computations or information not being considered in network re-evaluations.

A simulation of driver reaction to network incidents
As an illustration of the effects of incorporating the general CHMM agent knowledge representation, the proposed day to day traffic assignment method is performed on the network shown in figure 1 featuring a fixed vehicular demand of 7500 vehicles on each simulated day.The network consists of 13 links, 13 nodes and one origin to destination pair O0-D0.Each link is divided in to 10 cells with a capacity of 10 vehicles with two exceptions: link 6, which is divided in to 16 cells, and link 5, which is divided in to 3 cells.The probability of all network cells being affected by incidents on each day is 0% except cell 9 on link 6, whose capacity drops from 10 to 3 when an incident occurs in the same manner as is shown in figure 2 which illustrates how congestion forms in the CTM.On any simulated day there is a 30% chance that link 9 cell 6 is perturbed in this way.All other constants are set as in the model definition.
Although the CHMM knowledge model presented here suggests that all agents should re-evaluate their decision enroute, in reality not all drivers will be able to do so either due to personal reluctance or lack of knowledge regarding the area and alternative route options.Accordingly, each agent holds the CHMM behaviour model as described except for the following key differences: switchers will re-evaluate their route choices en-route and may choose to divert (although it is important to remember that not all will, the discrete choice model only provides a likelihood of choosing a route rather than a decision) and stayers who will not reevaluate any system perceptions en-route.As described in the literature overview, the vast majority of transportation forecasting models do not consider that agents will process information en-route, thus consist of a population of 100% stayers (0% switchers) who might be armed with perfect congestion information pre-trip.Figure 3 shows the day to day initial route choices of 7500 agents traversing the network in figure 1 over a period of 30 days.The population consisting of 100% switchers is capable of moving to an equilibrium which features fewer agents initially choosing the diversion route.This can be considered modelling the ability for switchers to 'take a chance' on the preferable major route and stayers being forced to consider average network performance in their routing decisions so more often initially choosing the diversion route.
As an analysis of within-day system behaviour, figure 4 shows the proportion of the agent population engaged in switching against time during a single incident affected day once route flows reached equilibrium (beyond day 50).The diverting proportion is calculated as the number of agents which have chosen to switch routes and are present on links 7, 8, 9, 10, 11 and 12 (the diversion section) against the number of agents present on links 6, 7, 8, 9, 10, 11 and 12 (the combination of the diverting section and incident affected link).As figure 4 shows, the 7500 agents take between 15000 seconds (≈4 hours) and 30000 seconds (≈8 hours) to pass through the network for the strategy mixtures examined.As would be expected, for lower percentages of switchers present in the population the proportion of agents engaged in switching is capped by that percentage.It is logical that if few switchers exist in the population each will engage in diverting, enjoying a reduced travel time of close to free flow conditions on the diversion route.Due to the discrete choice model used, if an agent predicts that link 6 has a higher travel time than at the start of their journey, it only becomes more likely that it will divert, hence not every switcher agent chooses to divert.
The trend of 'maximum numbers of switching agents divert' would not be expected to continue with increasing the proportion of switchers in the population.If a population of 100% switchers exists and the maximum number divert then the major incident affected route would hold a close to free flow travel time and thus be faster than the diversion route.
Figure 5 summarises and extends figure 4, showing the proportion of only the switching agents engaged in diverting during a 12500 second portion (≈3.5 hours) of the simulated day for varying proportions of switchers.Below 60% switcher populations, the mean proportion of switchers engaged in diverting reaches 0.77, with the standard deviation decreasing to 0.09.Beyond 60% a clear system change appears in both figures 4 and 5 as the average proportion of agents diverting falls.There are two reasons for this; first the mechanism as described above suggests that some switching agents will choose not to divert -although this is few as figure 4 shows that the maximum proportion of agents diverting in a population of 100% switchers is close to 0.77.Secondly, the periodic 'wave' like diversion behaviour visible in figure 4 appears and at 100% switchers the proportion of agents switching is rarely steady as diversion behaviour regularly breaks down.To understand the impact of these trends on agent experience, figure 6 charts the average travel times between the route divergence point at the end of link 5 and route merge point at the beginning of link 13 (the same region examined by figure 4) experienced by switcher, stayer and all agents on an incident affected equilibrium day.This shows the (average) benefit to agents of adopting the two strategies.As has been discussed, prior to 60% switchers within the population, switcher agents enjoy a lower average travel time as stayer agents traverse the incident affected major route.Beyond 60%, the value of this benefit to switchers decreases even though in every simulation it is on average better for agents to adopt the switcher behaviour.
To examine the effect of varying population proportions on system performance, figure 7 charts the time required for all 7500 agents to pass through the network.As the proportion of switching agents in the population increases, up to around 70-80% switchers, the amount of time required falls, suggesting that the system can be considered to be acting in a more optimal fashion.Beyond 80% switchers in the population, despite the simulation consisting of more agents capable of making en-route diverting decisions in the hope of decreasing overall travel time, the time taken for all agents to traverse the network increases.
The graph in figure 7 also shows the 'optimal' time required to complete an incident affected day from a simulation where the likelihood of link 6 being affected by incidents is certain.This simulation length is lower because uncertainty is removed from the system and network attributes do not change in the day to day model.Agents can each optimise their initial route choices through the evolutionary route adaptation process and at equilibrium do not need to alter route choices within day.Consequently the negative diversion breakdown does not occur.

Diversion breakdown and the role of information
The simulation result in figure 4 has shown how, for higher proportions of switching agents existing within the population, when an incident arises the number of agents engaging in diverting rises and falls in a wave like motion which has a negative impact on overall network performance.
To demonstrate how this trend arises, figure 4 shows a series of simulation outputs at six time steps on an incident afflicted day (as shown by the capacity decrease on link 6).In a) (t = 1755s) agents are joining the network and, due to the existence of congestion on the links leading up to the route diverging point, perceive link 6 to be in a highly congested state.At this early point in the simulation the upstream queue is still forming and few agents are diverting.In b) (t = 3990s) the congestion stretches up the network but due to a larger number of agents engaging in diverting the size of the queue in each cell decreases -as in the peaks in figure 4. By c) (t = 5040s) the reduced congestion on preceding links means that agents are no longer capable of considering link 6 to be in a highly congested state even though link 6 remains affected by the incident.At d) (t = 5530s) few agents are engaged in diverting and most agents join link 6 believing it to be clear as in the troughs from figure 4. In e) (t = 5930s) queues re-form on links preceding link 6 and by f) (t = 6490s) agents once again perceive that link 6 is in the heavily congested state and again engage in diverting as in the peaks in figure 4.
The simulation has shown how, when an incident occurs, agents anticipate its presence and more agents which are capable of diverting do so, the queue on preceding links decreases and agents joining the simulation receive no information about any queues occurring ahead, so are unable to predict that link 6 is in a highly congested state.Thus all incoming agents naively remain on the major route believing it clear, eventually creating more queues which then back up the carriageway restarting the cycle.
Diversion breakdown has the result of decreasing network performance despite being caused by agents trying to decrease their own travel times, suggesting that in this simulation some level of queueing on upstream links can be seen as positive.In order to reduce average travel times in a network without information provided, some drivers are required to wait in congestion so that others can benefit from observing the presence of queues.

Conclusions
This model has demonstrated a plausible road traffic phenomena in the form of diversion breakdown which is created in simulation by incorporating within the model both inter-vehicle interactions and a driver knowledge representation which focuses on experience gathered within trip and between anticipated link states.Although many other models of driver route choice exist, as well as models which explore other aspects of driver behaviour, this work has sought to explore the consequences of a single type of behaviour -adopting en-route diversions.Scope exists to incorporate the findings of this work with existing and future driver route choice models.
The findings from this model are also more relevant to real world road networks where users can be considered to be experienced.For example in areas where drivers are not expected to possess local knowledge, such as on major road,  they may be reluctant to divert.Additionally, a driver unfamiliar with an area may hold their own potentially incorrect assumptions regarding traffic flows which will influence their routing and en-route diversion behaviour.Network structure will also play a key role in whether and when diversion breakdown occurs and there may be multi-ple opportunities for drivers to divert.Additionally, overall network performance is only improved if the diversion route can accommodate increased volumes of traffic which is uncertain, even unlikely, in most real world traffic networks.
To summarise, this paper has presented a plausible and general agent behaviour model of driver road network perceptions.Through the modelling of road conditions as belonging to one of a set number of states, a coupled hidden markov model can model the relationships between states and provide expectations of driver behaviour.The simulation has shown the competing pressures on drivers as choosing to remain on their initial route choices or be open to diverting on to alternatives.
Free flow Travel time on link is less than or equal to 1.3•(Free flow travel time on link) Moderate congestion Travel time on link is less than or equal to 2.0•(Free flow travel time on link) and greater than 1.3•(Free flow travel time on link) Heavy congestion Travel time on link is greater than 2.0•(Free flow travel time on link)

Figure 1 :
Figure 1: The network structure under examination, also showing the cells associated with links and vehicles traversing the network in a non incident affected day.

Figure 2 :
Figure 2: Cell transmission model output showing a stream of vehicles encountering a cell of lower capacity, resulting in the formation of an upstream queue.

Figure 3 :
Figure 3: Numbers of agents choosing the diversion route as their initial route choice against simulation day.

Figure 4 :
Figure 4: Proportion of agents engaged in diverting throughout an incident affected simulated day at equilibrium.

Figure 5 :
Figure 5: Mean and standard deviation of proportion of switching agents engaged in diverting between t = 2500s and t = 15000s.

Figure 6 :
Figure6: Mean of travel times between diverging and merging points experienced by switcher, stayer and all agents during a single incident affected equilibrium day for varying proportions of switchers in the population.

Figure 7 :
Figure7: Time required to complete the movement of 7500 agents at equilibrium on an incident affected day.Also shown is the 'optimal' time required when the probability of an incident on link 6 cell 9 is 1.0.

Figure 8 :
Figure8: Simulation outputs at a number of time steps exploring system behaviour on an incident affected day at equilibrium with a 100% switchers population.Agents occupying each cell are coloured according to their belief of the current state of link 6.