Weighted DAG Automata for Semantic Graphs

Statistical models of natural language semantics are making rapid progress. At the risk of oversimplifying, work in this area can be divided into two streams. One stream, semantic parsing (Mooney 2007), aims to map from sentences to logical forms that can be executed (for example, to query a knowledge base); work in this stream tends to be on small, narrow-domain data sets like GeoQuery. The other stream aims for broader coverage, and historically tackled shallower, piecemeal tasks, like semantic role labeling (Gildea and Jurafsky 2000), word sense disambiguation (Brown et al. 1991), coreference resolution (Soon, Ng, and Lim 2001), and so on. Correspondingly, resources like OntoNotes (Hovy et al. 2006) provided separate resources for each of these tasks.

This piecemeal situation parallels that of early work on syntactic parsing, which focused on subtasks like part-of-speech tagging (Ratnaparkhi 1996), noun-phrase chunking (Ramshaw and Marcus 1995), prepositional phrase attachment (Collins and Brooks 1995), and so on. As the field matured, these tasks were increasingly synthesized into a single process. This was made possible because of a single representation (phrase structure or dependency trees) that captures all of these phenomena; because of corpora annotated with these representations, like the Penn Treebank (Marcus, Marcinkiewicz, and Santorini 1993); and because of formalisms, like context-free grammars, which can model these representations practically (Charniak 1997; Collins 1997; Petrov et al. 2006).

In a similar way, more recent work in semantic processing consolidates various semantics-related tasks into one. For example, the Abstract Meaning Representation (AMR) Bank (Banarescu et al. 2013) began as an effort to unify the various annotation layers of OntoNotes. It has driven the development of many systems, chiefly string-to-AMR parsers like JAMR (Flanigan et al. 2014) and CAMR (Wang, Xue, and Pradhan 2015a, b), as well as many other systems submitted to the AMR Parsing task at SemEval 2016 (May 2016). AMRs have also been used for generation (Flanigan et al. 2016), summarization (Liu et al. 2015), and entity detection and linking (Li et al. 2015; Pan et al. 2015).

But the AMR Bank is by no means the only resource of its kind. Others include the Prague Dependency Treebank (Böhmová et al. 2003), DeepBank (Oepen and Lønning 2006), and Universal Conceptual Cognitive Annotation (Abend and Rappoport 2013). By and large, these resources are based on, or equivalent to, graphs, in which vertices stand for entities and edges stand for semantic relations among them. The Semantic Dependency Parsing task at SemEval 2014 and 2015 (Oepen et al. 2014, 2015) converted several such resources into a unified graph format and invited participants to map from sentences to these semantic graphs.

The unification of various kinds of semantic annotation into a single representation, semantic graphs, and the creation of large, broad-coverage collections of these representations are very positive developments for research in semantic processing. What is still missing—in our view—is a formal framework for creating, combining, and using models involving graphs that parallels those for strings and trees. Finite string automata and transducers served as a framework for investigation of speech recognition and computational phonology/morphology. Similarly, context-free grammars (and push-down automata) served as a framework for investigation of computational syntax and syntactic parsing. But we lack a similar framework for learning and inferring semantic representations.

Two such formalisms have recently been proposed for NLP: one is hyperedge replacement graph grammars, or HRGs (Bauderon and Courcelle 1987; Habel and Kreowski 1987; Habel 1992; Drewes, Kreowski, and Habel 1997), applied to AMR parsing by various authors (Chiang et al. 2013; Peng, Song, and Gildea 2015; Björklund, Drewes, and Ericson 2016). The other formalism is directed acyclic graph (DAG) automata, defined by Kamimura and Slutzki (1981) and extended by Quernheim and Knight (2012). In this article, we study DAG automata in depth, with the goal of enabling efficient algorithms for natural language processing applications.

After some background on the use of graph-based representations in natural language processing in Section 2, we define our variant of DAG automata in Section 3. We then show the following properties of our formalism:

• • 

  Path languages are regular, as is desirable for a formal model of AMRs (Section 4.1).

• • 

  The class of hyperedge-replacement languages is closed under intersection with languages recognized by DAG automata (Section 4.2).

• • 

  Emptiness is decidable in polynomial time (Section 4.3).

We then turn to the recognition problem for our formalism, and show the following:

• • 

  The recognition problem is NP-complete even for fixed automata (Section 5.1).

• • 

  For input graphs of bounded treewidth, there is an efficient algorithm for recognition or summing over computations of an automaton for an input graph (Section 5.2).

• • 

  The recognition/summation algorithm can be asymptotically improved using specialized binarization techniques (Section 6).

• • 

  All closure and decidability properties mentioned above continue to hold for the extended model, and the path languages stay regular (Section 7.3).

• • 

  We provide a practical recognition/summation algorithm for the novel model (Section 7.4).

Graphs, or representations equivalent to graphs, have been used by many linguistic formalisms and natural-language processing systems to model semantic dependencies. For example, unification-based grammar formalisms use feature structures, like lexical functional grammar f-structures (Kaplan and Bresnan 1982) and head-driven phrase structure grammar synsem objects (Pollard and Sag 1994), that can be drawn as rooted, directed, (usually) acyclic graph structures (Shieber 1986). The Prague Dependency Treebank’s tectogrammatical trees (Böhmová et al. 2003) can be turned into graphs using coreference and argument-sharing annotations, and DeepBank’s annotations using Minimal Recursion Semantics can be stripped down to Elementary Dependency Structures, which are graphs (Oepen and Lønning 2006). Universal Conceptual Cognitive Annotation (Abend and Rappoport 2013) uses several annotation layers, which are graphs. AMRs, whose format is derived from the PENMAN generation system, are equivalent to graphs (Banarescu et al. 2013). Several of these graph representations have been the target of the Semantic Dependency Parsing task at SemEval 2014 and 2015 (Oepen et al. 2014, 2015).

In this section, we focus on AMRs, but the formalisms we work with in the remainder of the article could, in principle, be used on any of the other graph representations listed above. Although the standard AMR format somewhat resembles the Penn Treebank’s parenthesized representation of trees, AMRs can be thought of as directed graphs. Examples of these two representations, from the AMR Bank (LDC2014T12), are reported in Figures 1 and 2. Nodes are labeled, in order to convey lexical information. Edges are labeled to convey information about semantic roles. Labels at the edges need not be unique, meaning that edges impinging on the same node might have the same label. Furthermore, our DAGs are not ordered, meaning that there is no order relation for the edges impinging at a given node, as is usually the case in standard graph structures. A node can appear in more than one place (for example, in Figure 1, node s2 appears six times); we call this a reentrancy, analogous to a reentrant feature structure in unification-based grammar formalisms.

Cycles and multiple roots. Although the AMR guidelines¹ describe AMRs as acyclic graphs, the AMR Bank in fact contains some graphs with cycles. The majority of these cyclic graphs involve an edge labeled with an inverse role such as ARG0-of, which means that the parent node is the ARG0 of the child node. The purpose of these inverse roles is to make the graph singly-rooted. If we reverse such edges, most cyclic graphs become acyclic (but multiply-rooted).

Most remaining cycles are caused by a relatively small number of roles. By “reifying” these, that is, changing them into nodes (see Figure 3), these cycles can be eliminated. Table 1 shows some statistics on the December 2014 internal release of the AMR Bank.²

The small percentage of graphs that are cyclic is reduced by reversing *-of edges, and all but eliminated by reification. The three cyclic graphs that remain (out of 20,628) were clearly annotation mistakes and were subsequently corrected.

The table also shows that the average number of roots more than doubles as a result of these transformations. (The original corpus had a small number of instances that contained more than one sentence, and were annotated as multiple graphs under a multi-sentence node; we counted these as multiple roots.)

In summary, we can think of AMRs as singly-rooted, possibly cyclic directed graphs, or as multiply-rooted directed acyclic graphs.

Node degree. The in-degree (out-degree) of a node in a DAG is the number of incoming (outgoing, respectively) edges at that node. AMRs have unbounded in-degree and out-degree. Unbounded in-degree is needed for instance in the semantic representation of sentences with coreference relations, in which some concept is shared among several predicates. Unbounded out-degree allows the attachment to a given predicate a number of optional modifiers that can grow with the length of the sentence. We studied the degree distribution of nodes in the AMR Bank.³ The maximum degree (in-degree plus out-degree) is 17, and the average is 2.12. The full degree distribution is shown in Figure 4. In practice, AMRs strongly favor nodes of low degree. Nonetheless, the presence of nodes with large degree indicates that practical applications are likely to benefit from algorithms capable of handling potentially unbounded degree, which we develop in Section 7.

Multiple edges. In the standard definition for graphs, also called simple graphs, there can be at most one edge between two nodes. As opposed to simple graphs, multigraphs allow more than one edge between two nodes, called multiple edges. In semantic representations this is very useful. For instance, in the AMR for the sentence “John likes himself,” the node for the predicate “like” has its ARG0 and ARG1 semantic roles filled by the same argument “John.” Accordingly, we use multigraphs to represent AMR. This also simplifies the definition of a recognition model for AMRs, since a check to avoid multiple-edges would in some sense add an external condition, making the theory more difficult to develop.

Treewidth. Several of the algorithms presented in this article depend on the graph-theoretical notion of treewidth. The treewidth of a graph G, written tw(G), is a natural number that formalizes the degree to which G is “tree-like,” with trees having treewidth of 1. We will postpone the mathematical definition of tw(G) to the next section.

For a graph G and a value k given as input, it is NP-complete to determine whether G has treewidth at most k. However, for the semantic graphs we are dealing with, the worst case might not be realized. Using a reimplementation of the QuickBB algorithm (Gogate and Dechter 2004), with only the “simplicial” and “almost-simplicial” heuristics, we found that we could compute the exact treewidth of all the graphs in the AMR Bank in a few seconds. The results (deleting multi-sentence nodes) are shown in Table 1: The average treewidth is only about 1.5, and the maximum treewidth is only 4. An example of a graph with treewidth 4 is shown in Figure 2. As we will see, this means that algorithms with an exponential dependence on treewidth can be practical for real world AMRs.

In this section we formally specify the type of DAGs that we use in this article. We then define a family of automata that process languages of these DAGs, under the restriction that nodes have bounded degree. We also briefly discuss the existing literature on DAG automata. The restriction on node degree will later be dropped, in Section 7.

We make frequent use of finite multisets. Formally, given a set Q, a multiset over Q is a mapping μ: Q → $N$⁠. Intuitively, μ(q) = n means that q occurs n times in μ. The collection of all finite multisets over Q is denoted by $M$(Q). We usually specify a multiset μ ∈ $M$(Q) by listing its elements using a set-like notation such as {q₁, …, q_n}. Note, however, that q₁, …, q_n may contain repeated elements, in contrast to ordinary sets. We also use the latter, but the context will always disambiguate the two different meanings. The union of multisets is denoted by the operator ⊎ and is defined by pointwise addition: (μ ⊎ μ′)(q) = μ(q) + μ′(q) for all q ∈ Q. Thus, if μ = {q₁, …, q_m} and μ′ = {q₁′, …, q_n′} then μ ⊎ μ′ = {q₁, …, q_m, q₁′, …, q_n′}. If f: Q → P is a function, we extend it to a function from $M$(Q) to $M$(P) in the canonical way: f({q₁, …, q_n}) = {f(q₁), …, f(q_n)}.

An alphabet is a finite set Σ that we are going to use as node labels for our graphs. We consider graphs that are directed and unordered, have nodes labeled by symbols from Σ, and have multiple edges. We do not use edge labels, despite the fact that the AMR structures we want to model have labels at their edges. Our choice is motivated by our goal to simplify the notation. Graphs with labels only at their nodes can easily encode graphs with edge labels by splitting every edge into two, and putting an extra node in the middle, whose label is the label of the edge. We will come back to the discussion of this encoding at the end of this section, after our definition of DAG automata.

Note that our definition does not identify an edge with the pair of nodes that the edge is incident upon. In the terminology of standard graph theory, this means that our graphs are not simple graphs. This allows us to use multiple edges incident upon the same pair of nodes, a feature that is not only natural for AMRs (see the previous section) but will also be used in several of our algorithms.

A graph D as above is a directed acyclic graph if it is acyclic. More precisely, there do not exist e₀, …, e_k−1 ∈ E with k > 0 such that tar(e_i−1) = src(e_{i mod k}) for 1 ≤ i ≤ k. In this article, we will only consider directed acyclic graphs that are nonempty and connected. We call them DAGs, for short, and denote the set of all DAGs over Σ by $DΣ$⁠. Note that a DAG can have multiple roots, that is, there may be more than one node v ∈ V such that tar(e) ≠ v for all e ∈ E. (By acyclicity, there is always at least one root.) For a node v ∈ V we define the sets of incoming and outgoing edges of v in the obvious way: in(v) = {e ∈ E ∣ tar(e) = v} and out(v) = {e ∈ E ∣ src(e) = v}.

As usual, the graph D is a tree if there is a node r ∈ V, the root of D, such that every node v ∈ V ∖ {r} is reachable from r on exactly one directed path, i.e., there is exactly one sequence of edges e₁, …, e_k with k > 0 such that r = src(e₁), tar(e_i) = src(e_i+1) for all 1 ≤ i < k, and tar(e_k) = v. We use standard terminology regarding trees. In particular, a node v is a child of a node u if out(u) ∩ in(v) ≠ ∅.

As mentioned in the previous section, the treewidth of DAGs plays an important role for the algorithms proposed in this article. We now recall the notions of tree decompositions and treewidth, at the same time introducing the specific notation that will be used later in the article.

We note here that, in most definitions in the literature, the edges of a tree decomposition are undirected. In the context of this article, however, it is more convenient to define tree decompositions to be directed trees, because later on we will define algorithms that process our DAGs in an order that is guided by the arc directions in the associated tree decompositions. In order to turn an undirected tree decomposition into a directed one, just choose an arbitrary bag as the root, and establish edge directions accordingly.

Several other tree decompositions can be constructed for D. For instance, a trivial tree decomposition of D is the tree containing a single bag with all the nodes of D. However, it is not difficult to argue that every tree decomposition of D must have a bag that contains at least 4 nodes from D. Thus, the treewidth of D is 3. Informally, the size of the largest bags in a tree decomposition increases with the number of reentrancies that can be found along a path in the DAG.

Let us now embark on the definition of DAG automata. Informally, a DAG automaton consists of a set of nondeterministic transitions that read DAG nodes and associate states with their incoming and outgoing edges. Because we do not only want to recognize DAG languages but, more generally, want to be able to use DAG automata to associate a weight with each DAG, we define a more general version in which the transitions have weights taken from some semiring $K$⁠. Throughout the entire paper, all semirings are assumed to be commutative—that is, not only the additive but also the multiplicative operator is commutative.

As already mentioned, a DAG automaton processes an input DAG by assigning states to edges. A transition of the form of Equation (1) gets m states on the incoming edges of a node and puts n states on the outgoing edges. Alternatively, we may read the transition bottom–up, that is, it gets n states on the outgoing edges and puts m states on the incoming edges. As two special cases, note that when m = 0 in Equation (1) then the transition processes a root node, and when n = 0 it processes a leaf node.

Note that the transition function $δ:Θ→K∖{0}$ assigns nonzero weights to the transitions of a DAG automaton. Intuitively, the weight of all transitions not in Θ is 0. Reflecting this intuition, we extend δ to the set of all possible transitions t ∈ $M$(Q) × Σ × $M$(Q) by defining δ(t) = 0 for every t ∉ Θ. In this way, δ is turned into a total function, which is sometimes convenient.

The use of multisets of states in Equation (1) is needed because, when processing a node v, the same state might be assigned to several of the edges in in(v) or in out(v), and we have to specify the collection of all these state occurrences. As an example, assume |in(v)| = 3. Then we should distinguish between the scenario where the assigned states are {q, q, q′} and the scenario where the assigned states are {q, q′, q′}.

An unweighted DAG automaton is the special case of a DAG automaton in which $K$ is the Boolean semiring. In this case, $M:DΣ→{true,false}$ is the characteristic function of a subset of $DΣ$⁠. We generally identify such a characteristic function with the corresponding set, that is, $M={D∈DΣ∣M(D)=true}$⁠, and call it the DAG language recognized by M. DAG languages that can be recognized by unweighted DAG automata are recognizable DAG languages. Note that because false is the zero element of the Boolean semiring, all transitions appearing in an unweighted DAG automaton are of the form ${q1,…,qm}σ/true→{r1,…,rn}$⁠. So we can simplify the notation of such a transition by writing ${q1,…,qm}σ→{r1,…,rn}$⁠. An accepting run of M is a run whose weight is true, i.e., which uses only transitions of M.

It may be instructive to note that the construction of a run of the automaton can be understood as a top–down or a bottom–up process. Under the top–down view, this particular automaton is deterministic: for each node the states on the incoming edges uniquely determine those on the outgoing edges. In contrast, under a bottom–up view, thus essentially reading transitions backwards, the transitions for b create a nondeterministic behavior.

Similarly, our DAG automata generalize tree automata (Comon et al. 2002), because a DAG automaton with transitions having at most one incoming state and any number of outgoing states will recognize a tree.

As already mentioned, although the standard AMR representation has labels on both edges and nodes, for simplicity we only consider DAGs with labels on nodes. We represent the edge labels of AMR, such as ARG0 and ARG1, as nodes with one incoming and one outgoing edge.

We observe that our DAG automata could, without any change in the definitions, also be applied to directed acyclic graphs that may be disconnected, or even to graphs over Σ containing cycles if this turns out to be of interest for some application. Of course, the algorithmic results presented in the following could not necessarily be assumed to hold in such a generalized case anymore.

Other than in the perspective of natural language processing, DAG automata have been investigated in several different domains—for instance, to represent derivations in Chomsky type-0 phrase structure grammars (Kamimura and Slutzki 1981), to solve systems of set constraints (Charatonik 1999), or to process series-parallel graphs in pattern matching applications (Fujiyoshi 2010).

Kamimura and Slutzki (1981) define automata for two classes of DAGs. They primarily consider so-called d-DAGs, a recursively defined type of ordered planar DAG, where ordered means that there is a global total order on the set of nodes of the graph that implicitly orders the incoming and outgoing edges of each node. These d-DAGs are intended to model the derivations of type-0 grammars (equivalent to Turing machines). Accordingly, d-DAGs have bounded node degree and cannot have subgraphs matching certain Z-like patterns that would correspond to the same node being rewritten by two different rules. These restrictions are unsuitable when modeling natural language semantic structures. The authors also briefly consider DAGs without the planarity restriction, but still ordered in the sense mentioned above.

Our definition of DAG automata is based on that of Quernheim and Knight (2012). Also motivated by modeling semantic representations of natural languages, Quernheim and Knight extend the automata of Kamimura and Slutzki (1981) by adding weights and by dropping the planarity restriction as well as the bound on the in-degree. In order to process nodes with unbounded in-degree, Quernheim and Knight exploit some ordering on the incoming edges at each node, and introduce so-called implicit rules that process these edges in several steps. In Section 7 we take a different, simpler approach for processing DAGs with unbounded node degree that can also handle unbounded out-degree. Overall, this article can be viewed as an in-depth exploration of the theoretical properties of a somewhat simplified version of the formalism of Quernheim and Knight.

There are also major notational differences with respect to our proposal: Quernheim and Knight (2012) essentially view computations as top–down rewriting processes, and the rewriting relation is defined via the introduction of specialized DAGs, called incomplete DAGs. In contrast, in our definition of run in Sections 3.1 and 7.2, there is no commitment to a specific rewriting process, which makes the notation somewhat simpler. Quernheim and Knight also show how to obtain weighted DAG-to-tree transducers, which could form the basis of a natural language generation system.

With the goal of modeling ground terms in logical languages, Charatonik (1999) proposes devices that are mainly bottom–up tree automata running on DAGs, and states the external restriction, not implemented through the defined automata, that for these DAGs common substructures should be maximally shared. This maximal sharing condition is quite common in the literature on unification, but is unsuitable when modeling natural language semantic structures: Two copies of the same semantic substructure should be shared only when they refer to the same concept or action. A consequence of the maximal sharing is that, even in a nondeterministic automaton, isomorphic sub-DAGs are assigned the same state (because they are actually identical). This is exploited in the main result of Charatonik, the NP-completeness of the emptiness problem. This is in contrast with the polynomial time result for the same problem for our DAG automata, presented in Section 5.1.

Anantharaman, Narendran, and Rusinowitch (2005) also work under the maximal sharing assumption, and solve in the negative the problem of closure under complementation that had been left as an open question by Charatonik (1999). The authors consider the uniform membership problem for their automata, showing NP-hardness. Here, uniform means that the automaton is considered as part of the input. In our article and relative to our family of automata, we consider the easier problem of deciding membership for a fixed automaton, given only the DAG as input. Despite the more restricted question, we can show NP-hardness. Anantharaman, Narendran, and Rusinowitch also show that universality is undecidable for their automata. Finally, with the motivation of representing sets of terms by means of a single DAG, they also consider DAGs where each node has an additional Boolean label. This representation does not seem to be relevant for modeling of natural language semantic structures.

Fujiyoshi (2010) considers DAG automata that are essentially top–down tree automata. Such an automaton is said to accept a DAG if there exists a spanning tree of the DAG that is accepted by the automaton (viewed as a tree automaton). In particular, whenever a DAG is accepted, every other DAG obtained by adding edges is also accepted. This property does not seem to be desirable for modeling semantic structures. Similarly to our result, Fujiyoshi proves that the non-uniform membership problem is NP-complete, but although he also uses a reduction from SAT, the reduction itself is very different from ours (as expected, due to the differences in the automata models).

Among the types of DAG automata studied in theoretical computer science, the model by Priese (2007) is the one that comes closest to the extended DAG automaton introduced in Section 7, even though Priese uses an algebraic setting to describe it. The major difference is that the DAG automata of Priese are able to check that the multiset of states assigned to the roots and leaves of the input DAG belongs to a given regular set, in the sense of Section 7.1. For example, it is possible to express the condition that recognized DAGs shall have a unique root. At first sight, this may appear to be a minor point, but this is not so. Section 4.1 shows that the path languages of our model are regular, whereas they are not even context-free once it becomes possible to express that a DAG has a unique root (which is also observed by Priese [2007]). We consider this to be an indication that our DAG automata are better suited for studying semantic structures because we expect those to have regular path languages, and in the interest of algorithmic results one should not use unnecessarily powerful models. In the more general setting of Priese, our recognition algorithm does not apply, and our proof of the polynomial decidability of the emptiness problem, and the corresponding result for finiteness of Blum and Drewes (2016), break down. Apart from the mentioned study of path languages, the questions studied by Priese are essentially disjoint with those studied in this article.

Another automaton model for graph processing is the graph acceptor by Thomas (1991, 1996). A graph acceptor consists mainly of a finite set of pairwise non-isomorphic r-tiles that play the role of the rules. Each tile is an r-sphere (i.e., a graph with a center node whose distance to all other nodes is at most r). Each node of such a tile carries a state. A run on an input graph G is then a mapping of states to the nodes of G such that each node is the center of one of the tiles. The definition of the graph acceptor includes an occurrence constraint, a Boolean combination of conditions that restrict the number of occurrences of each tile. A given run is accepting if the occurrence constraint is satisfied. The expressiveness of the model can be characterized by existential monadic second-order logic (Thomas 1996), and it can be extended by weights (Droste and Dück 2015). Similar to our basic (non-extended) model, graph acceptors of this type recognize graph languages of bounded degree. However, because of the overlapping of tiles in runs and the occurrence constraint, they are considerably more powerful than our DAG automata (and thus too powerful for our purposes) unless the tiles are required to have the radius 0 (i.e., they are single nodes). The latter restriction results in too weak a model, because it cannot say anything about the edges in the graph if each tile is just a single node.

More results on the (non-extended) DAG automata invented in this article were proved by Blum (2015) and Blum and Drewes (2016). In particular, an alternative proof of the regularity of path languages was given in Blum (2015) (which is simpler and more constructive, but was conceived after the proof in Section 4.1), and the polynomial decidability of the finiteness problem was proved.

Without going into further detail, we mention here some additional publications by diverse authors on automata recognizing DAGs or graphs: Bossut, Dauchet, and Warin (1988); Kaminski and Pinter (1992); Potthoff, Seibert, and Thomas (1994); Bossut, Dauchet, and Warin (1995). Furthermore, there exists considerable work within the XML community on evaluating tree automata and logical queries on compressed representations of trees, which are DAGs (see, e.g., Frick, Grohe, and Koch 2003; Lohrey and Maneth 2006). This work seems to be only tangentially related to the present article because it is not interested in the DAG as a structure in its own right (and automata that define DAG languages), as we are.

In this section we consider only unweighted DAG automata. We explore three properties of such DAG automata and of the (unweighted) DAG languages recognized by them:

• • 

  With multiple roots, the path languages of DAG automata are regular; but not under the constraint of a single root (Section 4.1).

• • 

  Hyperedge replacement graph languages are closed under intersection with languages recognized by DAG automata (Section 4.2).

• • 

  Testing for emptiness of DAG automata is decidable under our definition, but not under the original definition by Kamimura and Slutzki (Section 4.3).

Reading the labels of nodes on the paths in a DAG D from a root to a leaf yields the path language of the DAG, denoted by paths(D). (In the following, all paths are assumed to start at a root; their rootedness will thus no longer be mentioned.) The path language of a set L of DAGs is the union $paths(L)=⋃D∈Lpaths(D)$ of the path languages of its individual DAGs. We now show that the path language of a recognizable DAG language is always a regular string language. Thus, in this respect our DAG automata are similar to those by Kamimura and Slutzki (1981), whose path languages are trivially regular. However, if we restrict recognizable DAG languages to DAGs with only one root, then this no longer holds. In fact, in this case even non-context-free path languages are obtained, as in the case of Priese (2007).

Let us first show that path languages of recognizable DAG languages (without the restriction to unique roots) are regular. To this end, recall that we have defined DAGs as connected directed acyclic graphs. Let us now drop the connectedness assumption, and consider arbitrary directed acyclic graphs, which we call nc-DAGs. Then any nc-DAG can be written as the finite disjoint union D₁ + ⋯ + D_k of (connected) DAGs D₁, …, D_k, for k ≥ 1. Here D + D′ is used to denote the disjoint union of DAGs D and D′.

We define as $M+$ the language of nc-DAGs recognized by M: In words, each nc-DAG in $M+$ is the disjoint union of one or more DAGs from $M$⁠. We extend our definition of path language of a DAG to nc-DAGs and to languages of nc-DAGs. Let $D1,…,Dk∈M$⁠. We have paths(D₁ + ⋯ + D_k) = paths(D₁) ∪ ⋯ ∪ paths(D_k). It directly follows that $paths(M)$ and $paths(M+)$ coincide. This observation will be used later to simplify our proof.

Another useful observation is the following. Consider a DAG D = (V, E, lab, src, tar) and two edges e₁, e₂ ∈ E. Let $D[e1↔e2]$ denote the graph that is obtained from D by interchanging the targets of e₁ and e₂. More precisely, if v_i (i = 1, 2) is the node such that e_i ∈ in(v_i), then $D[e1↔e2]$ has e₁ ∈ in(v₂) and e₂ ∈ in(v₁) but is otherwise identical to D. It is not difficult to see that the edge interchange operator we have just defined might introduce cycles, that is, $D[e1↔e2]$ may no longer be a DAG. However, in what follows we will use this operator in a restricted way, such that the resulting graph is still a DAG.

Suppose that $D∈M+$ and let ρ be an accepting run of M on D. If D = D₁ + D₂ and, for i = 1, 2, e_i is an edge of D_i such that ρ(e₁) = ρ(e₂), then $D[e1↔e2]∈M+$⁠. This is true because $D[e1↔e2]$ is still acyclic (because e₁ and e₂ belong to distinct connected components of D) and ρ is an accepting run on $D[e1↔e2]$ as well.

Now, let us turn M into M′ by adding a unique leaf symbol ℓ, adding a new state f and the transition ${f}ℓ→∅$⁠, and turning all original transitions of the form ${q1,…,qk}a→∅$ into ${q1,…,qk}a→{f}$⁠. Thus, the DAGs recognized by M′ are those originally recognized by M, but with additional leaves labeled ℓ added as leaves below the original leaves, and the accepting runs of M′ are those of M, extended by labeling the (unique) outgoing edges of the original leaves with f.

For a string w ∈ Σ⁺, let Δ(w) denote the set of all states q for which there exists an accepting run ρ of M′ on a DAG D such that some path labeled w leads to an edge e with ρ(e) = q. Hence, $paths(M)={w∣f∈Δ(w)}$⁠. By the Myhill-Nerode theorem, it therefore suffices to show that the equivalence relation ∼, given by w₁ ∼ w₂ if and only if Δ(w₁) = Δ(w₂), is a right congruence. In other words, if Δ(w₁) = Δ(w₂) and w is any string, then $w1w∈paths(M)$ if and only if $w2w∈paths(M)$⁠.

So, assume that Δ(w₁) = Δ(w₂) and $w1w∈paths(M)$⁠. Then there is some D₁ ∈ $M′$ containing a path p whose node labels are w₁wℓ. Let ρ₁ be a run on D₁ and consider the |w₁|-th edge e₁ on p, i.e., the edge between w₁ and w. Then ρ₁(e₁) ∈ Δ(w₁). Choose any D₂ ∈ $M′$⁠, edge e₂, and accepting run ρ₂ such that some path to e₂ in D₂ is labeled by w₂ and ρ₂(e₂) = ρ₁(e₁). Note that D₂, e₂, and ρ₂ exist because Δ(w₁) = Δ(w₂). Now, let D = D₁ + D₂. By the observation above the graph $D[e1↔e2]$ is in $M′$₊. Furthermore, it obviously contains the path w₂wℓ, which means that f ∈ Δ(w₂w) and thus $w2w∈paths(M)$⁠, as required.

We have thus shown that the path language of every recognizable DAG language is a regular string language. The proof of this statement relies crucially on the fact that DAGs in $M$ may have several roots: We considered the disconnected graph D = D₁ + D₂ ∈ $M′$₊ and turned it into $D[e1↔e2]∈[[M′]]+$⁠. However, the latter may be connected and may thus, in fact, be an element of $M′$⁠, while containing the roots of both D₁ and D₂.

To end this section, we discuss two examples showing that, indeed, the regularity of path languages (and even its context-freeness) is lost if single-rootedness is imposed on the DAGs (see Priese [2007] for similar arguments). More precisely, let $Ms={D∈M∣Dhas only one root}$⁠. Then $paths(Ms)$ is not necessarily context-free, as the following two examples show.

In an accepting run on a DAG having a single root (labeled by a) every a is related to a uniquely determined b, and vice versa, by paths of the form $a→∘→b$ (where ρ(e) = r and ρ(e′) = r′ for the incoming and outgoing edge, respectively). Hence, the intersection of the path language of $Ms$ with a*b* is {aⁿbⁿ | n ≥ 1}, a strictly context-free language. This means that the path language of $Ms$ cannot be regular. Note that the construction breaks down if arbitrarily many roots are allowed (i.e., if $M$ is considered); in this case, no “counting” is possible and we simply get a⁺b⁺.

An example run of this automaton (for k = 3) is illustrated in Figure 8. Similarly to the previous example, taking the intersection of $paths(Ms)$ with the regular language {∘w ∣ w ∈ {a₁, …, a_k}*} yields the intended language. The reader should easily be able to adapt the automaton in such a way that the initial ∘ is dropped.

Note that, whereas MIX(k) is well known to be context-free, MIX(k) is not context-free for any k > 2. It has been recently shown (Salvati 2014) that MIX(3) can be generated by a Linear Context-Free Rewriting System (Vijay-Shanker, Weir, and Joshi 1987), but it is unknown whether MIX(k), k > 3, can be generated by this class.

A hyperedge replacement grammar (HRG, see Drewes, Kreowski, and Habel [1997] for an overview) is a context-free type of graph grammar. It can in particular be used to generate DAG languages. Recognition algorithms for HRGs (Lautemann 1990; Chiang et al. 2013) can be thought of as constructions that intersect the graph language $G$ generated by an HRG G with a single graph. But just as the Cocke-Kasami-Younger algorithm for context-free grammars can be thought of as a special case of intersecting a context-free language with a regular language (Bar-Hillel, Perles, and Shamir 1961), we would more generally like to be able to intersect $G$ with any recognizable DAG language. In other words, given an unweighted DAG automaton M, we would like to construct an HRG G′ such that $[[G′]]=G∩M$⁠.

To discuss briefly how this can be done, we need to give a rough introduction to HRGs (adapted to the setting and terminology of the current article). An HRG comes with a ranked alphabet N of nonterminal hyperedge labels, in addition to the alphabet Σ of node labels. Here, saying that N is ranked means that N is specified as a disjoint union $N=⋃kNk$⁠, where the elements of N_k are said to be the symbols of rank k. A hypergraph H is a graph that may, in addition to the usual elements, contain a finite set of hyperedges. Each hyperedge h has a label lab(h) ∈ N_k for some k ∈ $N$ and a sequence att(h) ∈ V^k of attached nodes. We also view h as having ktentacles that connect it to its k attached nodes.

An HR rule r = (L :: = R) consists of a left-hand side L and a right-hand side R. L is a hypergraph that consists of a single hyperedge h labeled by some X ∈ N_k, together with the attached nodes of h, say u₁, …, u_k. These nodes should be thought of as being unlabeled as their label is irrelevant. The right-hand side is a hypergraph whose set of nodes also contains u₁, …, u_k (among other nodes). Suppose that a host hypergraph H contains a hyperedge h′ labeled with X and attached to nodes v₁, …, v_k. Then the rule r can be applied to it, which yields the hypergraph obtained by removing h′ from H and inserting the right-hand side of r in its place by identifying each u_i with the corresponding v_i. Figure 9 shows an example of a rule and its application. An HRG G consists of an alphabet N of nonterminals as before, an initial nonterminal of rank 0, and a finite set of HR rules. The generated graph language $G$⁠, called an HR language, consists of all graphs that can be derived from the initial nonterminal by repeated rule application.

Suppose now that we are given an HRG G that generates graphs over Σ, and an unweighted DAG automaton M that recognizes a DAG language over Σ. We want to construct another HRG G′ such that $[[G′]]=G∩M$⁠. That this is possible follows from several known results, but most easily using monadic second-order (MSO) logic. Courcelle (1990, Corollary 4.8) shows that the restriction of an HR language by a property expressible in MSO logic yields an HR language (for which a suitable HRG can effectively be constructed). Thus, it suffices to argue that every recognizable DAG language is definable by an MSO formula. Suppose we want to express in MSO logic that a given DAG automaton with state set Q = {q₁, …, q_n} accepts a DAG D = (V, E, lab, src, tar). We can do this by constructing an MSO formula that “guesses” an accepting run ρ. The formula states that there exists a partition of E into subsets E₁, …, E_n such that the following holds: For every node v ∈ V with in(v) = {e₁, …, e_m} and out(v) = {e₁′, …, e_n′}, there exist i₁, …, i_m and j₁, …, j_n such that

• 1. 

  ${qi1,…,qim}lab(v)→{qj1,…,qjn}$ is a transition of M and

• 2. 

  $er∈Eir$ for 1 ≤ r ≤ m and $es′∈Ejs$ for 1 ≤ s ≤ n.

Let us now sketch a direct construction of G′. Without loss of generality, we may assume that $G$ is a set of DAGs, because it is well known that the class of HR languages is closed under intersection with the set of all connected acyclic graphs. (This is, in fact, another application of the closedness under intersection with MSO properties.) The idea behind the construction of G′ is to use a guess-and-verify strategy to guarantee that only those graphs are generated that have accepting runs in M. To implement this strategy, we augment the nonterminal labels of hyperedges with the guessed information. To understand this, note that every tentacle of a hyperedge intuitively controls a node to which the derivation of this hyperedge will eventually attach a number of incoming and outgoing edges. We have to guess beforehand the multiset of states that an accepting run will assign to these edges. To keep track of this guess, we have to remember two multisets of states for each tentacle, one referring to outgoing edges and one referring to incoming edges that will be generated. Consequently, the new sets N_k′ of nonterminal labels of rank k consist of all (X, μ₁ ⋯ μ_k, μ₁′ ⋯ μ_k′) such that X ∈ N_k and μ₁, …, μ_k, μ₁′, …, μ_k′ are multisets of states in Q. The size of these multisets is bounded by the maximum size of multisets in the transitions of M. This makes sure that the set of nonterminals is finite. The initial nonterminal is (X₀, λ, λ), where X₀ is the initial nonterminal of G and λ is the empty sequence.

To define the rules of G′, we need a few preparations. Consider a hypergraph H over Σ and N′ and a function ρ that maps every (ordinary) edge e of H to a state ρ(e) ∈ Q. In the following, we call ρ a state assignment for H. Given such a state assignment and a node v of H, we let in_ρ(v) denote the multiset of states obtained by taking the union of, first, all {ρ(e)} with e ∈ in(v) and, second, all μ_i such that there is a hyperedge h labeled (X, μ₁ ⋯ μ_k, μ₁′ ⋯ μ_k′) whose ith tentacle is attached to v. Similarly, out_ρ(v) is the union of all {ρ(e)} with e ∈ out(v) and all μ_i′ such that there is a hyperedge h labeled (X, μ₁ ⋯ μ_k, μ₁′ ⋯ μ_k′) whose ith tentacle is attached to v.

Now, consider all HR rules L :: = R that can be obtained from rules of G by augmenting each nonterminal label in all possible ways. A rule L :: = R obtained in this way becomes a rule of G′ if there exists a state assignment ρ for R such that

• 1. 

  in_ϵ(v) = in_ρ(v) and out_ϵ(v) = out_ρ(v) for all nodes v of L, where ϵ is the unique (empty) state assignment for L, and

• 2. 

  $inρ(v)lab(v)→outρ(v)$ is a transition of M for every node v of R which is not in L.

By induction on the length of derivations it can be shown that D ∈ $G′$ if and only if $D∈G$ and there exists an accepting run of M on D. In other words, $[[G′]]=G∩M$⁠, as required.

The emptiness problem for DAG automata asks, for an unweighted DAG automaton M as input, whether $M=∅$⁠. As mentioned earlier, the DAG automata of Kamimura and Slutzki (1981) can encode computations of Turing machines. In particular, this means that their emptiness problem is undecidable. As we shall see next, this sharply distinguishes their DAG automata from ours, whose emptiness problem can be decided in polynomial time as it can be reduced to a particular case of the reachability problem for Petri nets. A similar idea was used by Kaminski and Pinter (1992) to prove the decidability of the emptiness problem for their graph automata. However, in their case it required the use of the general Petri net reachability problem, which leads to an algorithm whose running time is non-elementary. In contrast, we obtain a polynomial algorithm.

Let us first briefly recall Petri nets. A Petri net is an unlabeled directed graph N = (V, E, src, tar) such that V consists of disjoint sets T and P of transitions and places. Edges only point from places to transitions and from transitions to places (i.e., N is a bipartite graph). A marking is a mapping μ: P → $N$ that assigns to every place p a number μ(p) of tokens. Intuitively, the idea is that a transition t consumes tokens via edges leading from places to t and it produces tokens via edges leading from t to some places. We make this more precise, as follows. For a place p and a transition t let input_t(p) = |in(t) ∩ out(p)| be the number of times p occurs as an input place of t. Similarly, let output_t(p) = |out(t) ∩ in(p)| be the number of times p occurs as an output place of t. For a given marking μ, a transition t can fire if μ(p) ≥ input_t(p) for each place p, that is, if there are enough tokens on the input places of t. In this case, the firing of t yields the marking μ′ given by μ′(p) = μ(p) − input_t(p) + output_t(p) for all p ∈ P.

Note that a place p can be an input and output place of t at the same time—that is, we may have input_t(p) > 0 and output_t(p) > 0. A simple example of a Petri net consisting of one transition together with its input and output places is shown in Figure 10, where the bar represents the transition and the circles represent places. The transition can fire if the topmost place contains at least one token. If it does fire, the token on the topmost place is immediately reproduced. At the same time, four additional tokens are placed on the places at the bottom, namely, two on the place in the middle and one on each of the other two places.

Naturally, a firing sequence is a sequence of admissible firings. It transforms an initial marking into a final marking. The Petri net reachability problem is the following problem: Given a Petri net and two markings μ, μ′, is μ′ reachable from μ via some firing sequence? This problem is known to be decidable, but no solution with a primitive recursive running time is known (Reutenauer 1990; Esparza and Nielsen 1994). Fortunately, for our purpose it suffices to consider the case where both μ and μ′ are equal to the zero marking 0, that is, μ(p) = μ′(p) = 0 for all places p. If 0 is reachable from itself in a Petri net N via a nonempty firing sequence, then we say that N is structurally cyclic, because then it holds for all markings μ that μ is reachable from itself. Drewes and Leroux (2015) show that it is decidable in polynomial time whether a Petri net is structurally cyclic.

We can reduce the emptiness problem for DAG automata M to the question of whether a Petri net is structurally cyclic, as follows. Every state of the DAG automaton becomes a place of the Petri net N and every transition $t=({q1,…,qm}σ→{r1,…,rn})$ of M becomes a transition of N in an obvious way: for 1 ≤ i ≤ m, there is an edge pointing from q_i to t, and for 1 ≤ j ≤ n, there is one pointing from t to r_j.

To argue for the correctness of the construction, let us consider DAGs that are partial in the sense that, for some edges e, there is no node v with e ∈ in(v). Such edges are “downward dangling.” Now, given a firing sequence starting with the empty marking, we can inductively construct a run on a corresponding partial DAG. The initial empty marking of N corresponds to the empty DAG (with no nodes and zero dangling edges). After some firings the Petri net has reached a marking μ and we have inductively constructed a partial DAG D and a run ρ on D such that for each state q, there are exactly μ(q) dangling edges e with ρ(e) = q. Now suppose that, in N, a transition fires, which was obtained from transition ${q1,…,qm}σ→{r1,…,rn}$ of the DAG automaton M. To reflect the firing of t, we add a node v labeled by σ to D and choose previously dangling edges e₁, …, e_m with ρ(e_i) = q_i as incoming edges of v; n new outgoing dangling edges e₁′, …, e_n′ are attached to v, and ρ is extended by defining ρ(e_i′) = r_i for 1 ≤ i ≤ n. Clearly, D is a DAG without dangling edges if μ = 0. Thus, $M≠∅$ if 0 is reachable from itself in N. In a similar way, if M accepts a DAG D, a run of M on D can easily be turned into a nonempty firing sequence of N (under the top–down interpretation of runs) that turns 0 into itself. Thus, we have reduced the emptiness problem for DAG automata to the problem of deciding whether a Petri net is structurally cyclic. Clearly, the reduction can be computed in polynomial time (and, in fact, in logarithmic space). Using the main result of Drewes and Leroux (2015) mentioned earlier, we conclude that the emptiness problem for DAG automata is decidable in polynomial time.

We consider the recognition problem for unweighted DAG automata: For a DAG automaton M and a DAG D, does M accept D? This problem turns out to be NP-complete even in case M is fixed (i.e., instead of both M and D, only D is the input). (In theoretical computer science, the variant where M is part of the input is called the uniform membership problem; accordingly, the one where M is fixed is the potentially easier non-uniform one.) The situation is similar to that of the recognition problem based on the hyperedge replacement grammar introduced in Section 4.2, which is NP-complete even for a fixed grammar (Aalbersberg, Rozenberg, and Ehrenfeucht 1986; Lange and Welzl 1987). On the positive side, as we shall see in Section 6, recognition by a (fixed) DAG automaton can be done in polynomial time for input graphs of bounded treewidth, which is encouraging in view of Table 1.

It is easy to see that recognition is in NP even if the automaton is part of the input: We can nondeterministically “guess” an assignment of states to the edges of D and check in linear time whether it constitutes an accepting run of M. Next, we show that recognition is NP-complete. Like Fujiyoshi (2010), we do this by reduction from SAT, but the reduction is different (because our DAG automata differ essentially from his).

Because we want to prove NP-completeness of the non-uniform membership problem, that is, for a fixed DAG automaton, we construct a single DAG automaton M and a reduction that turns any propositional formula φ into a DAG D_φ that is accepted by M if and only if φ is satisfiable. We first define D_φ. Thus, assume that we are given a propositional formula φ (which we do not require to be in conjunctive normal form). We use the alphabet Σ = {true, ∧, ∨, ¬, x}. First, we construct in the obvious way the tree T_φ corresponding to φ (where every occurrence of a variable x_i is represented by a node labeled x). We then add a special root node labeled true on top of the tree. Intuitively, the root node represents the claim that φ evaluates to true under an appropriate assignment. Finally, for every variable x_i, if there are n + 1 nodes u₀, …, u_n in T_φ that represent the occurrences of x_i in φ from left to right, we add edges from u_j−1 to u_j for j = 1, …, n. Thus, all nodes representing the same variable are linked together in a chain.

We can easily construct a DAG automaton M that, for every formula φ, accepts DAG D_φ if and only if φ is satisfiable. The automaton has just two states, t and f, to compute a truth value for each node in a consistent way by means of a guess-and-verify technique. The only transition for true is $∅true→{t}$⁠.

Note that, no matter whether we construct runs top–down or bottom–up, there is always nondeterminism involved. Under the top–down view, the transitions for ∧ and ∨ are nondeterministic (reflecting the fact that ∧ and ∨ are not injective) whereas those for x are deterministic. Conversely, under the bottom–up view, the transitions for x become nondeterministic whereas those for ∧ and ∨ become deterministic (because ∧ and ∨ are functions). Intuitively, the top–down process corresponds to guessing the values of subtrees and verifying consistency. In contrast, the bottom–up process guesses an assignment of truth values and computes the resulting truth value of φ deterministically in order to check that it results in true. In both cases, the outlined computational difficulty is preserved.

We provide an algorithm for a more general problem than the recognition problem for unweighted DAG automata: Given a weighted DAG automaton M and a DAG D, what is the total weight (in the semiring $K$⁠) of all runs of M for D? This includes in particular the recognition problem, because unweighted DAG automata are a special case of general DAG automata, as explained at the end of Section 3.1. We also obtain an analogue of the Viterbi algorithm if we define ⊗ and ⊕ to be multiplication and maximum. In Section 5.3 we will also discuss how to use this algorithm for learning transition weights from data.

We have already discussed in Example 3 how our DAG automata generalize finite automata for strings. In order to introduce our algorithm for DAG automata, we therefore consider the analogous problem for finite automata: Given an input string w, find the total weight of all runs of a nondeterministic weighted finite automaton M on w. Let Q be the state set of M. A naïve algorithm for this problem would consider all possible assignments of states in Q to the |w| + 1 inter-symbol positions of w, under the restriction that the first position is assigned the unique starting state for M. For each such assignment, we then check against M’s transitions that it corresponds to a run of M and, if this is the case, we add in the weight of that run. The number of possible assignments is |Q|^|w| and each assignment can clearly be checked in time $O|w|$⁠. If we assume that the semiring operations can be computed in constant time, the algorithm runs in time $O|Q||w||w|$⁠.

A better algorithm, the forward algorithm (Baum 1972), uses dynamic programming to run in polynomial time in the size of both w and M. This is reported in Algorithm 1. We view w as a sequence of tokens w_i from the alphabet of M. Symbols s and F denote the initial state and the final state set, respectively, of M. Symbol δ denotes the transition function, mapping a pair of states and an input symbol from M to a weight. For instance, δ(q, w_i, r) is the weight of the transition that takes M from state q to state r upon reading token w_i.

The algorithm processes w from left to right, computing the weights of larger and larger prefixes of w. More precisely, for each prefix w₁w₂ ⋯ w_i of w and for each state r ∈ Q, we compute the sum of the weights of all runs of M that start in s, read w₁w₂ ⋯ w_i, and end up in r. This quantity is then stored in a chart entry α[i, r], for future reuse. In fact, the basic idea underlying Algorithm 1 is that α[i, r] can be computed as a function of all quantities α[i − 1, q], q ∈ Q, combined with all possible transitions of M over token w_i, using a recursive relation. We call each chart entry α[i, r] a partial analysis of w. Observe that each partial analysis of w is uniquely identified by the inter-symbol position i we have reached on w, and by the state r we have reached on M.

The complexity analysis of Algorithm 1 is rather straightforward. Considering the two embedded for-loops and the summation performed at the inner loop, we get a running time of $O|Q|2|w|$⁠.

We are now in a position to discuss the same problem for DAG automata. Let D be an input DAG and let M be our DAG automaton with state set Q. In order to strengthen the similarity with the string case, we view the nodes of D like the tokens of w and the edges of D like the inter-symbol positions of w. A naïve algorithm, similar to the one for finite automata, can be developed for computing the total weight for all runs of M on D. We iterate over all possible assignments of states from Q to edges in E, that is, over all runs, and sum up their weights. The total number of runs is |Q|^|E|, and the weight of each run can be checked in time $O|E|$⁠. We thus conclude that the algorithm runs in time $O|Q||E||E|$⁠.

Once again, we can do much better by using dynamic programming. The main difference with respect to the string case is that the tokens of D are now organized in some partial order, so we can no longer parse the input from left to right. To deal with this, our algorithm assumes a total ordering of the edges of D, which is provided along with D, and parses D accordingly, as we explain now.

Informally, our parsing algorithm consists of the following two phases.

• • 

  First, we make a partial analysis for each node v of D. Each partial analysis records what states the incoming edges might be in and what states the outgoing edges might be in, together with a weight.

• • 

  Second, we merge partial analyses into larger and larger partial analyses. For each edge e (following the total ordering of edges provided as input), we contract it, replacing its source node src(e) and target node tar(e) with a new node z. We then retrieve partial analyses associated with src(e) and tar(e) and merge them into new partial analyses associated with z. This process is repeated, ending when all of D has been contracted to a single node with a single analysis. The weight of this analysis is the weight of all the runs on D.

In order to gain a better understanding of these ideas, we discuss a simple example, before providing a precise specification of the algorithm itself.

Just as DAG automata generalize traditional finite automata defined on strings, our DAG parsing algorithm generalizes Algorithm 1. To see this, imagine applying our DAG parsing algorithm to a DAG consisting of a single long chain of edges. If the edges are contracted in order from left to right, our DAG parsing algorithm performs the same computation as Algorithm 1, building partial analyses for longer and longer prefixes of the chain. Of course, under some other ordering of the edges, a partial analysis may correspond to a sub-chain of D that is not a prefix. As we will see later, the choice of ordering does affect the overall computational complexity of the algorithm.

We note that the problem of summing over state assignments is an instance of the general problem of weighted constraint satisfaction, where each edge in our input DAG is a variable whose values are states of M, and each node in our DAG is a weighted constraint, with weights specified by the transitions in the automaton. We can solve this problem using general techniques for graphical models (Jensen, Lauritzen, and Olesen 1990; Shafer and Shenoy 1990); the algorithm here is an adaptation of the variable elimination algorithm to our setting.

The pseudocode of our recognition algorithm for DAG automata is reported in Algorithm 2. It uses some additional notation, which we define here. For a node v of D, let star(v) = in(v) ∪ out(v). In words, star(v) is the set of edges connecting v to its neighbor nodes. In order to assign states to these edges, we use functions $f:star(v)→Q$⁠. For an edge set I ⊆ star(v), we also write f|_I to denote f restricted to I, and f[I] to denote the multiset of all f(e) such that e ∈ I, i.e., if I = {e₁, …, e_n} then f[I] = {f(e₁), …, f(e_n)}.

The complexity of Algorithm 2 depends both on the structure of the input DAG and the order in which we contract its edges. More precisely, the complexity of the optimal edge ordering is determined by the treewidth (see Definition 2 in Section 3.1) of the line graph of D.

Because we want to make use of the treewidth of a line graph, and line graphs are hypergraphs (see Section 4.2), we extend the notion of tree decompositions to hypergraphs in the obvious way: For every hyperedge e, there must be a bag of the tree decomposition that contains all of the attached nodes of e. Note that the bags of the tree decomposition of $LG(D)$ contain nodes of $LG(D)$⁠, which correspond to edges of D.

To obtain an optimal edge ordering, first find an optimal tree decomposition, that is, a tree decomposition with minimal width, which we call k. This takes time $O(|E|k+2)$ using the algorithm of Arnborg, Corneil, and Proskurowski (1987). We can also take advantage of the various heuristics and approximation algorithms that are available for treewidth (Gogate and Dechter 2004; Feige, Hajiaghayi, and Lee 2005); as mentioned in Section 2, these heuristics work extremely well on AMR.

Second, visit the bags bottom–up. For each bag b, contract the edges that are in b but not in the parent of b. It can be shown (Rose 1970; Arnborg, Corneil, and Proskurowski 1987) that the maximum degree of any node created by an edge contraction is k. This means that there are at most (k + 1) edges in star(u) ∪ star(v), and at most |Q|^k+1 possible state assignments to those edges in the innermost loop of the algorithm.

We briefly discuss here the problem of learning the weights of our DAG automata, though this in itself is a broad topic worthy of further research. Throughout this section, we assume that our semiring of weights $K$ is the semiring of real numbers, with the usual addition and multiplication operations.

The gradient of LL is:

Unfortunately, we cannot derive a closed-form solution for the zeros of Equation (3). We therefore use gradient ascent. In CRF training for finite automata, the expectation in Equation (3) is computed efficiently using the forward-backward algorithm; for DAG automata, the expectation can be computed analogously. Algorithm 2 provides the bottom–up procedure for computing a chart of inside weights. If we compute weights in the derivation forest semiring (Goodman 1999), in which ⊗ creates an “and” node and ⊕ creates an “or” node, the resulting and/or graph has the same structure as a CFG parse forest generated by CKY, so we can simply run the inside-outside algorithm (Lari and Young 1990) on it to obtain the desired expectations. Alternatively, we could compute weights in the expectation semiring (Eisner 2002; Chiang 2012). Because the log-likelihood LL is concave (Boyd and Vandenberghe 2004), gradient ascent is guaranteed to converge to the unique global maximum.

Let M be a DAG automaton with set of states Q and let D be an input DAG with set of edges E_D. As we have seen in Section 5.2, the time complexity of Algorithm 2 is $O|ED|⋅|Q|tw(LG(D))+1$⁠, where $tw(LG(D))$ is the treewidth of the line graph $LG(D)$⁠. By definition, $tw(LG(D))$ is at least the degree of nodes of D minus one, because every node of degree k is turned into a hyperedge of size k that must be covered by some bag. The treewidth of $LG(D)$ can therefore be quite large. We can improve Algorithm 2 by binarizing both the input DAG and, accordingly, the transitions of our DAG automaton. In this section we develop specialized techniques for the binarization of DAGs and for the binarization of transitions of DAG automata, and prove some relevant properties. Our techniques will further be developed in Section 7 to process DAG languages with unbounded node degree.

A binary DAG is one in which each node has at most two incoming edges and one outgoing edge, or else one incoming edge and two outgoing edges. In order to produce a binary DAG D′ from a source DAG D, we introduce a construction that replaces every node of D with a treelet consisting of fresh nodes, and connects the edges of D to these fresh nodes in such a way that the resulting DAG D′ is binary. Furthermore, D′ preserves all of the information in D, in a way that will be specified later.

Our technique is a generalization of what is known from the theory of tree automata, in particular unranked tree automata, where nodes of any rank are encoded by subtrees entirely consisting of binary nodes; see Comon et al. (2002, Section 8.3) for details. We introduce the idea underlying our DAG binarization technique by discussing a simple example.

Because the leaves of T_v correspond one-to-one to the edges of D incident on v, they can be used as “docking places” for the original edges. More precisely, each edge e in D such that src(e) = v and tar(e) = v′ is used in D′ to connect some leaf of T_v to some leaf of T_v′. Note that, according to this construction, the edge set of D′ can be partitioned into the set of fresh edges coming from the treelets and the set of edges coming from the source DAG D; the latter are exactly those edges whose source nodes carry a label σ ∈ Σ, and are drawn with thick lines in Figure 14(b).

In the following, the specific topology of each treelet T_v will be obtained from a tree decomposition of D. Because the leaves of T_v have only one parent and no child, the construction yields a binary DAG.

Along with DAG binarization, we must also replace each transition t of the DAG automaton with a set of “binary” transitions that process the nodes of the binarized graph. The binary transitions have at most two states in the left-hand side and one state in the right-hand side, or else at most one state in the left-hand side and two states in the right-hand side. Again, we demonstrate the intuitive idea underlying the construction by means of a simple example.

For the binary transitions we use the states p, q, r, s appearing in t, along with some new states of the form (I, O), where I is a subset of the multiset in the left-hand side of t and O is a subset of the multiset in the right-hand side of t. States p, q, r, s will be assigned to the edges of D′ that were also present in D, drawn with thick lines in Figure 15(b). States of the form (I, O) will be assigned to the fresh edges of the treelet T_v.

Consider an edge e of T_v. Let T be the subtree of T_v whose root node is the target node of e. Let also $ST$ be the set of edges of D′ that are connected to the leaves of T, not including edges internal to T. When viewed as edges from D, the edges in $ST$ are a subset of the edges incident on v. Informally, the meaning of a state (I, O) being assigned to edge e is that we expect to find the states in I on the edges within $ST$ that are incoming at v, and likewise we expect to find the states in O on the edges within $ST$ that are outgoing at v.

Now that we have seen an intuitive description of the procedures for binarizing a DAG and for binarizing a DAG automaton, we can outline the improved version of Algorithm 2:

• 1. 

  For each transition in the input DAG automaton M, construct the corresponding set of binary transitions to form a new automaton M′.

• 2. 

  Binarize the input DAG D into DAG D′.

• 3. 

  Run Algorithm 2 on the binarized DAG D′ with automaton M′.

Let D be some input DAG and let D′ be a binarized DAG derived from D. We have already discussed in Section 2 how AMR structures have very small treewidth. For this reason, in the following discussion we use as a term of comparison quantity tw(D), the treewidth of D.

When processing D′, the running time of Algorithm 2 depends on $tw(LG(D′))$⁠, the treewidth of the line graph of D′, which in turn depends on the choice of the binarization of D. There are several ways in which we can binarize D, resulting in different values of $tw(LG(D′))$⁠. However, a bad choice of binarization for D may result in $tw(LG(D′))$ much larger than tw(D). Our objective should therefore be to binarize D in such a way that $tw(LG(D′))$ is not much larger than tw(D). We provide an algorithm for constructing D′ from a tree decomposition of D, and we show that tw(D′) ≤ tw(D) + 1 and $tw(LG(D′))≤2(tw(D)+1)$⁠.

In what follows, we exclude from our DAG automata transitions of the form $∅a→∅$⁠, which only accept DAGs consisting of a single isolated node. Clearly, this is an uncritical assumption because $DΣ$ contains only |Σ| of these DAGs. This assumption is similar to the exclusion of the empty string from context-free languages when parsing with the CKY algorithm that uses context-free grammars in Chomsky normal form (Aho and Ullman 1972).

We will have to refer to the components of different graphs and tree decompositions. To disambiguate the notation, we will index the components of such an object by the object in question. For example, the edge set of a DAG D will be referred to as E_D, the source of an edge e ∈ E_D by src_D(e), and the set of bags of a tree decomposition T by V_T.

In this section we consider tree decompositions of DAGs that are in a special form that we call binary. This has the advantage of greatly simplifying the binarization construction. A tree decomposition T of a DAG D is binary if both of the following conditions are met:

• • 

  every bag of T has at most two children;

• • 

  each edge e ∈ E_D is explicitly assigned to a unique leaf b of T.

Our method of binarization is illustrated in Figure 16 and explained in the following.⁴ Let $D∈DΣ$⁠. For every symbol σ ∈ Σ, we let σ′ be a fresh copy of σ. In the binarized DAG, every node v ∈ V_D will be represented by a treelet, each of whose nodes is labeled by σ or σ′. To define the binarized version of D, consider a binary tree decomposition T of D. By the definition of tree decomposition, the subtree of T induced by {b ∈ V_T ∣ v ∈ cont_T(b)} forms itself a (binary) tree. Let us denote this treelet by T_v. To distinguish between the copies of b ∈ V_T appearing in the different treelets T_v such that v ∈ cont_T(b), we let [v, b] denote the copy of b in T_v. In DAG D of Figure 16, its nodes x, y, u, v are shown instead of their node labels. In the tree decomposition T, the bags b are identified with their Gorn addresses and the boxes show their contents.

Binarization replaces each node v ∈ V_D by T_v. Formally, D_T is the DAG obtained from the union of all T_v, for v ∈ V_D, by labeling the nodes and inserting the edges of D as follows:

• • 

  For every node [v, b] of T_v, $labDT([v,b])=labD(v)$ if b is a leaf of T and $labDT([v,b])=labD(v)′$ otherwise.

• • 

  Let e ∈ E_D with (src_D(e), tar_D(e)) = (x, y) and $b=edgT−1(e)$⁠. Then T_x and T_y contain the leaves [x, b] and [y, b], respectively, and we set $srcDT(e)=[x,b]$ and $tarDT(e)=[y,b]$⁠.

The construction is illustrated in the middle of Figure 16. (The figure does not show node labels, however. For example, in the sub-DAG resulting from T_x, if lab_D(x) = σ then the label of [x, ϵ], [x, 1], and [x, 2] is σ′ and the label of [x, 1.2] and [x, 2.2] is σ.)

Clearly, D_T can be turned into D by contracting each sub-DAG T_v into a single node (with the appropriate label). D_T is binary because the T_v are treelets and each leaf [v, b] is attached to exactly one of the original edges e ∈ E_D, namely, to edg_T(b). In particular, D_T does not have cycles.⁵

Note that, as D_T depends on T, one of the effects of binarization is that DAGs representing the same semantic information are not necessarily isomorphic anymore. However, this is not a severe disadvantage because binarization is only a technical tool that allows us to derive efficient algorithms and, in Section 7, transfer results from the ranked case to the unranked one.

Proof. As a first step, consider the tree T′ that is identical to T, but with the content of bag b ∈ V_T = V_T′ being given by cont_T′(b) = {[v, b] ∣ v ∈ cont_T(b)}. Intuitively, (the content of bags of) T′ is obtained by overlaying the different copies T_v of T; see again Figure 16. With this definition, T′ is not a tree decomposition of D_T yet, but we note that |cont_T′(b)| = |cont_T(b)| for all b ∈ V_T and that every edge e ∈ E_D can be assigned to the bag $b=edgT′−1(e)$ because, in D_T, e connects the two nodes in cont_T′(b). Furthermore, every node [v, b] of D_T occurs in precisely one bag: [v, b] ∈ cont_T′(b). Hence, T′ is a tree decomposition of width k except for the fact that those edges of D_T which are arcs of the treelets T_v are not covered by any bags. For this, we shall add intermediate bags to T′ in order to construct a valid tree decomposition T″ of D_T.

Proof. For a node [u, b] of D_T (where u ∈ V_D and b ∈ V_T) which is not the root of treelet T_u, let e(u, b) denote the arc of T_u (which is an edge of D_T) whose target is [u, b]. As an illustration, Figure 17 (top) shows the line graph of the binarized DAG D_T from Figure 16. The nodes of D_T have become hyperedges drawn as boxes and the edges have become nodes drawn as bullets. The node e(x, 1.1), for instance, is the one on top of the hyperedge [x, 1.1].

Consider a bag b of T and let D_T(b) be the sub-DAG of D_T induced by the treelet nodes [v, c] such that c is a descendant of b (or b itself) and v ∈ cont_T(c). Let, furthermore, $LG(DT,b)$ be the subgraph of $LG(DT)$ having as nodes the edges of D_T(b) as well as all edges e(v, b) with v ∈ cont_T(b), and as hyperedges the nodes of D_T(b). As an example, Figure 17 (middle) indicates $LG(DT,1)$ by means of thick lines.