Synchronous Context-Free Grammars and Optimal Parsing Strategies

Synchronous context-free grammars (SCFGs) generalize context-free grammars (CFGs) to generate two strings simultaneously. The formalism dates from the early days of automata theory; it was developed under the name syntax-direct translation schemata to model compilers for programming languages (Lewis and Stearns 1968; Aho and Ullman 1969). SCFGs are widely used today to model the patterns of re-ordering between natural languages, and they form the basis of many state-of-the-art statistical machine translation systems (Chiang 2007).

Despite the fact that SCFGs are a very natural extension of CFGs, and that the parsing problem for CFGs is rather well understood nowadays, our knowledge of the parsing problem for SCFGs is quite limited, with many questions still left unanswered. In this article we tackle one of these open problems.

Unlike CFGs, SCFGs do not admit any canonical form in which rules are bounded in length (Aho and Ullman 1972), as for instance in the well-known Chomsky normal form for CFGs. A consequence of this fact is that the computational complexity of parsing with SCFG depends on the grammar. More precisely, for a fixed SCFG, parsing is polynomial in the length of the string. The degree of the polynomial depends on the length of the grammar's rules, the re-ordering patterns represented by these rules, as well as the strategy used to parse each rule. The complexity of finding the best parsing strategy for a fixed SCFG rule has remained open. This article shows that it is NP-hard to find the binary parsing strategy having the lowest space complexity. We also consider the problem of finding the parsing strategy with the lowest time complexity, and we show that it would require progress on long-standing open problems in graph theory either to find a polynomial algorithm or to show NP-hardness.

The parsing complexity of SCFG rules increases with the increase of the number of nonterminals in the rule itself. Practical machine translation systems usually confine themselves to binary rules, that is, rules having no more than two right-hand side nonterminals, because of the complexity issues and because binary rules seem to be adequate empirically (Huang et al. 2009). Longer rules are of theoretical interest because of the naturalness and generality of the SCFG formalism. Longer rules may also be of practical interest as machine translation systems improve.

For a fixed SCFG, complexity can be reduced by factoring the parsing of a grammar rule into a sequence of smaller steps, which we refer to as a parsing strategy. Each step of a parsing strategy collects nonterminals from the right-hand side of an SCFG rule into a subset, indicating that a portion of the SCFG rule has been matched to a subsequence of the two input strings, as we explain precisely in Section 2. A parsing strategy is binary if it combines two subsets of nonterminals at each step to create a new subset. We consider the two problems of finding the binary parsing strategy with optimal time and space complexity. For time complexity, there is no benefit to considering parsing strategies that combine more than two subsets at a time. For space complexity, the lowest complexity can be achieved with no factorization whatsoever; however, considering binary parsing strategies can provide an important trade-off between time and space complexity. Our results generalize those of Crescenzi et al. (2015), who show NP-completeness for decision problems related to both time and space complexity for linear parsing strategies, which are defined to be strategies that add one nonterminal at a time.

Our approach constructs a graph from the permutation of nonterminals in a given SCFG rule, and relates the parsing problem to properties of the graph. Our results for space complexity are based on the graph theoretic concept of carving width, whose decision problem is NP-complete for general graphs. Section 3 relates the space complexity of the SCFG parsing problem to the carving width of a graph constructed from the SCFG rule. In Section 4, we show that any polynomial time algorithm for optimizing the space complexity of binary SCFG parsing strategies would imply a polynomial time algorithm for carving width of general graphs. Our results for time complexity are based on the graph theoretic concept of treewidth. In Section 5, we show that any polynomial time algorithm for the decision problem of the time complexity of binary SCFG strategies would imply a polynomial time constant–factor approximation algorithm for the treewidth of general graphs. In the other direction, NP-completeness of the decision problem of the time complexity for SCFG would imply the NP-completeness of treewidth of general graphs of degree six. These are both long-standing open problems in graph theory.

In this section we informally introduce the notion of synchronous context-free grammar and define the computational problem that we investigate in this article. We assume the reader to be familiar with the notion of context-free grammar, and only briefly summarize the adopted notation. Throughout this article, for any positive integer n we write [n] to denote the set {1, … , n}.

As usual, a CFG has a finite set of rules having the general form A → α, where A is a nonterminal symbol to be rewritten and α is a string of nonterminal and terminal symbols. A synchronous context-free grammar is a rewriting system based on a finite set of so-called synchronous rules. Each synchronous rule has the general form [A₁ → α₁, A₂ → α₂], where A₁ → α₁ and A₂ → α₂ are CFG rules. By convention, we refer to A₁ → α₁ and A₂ → α₂ as the Chinese and English components of the synchronous rule, respectively. Furthermore, α₁, α₂ must be synchronous strings. This means that there exists a bijection between the occurrences of nonterminals in α₁ and the occurrences of nonterminals in α₂, and that this bijection is explicitly provided by the synchronous rule. Nonterminal occurrences that correspond under the given bijection are called linked nonterminal occurrences, or just linked nonterminals when no ambiguity arises. We assume that linked nonterminals always have the same label.

Let S be a special nonterminal of our SCFG, which we call the starting nonterminal. Using our notion of derivation, one can start from [, ] and attempt to derive a pair of strings [u, v] entirely composed of terminal symbols. Whenever this is possible, we say that [u, v] is in the translation generated by the SCFG, meaning that v is one of the possible translations of u.

Example 1

Consider the following list of synchronous rules, where symbols s_i are rule labels to be used later as references

What follows is a valid derivation of the SCFG, obtained by rewriting at each step the linked nonterminals with the Chinese component occurring at the left-most position in the sentential form

It is not difficult to see that the given SCFG provides the translation {[a^pb^pc^qd^q, d^qc^qb^pa^p] | p, q ≥ 1}.

One can also associate SCFG derivations with parse trees, in a way very similar to what we do with CFG derivations. More precisely, an SCFG derivation is represented as a pair of parse trees, each generating one component in the derived string pair. Furthermore, linked nonterminals in the SCFG derivation are annotated in the parse trees to indicate the place of application of some synchronous rule.

Example 2

The SCFG derivation for string pair [aaabbbccdd, ddccbbbaaa] in Example (1) is associated with the following tree pair:

One common view of SCFGs is that each synchronous rule implements a permutation of the (occurrences of the) nonterminal symbols in the two rule components. More precisely, the nonterminals in the right-hand side of the Chinese rule component are re-used in the right-hand side of the English component, but with a different ordering, as defined by the bijection associated with the rule. This reordering is at the core of the translation schema implemented by the SCFG. We will often use the permutation view of a synchronous rule in later sections.

As already mentioned, we assume that linked nonterminals in a synchronous rule have the same label. In natural language processing applications, this is by far the most common assumption, but one might also drop this requirement. The results presented in this article do not rest on such a restriction.

The recognition and the parsing problems for SCFGs are natural generalizations of the same problems defined for CFGs. Given an SCFG of interest, in the recognition problem one has to decide whether an input pair [u, v] can be generated. In addition, for the parsing problem one has to construct a parse forest, in some compact representation, with all tree pairs that the SCFG produces when generating [u, v]. When dynamic programming techniques are used, it turns out that recognition and parsing algorithms are closely related, since the elementary steps that are performed during recognition can be memoized and later used to construct the desired parse forest.

Despite the similarity between CFGs and SCFGs, it turns out that there is a considerable gap in the space and time complexity for recognition and parsing based on these two classes. More specifically, for a fixed CFG, we can solve the recognition problem in time cubic in the length of the input string, using standard dynamic programming techniques. In contrast, for a fixed SCFG we can still recognize the input pair [u, v] in polynomial time, but the degree of the polynomial depends on the specific structure of the synchronous rules of the grammar, and can be much larger than in the CFG case. A similar scenario also holds for the space complexity. The reason for this gap is informally explained in what follows.

In the investigation of the recognition problem for CFGs and SCFGs, a crucial notion is the one of parsing strategy. Recognition algorithms can be seen as sequences of elementary computational steps in which two parse trees that generate non-overlapping portions of the input are combined into a larger parse tree, according to the rules of the grammar. We call parsing strategy any general prescription indicating the exact order in which these elementary steps should be carried out. Generally speaking, parsing strategies represent the order in which each parsing tree of the input is constructed.

For the CFG case, let us consider the standard top–down chart parsing algorithm of Kay (1980). The algorithm adopts a parsing strategy that visits the nonterminals in the right-hand side of a rule one at a time and in a left-to-right order, combining the associated parse trees accordingly. Parse trees are then compacted into so-called chart edges, which record the two endpoints of the generated substring along with other grammar information. Chart parsing can be generalized to SCFGs. However, in the latter case parse trees no longer generate single substrings: They rather generate tuples with several substrings. This can be ascribed to the added component in synchronous rules and to the re-ordering of the nonterminals across the two components.

Let us call fan-out the number of substrings generated by a parse tree (this notion will be formally defined later). It is well-known among parsing practitioners that the fan-out affects the number of stored edges for a given input string, and is directly connected to the space and time performance of the algorithm. A binary parsing strategy of fan-out φ has space complexity and time complexity at most where n is the sentence length (Seki et al. 1991; Gildea 2010). If we adopt the appropriate parsing strategy, we can reduce the fan-out, resulting in asymptotic improvement in the space and time complexity of our algorithm. To illustrate this claim we discuss a simple example.

Example 3

Consider the synchronous rule

The permutation associated with this rule is schematically visualized in Figure 1.

A naive parsing strategy for rule s would be to collect the nonterminals one at a time and in ascending order of k. For instance, at the first step we combine and , constructing a parse tree with fan-out 3, as seen from Figure 1. The worst case is attested when we construct the parse tree consisting of occurrences , which has a fan-out of 4.

Alternatively, we could explore more flexible parsing strategies. An example is depicted in Figure 2, where each elementary step is represented as an internal node of a tree. This time, at the first step (node n₁) we combine and , as depicted in Figure 3a. At the second step (node n₃), we combine the result of node n₁ and , again constructing a parse tree with fan-out three, as depicted in Figure 3b, and so on. This strategy is non-linear, because it might combine parses containing more than one nonterminal occurrence each; see Figure 3c. From Figure 1 it is not difficult to check that at each node n_k the constructed parse has fan-out bounded by 3. We therefore conclude that our second strategy is more efficient than the left-to-right one.

This example shows that the maximum fan-out needed to parse a synchronous rule depends on the adopted parsing strategy. In order to optimize the computational resources of a parsing algorithm, we need to search for a parsing strategy that minimizes the maximum fan-out of the intermediate parse trees. The problem that we investigate in this article is therefore the one of finding optimal parsing strategies for synchronous rules, where in this context optimal means with a value of the maximal fan-out as small as possible.

In this section we provide a mathematical definition for the concepts that we have informally introduced in the previous section. We need to do so in order to be able to precisely define the computational problem that is investigated in this article.

Assume a synchronous context-free rule s with r ≥ 2 linked nonterminals. We need to address occurrences of nonterminal symbols within s. We write 〈1, i〉 to represent the ith occurrence (from left to right) in the right-hand side of the Chinese component of s. Similarly, 〈2, i〉 represents the ith occurrence in the right-hand side of the English component of s. Each pair 〈h, i〉, h ∈ [2] and i ∈ [r], is called an occurrence.

We assume without loss of generality that the nonterminals of the Chinese component are indexed sequentially. That is, the index of 〈1, i〉 is i. Let π be the permutation over set [r] implemented by s, meaning that 〈2, i〉 is annotated with index π(i), and therefore is co-indexed with 〈1, π(i)〉. As an example, in rule (1) we have r = 8 and π(1) = 5, π(2) = 7, π(3) = 3, etc. Under this convention, each pair (〈1, π(i)〉, 〈2, i〉) and, equivalently, each pair (〈1, i〉, 〈2, π⁻¹(i)〉), i ∈ [r] is called a linked pair.

A parsing strategy for s is a rooted, binary tree τ_s with r leaves, where each leaf is a linked pair (〈1, π(i)〉, 〈2, i〉). As already explained, the intended meaning is that each internal node of τ_s represents the operation of combining the linked pairs below the left node with the linked pairs below the right node. These operations must be performed in a bottom–up fashion.

Let n be an internal node of τ_s, and let τ_n be the subtree of τ_s rooted at n. We write y(n) to denote the set of all occurrences 〈h, i〉 appearing in the linked pair of some leaf of τ_n. We say that occurrence 〈h, i〉 ∈ y(n) is a right boundary of n if i = r or if i < r and 〈h, i + 1〉 ∉ y(n). Symmetrically, we say that 〈h, i〉 ∈ y(n) is a left boundary if i = 1 or if i > 1 and 〈h, i − 1〉 ∉ y(n). Note that the occurrences 〈1, 1〉 and 〈2, 1〉 are always left boundaries, and 〈1, r〉 and 〈2, r〉 are always right boundaries. We let bd(n) be the total number of right and left boundaries in y(n).

In this section, we relate the fan-out of parsing strategies for an SCFG rule to the properties of a specific graph derived from the rule's permutation.

We begin by introducing some basic terminology. A multiset is a set where there can be several occurrences of a given element. We use ⨄ as the merge operation defined for multisets: This operation preserves the number of occurrences of the merged multisets. As usual, we denote an undirected graph as a pair G = (V, E) where V is a finite set of vertices and E is a set of edges, with each edge consisting of an unordered pair of vertices. A multigraph is a graph that uses a multiset of edges. This means that a multigraph can have several occurrences of an edge impinging on a pair of vertices.

A cyclic permutation multigraph is a multigraph G = (V, A ⨄ B) such that both P_A = (V, A) and P_B = (V, B) are Hamiltonian cycles, that is, cycles that visit all the vertices in V exactly once. In the following, the edges in A will be called red and the edges in B will be called green. Our definition is based on the permutation multigraphs of Crescenzi et al. (2015), which differ in that they consist of two acyclic Hamiltonian paths.

In this article, we use cyclic permutation multigraphs to encode synchronous rules. Let s be a synchronous rule with r ≥ 2 linked pairs. Let also (〈1, 0〉, 〈2, 0〉) be a special linked pair representing the left-hand side nonterminal of s. We construct the cyclic permutation multigraph M_s, representing s, as follows. The linked pairs (〈1, π(i)〉, 〈2, i〉), i ∈ [r], along with (〈1, 0〉, 〈2, 0〉), form the vertices of M_s. The red cycle of M_s begins with (〈1, 0〉, 〈2, 0〉), then follows the order in which nonterminals occur in the Chinese component of s, and finally returns to (〈1, 0〉, 〈2, 0〉). Similarly, the green cycle of M_s begins and ends with (〈1, 0〉, 〈2, 0〉), and follows the order in which nonterminals occur in the English component.

In the other direction, given any cyclic permutation multigraph, we can derive a corresponding SCFG rule by, first, choosing an arbitrary vertex as representing the left-hand side nonterminal, and then following the red cycle to obtain the sequence of Chinese nonterminals, and finally following the green cycle to obtain the sequence of English nonterminals.

An important property of M_s is that every edge connects two linked pairs that share a boundary in either the Chinese component of the rule (red edge) or in the English component (green edge). Note that, as a special case, we consider the linked pair (〈1, 0〉, 〈2, 0〉) as sharing two left boundaries with the linked pairs (〈1, 1〉, 〈2, π⁻¹ (1)〉) and (〈1, π(1)〉, 〈2, 1〉), and as sharing two right boundaries with the linked pairs (〈1, r〉, 〈2, π⁻¹ (r)〉) and (〈1, π(r)〉, 〈2, r〉).

Consider any set S of linked pairs from M_s such that (〈1, 0〉, 〈2, 0〉) ∉ S. Let be the complement set of S, that is, is the set of linked pairs from M_s not in S. It is not difficult to see that the multiset of edges connecting S and corresponds to the set of boundaries of any parse tree associated with the linked pairs in S. We can apply this property to parsing strategies, which we have previously defined as rooted binary trees, in order to count the boundaries that are open after completing some parsing step represented by an internal node n, which we have defined as bd(n). Consider for instance the parsing strategy shown in Figure 2, and consider the internal node n₃. At this node we have collected nonterminals , , and , and we have bd(n₃) = 6. If we consider the set S with the corresponding linked pairs (〈1, 2〉, 〈2, 7〉), (〈1, 3〉, 〈2, 3〉), and (〈1, 4〉, 〈2, 8〉), we can see that the number of edges of the multigraph that connect S with is six, that is, exactly the value of bd(n₃). In order to express these observations in a mathematical way, we need to introduce the notions of tree layout, width, and carving width, which we borrow from graph theory.

A tree layoutT of a graph G is an undirected binary branching tree having one vertex of G at each leaf. To avoid confusion, in what follows we use the term node when referring to vertices of layout trees and we use the term arc when referring to edges of layout trees. Edges of G are routed along the arcs of T. A simple example is provided in Figure 5, showing a graph G and a tree layout T of G.

We can now apply the notion of tree layout to parsing strategies for SCFG rules, and show an important property that relates the notions of fan-out (equivalently, boundary count) and carving width. Let s be an SCFG rule with r linked pairs. Recall that a parsing strategy for s is a rooted binary tree where each leaf node is a linked pair representing some nonterminal from the right-hand side of s. We attach to the root of our parsing strategy an additional leaf node, representing the special linked pair (〈1, 0〉, 〈2, 0〉). The resulting tree has r + 1 leaves, and can therefore be used as a tree layout for the cyclic permutation multigraph M_s encoding s. Furthermore, each internal node of the tree layout is associated with a parsing step.

The next lemma summarizes the connection between carving width and fan-out.

In this section, we reduce the problem of carving width of general graphs to the problem of carving width of cyclic permutation multigraphs, defined in Section 3. This reduction involves constructing a cyclic permutation multigraph from an input graph as outlined in Section 4.1. The details of the construction are given in Section 4.2, and the proof of NP-completeness for carving width of the resulting multigraphs is given in Section 4.3. This result is then used in Section 4.4 to show that finding a space-optimal binary parsing strategy for an SCFG rule is an NP-hard problem.

Suppose that we are given an instance of the general carving width problem, consisting of a graph G and a positive integer k, where we have to decide whether the carving width of G is less than or equal to k. We identify the vertices of G with positive integers in [n], where n ≥ 2 is the number of vertices of G. Our reduction consists in the construction of a cyclic permutation multigraph M such that the carving width of M is equal to 4k if and only if the carving width of G is equal to k.

The main idea underlying our construction is that each vertex i of G is associated with a gadget in M consisting of two components. The first component is a grid-like graph X_i of 2d_i rows and 2d_i columns, where d_i is the degree of vertex i in G. The second component is a grid-like graph G_i of 4k rows and 4k columns. Besides the edges of the grids X_i and G_i, M also includes some extra edges, called interconnection edges. For each i ∈ [n], there are some interconnection edges connecting G_i and X_i. Furthermore, for each edge (i, j) of G, there are interconnection edges connecting components X_i and X_j.

Interconnection edges serve two main purposes. First, they are used to connect the red and green paths within M, so that M satisfies the definition of cyclic permutation multigraph. Second, the connections between component pairs X_i and X_j mimic the structure of the source graph G, as will be explained in more detail later. This second condition guarantees that an optimal tree layout T_M of M can be obtained from an optimal tree layout T of G, by placing each X_i and G_i component under the leaf node of T that is associated with vertex i of G. A simple example of the construction is schematically depicted in Figure 7.

In this section, we define precisely the grid-like graphs X_i and G_i, for each vertex i of the source graph G, and outline the overall structure of the cyclic permutation multigraph M constructed from G.

To be used later in our construction, we need to define a special ordering of the edges of G. We do this here by constructing a path through G that starts at an arbitrary vertex and visits each edge exactly twice. We adapt the standard procedure for depth-first traversal of an undirected graph. More precisely, whenever we reach a vertex i, we continue our path by arbitrarily choosing an edge (i, j) that has not already been visited, and move to j. If all of the edges at i have already been visited, we backtrack to previous vertices of our path, until we reach edges that have not already been visited. The path construction stops when we reach the starting vertex and all of the edges of G have been visited. Algorithm 1 provides a recursive procedure for the incremental construction of the path, assuming that the path is a global variable initialized as the empty sequence of vertices. A simple example is also shown in Figure 8.

Let m be the number of edges of G. It is not difficult to show that the path produced by Algorithm 1 is a sequence 〈i₁, … , i_2m+1〉 of vertices from G satisfying both of the following properties.

• • 

  For each edge (i, j) of G, there is a unique k ∈ [2m] such that i_k = i and i_k+1 = j, and there is a unique h ∈ [2m] such that i_h = j and i_h+1 = i.

• • 

  For each j ∈ [n] we have |{k : k ≤ 2m, i_k = j}| = d_j. (Recall that d_j is the degree of vertex j in G.)

The graph component X_i, i ∈ [n], embeds a square grid with 2d_i rows and 2d_i columns. In addition to the edges of the grid, X_i also contains a set of interconnection edges that connect X_i to component G_i (specified later) and that connect X_i to all components X_j such that (i, j) is an edge of G. This is schematically depicted in Figure 9. (A complete example for components X_i and G_i will be presented later.)

More precisely, let be the vertex in the mth row and nth column of X_i. Similarly, be the vertex in the mth row and nth column of G_i.

• • 

  For each p ∈ [2d_i − 1], X_i has an interconnection edge . Furthermore, X_i has an interconnection edge .

• • 

  Symmetrically for each q ∈ [2d_i − 1], X_i has an interconnection edge . Furthermore, X_i has an interconnection edge .

For the connections between gadget X_i and other gadgets X_j, our strategy is more involved, because we need to reproduce the topology of G and, at the same time, we need to construct a red and a green path within M. For each vertex j that is a neighbor of vertex i in G, we add two red edges between the left column of X_i and the left column of Xj, and two green edges between the top row of X_i and the top row of X_j; see again Figure 9. Crucial to our construction, the order in which we visit the neighbors of i is induced from the depth-first traversal of G specified by Algorithm 1 in Section 4.2.1.

Let γ = 〈i₁, …, i2_m+1〉 be the path produced by Algorithm 1 when given vertex 1 as input, so that i₁ = i2_m+1 = 1. We use a function df_γ(i, k), for i ∈ [n] and k ∈ [2m], that counts the number of interconnection edges already established for the left column (equivalently, for the top row) of X_i, at step k of the depth-first traversal, that is, right before the kth edge (i_k, i_k+1) of γ is processed. In this way, df_γ (i, k) + 1 is the next available position in the left column (equivalently, top row) of X_i when visiting the kth edge of γ.

For technical reasons, grid X₁ needs a special treatment: the last edge (i_2m, i_2m+1) in γ is used to enter grid X₁ for the first time, and is therefore placed at vertex . Therefore, all other edges of γ that are placed at X₁ need to be shifted by one position. This means that we have to increase our count df_γ (i, k) by one unit when i = 1 and k < 2m, and we have to treat the case of i = 1 and k = 2m in a special way.

We are now ready to specify the interconnection edges for X_i. Recall from Section 4.2.1 that, for an edge (i, j) of G, there exists a unique k such that i_k = i and i_k+1 = j in γ, and there exists a unique k′ such that i_k′ = j and i_k′+1 = i.

• • 

  For each edge (i, j) of G, let k be as above. X_i has an interconnection edge . Symmetrically, X_i has an interconnection edge .

• • 

  For each edge (i, j) of G, let k′ be as above. X_i has an interconnection edge . Symmetrically, X_i has an interconnection edge .

We now turn to (multi)graph G_i, i ∈ [n], which embeds a square grid with 4k rows and 4k columns; here k is the positive integer in the input instance of the carving width problem in our reduction; see Section 4.1. We have already introduced interconnection edges and for G_i in Section 4.2.2. In addition to these edges, we need to double some occurrences of the edges internal to the grid G_i, in order to connect the red and green paths within M. We remind the reader that M is defined as a multigraph; therefore we can introduce multiple edges joining the same pair of vertices of G_i. As will be discussed in detail later, our construction always assigns different colors (red or green) to two edges joining the same pair of vertices. An overall picture of G_i is schematically presented in Figure 10 for d_i = 3 and k = 5.

We first provide the specification of the red edges of G_i.

• • 

  For each p ∈ [2k], we double edge .

• • 

  For each p with d_i ≤ p ≤ k − 1, we double edge .

• • 

  For each p with k + 1 ≤ p ≤ 2k − 1, we double edge .

• • 

  We add edge .

To summarize the previous sections, the cyclic permutation multigraph M contains components X_i and G_i for each i ∈ [n], where n is the number of vertices of the source graph G. Besides the edges of the grid-like components X_i and G_i , M also contains some interconnection edges. More precisely, for each vertex i of G, M contains 2d_i red edges and 2d_i green edges connecting X_i and G_i. Furthermore, for each edge (i, j) of G, M contains two red edges and two green edges connecting X_i and X_j.

We still need to show that M is a cyclic permutation multigraph, that is, all the edges mentioned above form a red Hamiltonian cycle and a green Hamiltonian cycle over the vertices of M. We start by observing that, within each component G_i, these two cycles are symmetric, in the sense that the green Hamiltonian cycle can be obtained from the red Hamiltonian cycle by switching the first and the second indices of each vertex in an edge. In other words, within G_i the green Hamiltonian cycle can be obtained from the red Hamiltonian cycle by a rotation along the axis from vertex to vertex . This is also apparent from Figure 10. A similar observation holds for the components X_i, as apparent from Figure 9, and for the interconnection edges. For this reason, we outline in the following only the red Hamiltonian cycle of M.

The red cycle of M starts and ends at vertex in G₁. The cycle follows a depth-first traversal of G specified by Algorithm 1 in Section 4.2.1. Each time the depth-first traversal visits node i of G, the red cycle travels across one row of X_i from left to right, then passes through G_i, and then travels across the next row in X_i from right to left, before proceeding to the gadget for the next vertex of G in the depth-first traversal. Exactly how the red cycle travels through G_i differs for the first d_i − 1 times that G_i is visited and the final time; see Figure 10. The first d_i − 1 times that G_i is visited, the red cycle travels from left to right across one row, descends to the next row, and traverses it from the right to left. On the final trip into G_i, the cycle visits all the remaining rows as follows. It travels left to right across row 2(d_i − 1) + 1 of G_i, then zig-zags across the next 2k − 2(d_i − 1) − 1 rows in the upper half of G_i. Upon reaching vertex , coming from its right and proceeding to the left, the red cycle moves to using an edge internal to the grid. This choice is designed to ensure, for reasons that will become clear later, that none of the extra edges added to the grid underlying G_i crosses between the grid's four quadrants.

Next, the red cycle travels from vertex toward the right to reach , and then zig-zags across the next 2k − 1 rows of the lower half of G_i. Upon reaching vertex , the red cycle jumps to vertex using a single extra edge. It then travels toward the right to reach , moves to using an edge internal to the grid, and finally proceeds toward the left to reach . This completes the last traversal of G_i by the red cycle.

From the red cycle leaves G_i and enters X_i at vertex . It then travels across the bottom row of X_i to reach vertex , and then proceeds to visit the gadget X_j for the next vertex j of G in the depth-first traversal, as already mentioned.

The main result in this section shows that deciding whether the carving width of a cyclic permutation multigraph is less than or equal to a given integer is an NP-complete problem. We develop this result by means of several intermediate lemmas, reducing from the carving width decision problem for a general graph. Throughout this section, we assume that G, k, and M are defined as in the previous sections.

In the case of d_i = k, the grid component X_i is the same as the top left quadrant of G_i. Therefore, the analysis provided in the proof of Lemma 2 can also be used to prove the next lemma.

The tree layout for X_i is a long chain, isomorphic to the tree layout of the upper left quadrant of G_i in Figure 12. In what follows, we refer to the node of the layout immediately above as top-most, and the node immediately above as bottom-most. We can now prove the correctness of our reduction for one direction.

We now deal with the other direction in the correctness of our reduction. We must show that if M has a tree layout of carving width 4k, then G has a tree layout of carving width k. This will be Lemma 8, for which we will need to develop a few intermediate lemmas. We introduce our first intermediate lemma by means of a simple example.

The special properties of arc a that we have mentioned here are not dependent on the choice of the tree layout T_M. In fact the next lemma shows that, for every choice of a tree layout of M having carving width 4k, and for every grid G_i in M, i ∈ [n], there always exists an arc that satisfies the properties of Example 7. Intuitively, this happens because the vertices in a grid have a relatively high degree of interconnections, and when we pick up a sufficiently large set of vertices of a grid G_i, we have a large number of edges connecting the vertices in our set to the remaining vertices of G_i. This in turn means that, in any tree layout T_M, any subtree T_i containing a sufficiently large set of vertices of G_i must route a large number of edges from G_i through the arc a that connects T_i to the rest of T_M. If T_M has bounded carving width, the edges routed through arc a saturate the width of this arc, making it impossible for other vertices not from G_i to be placed within T_i. In other words, there is always some core set of vertices from G_i that is placed in some subtree of a tree layout of M, and this subtree must exclude vertices from other grids.

Proof

For a subtree T of T_M, let L_i(T) be the number of vertices of G_i that are at the leaves of T. Consider an arc a of T_M, and let T₁ and T₂ be the two subtrees of T_M obtained by removing a from T_M; this is exemplified in Figure 13. We say that a is balanced for G_i if the choice of a minimizes |L_i(T₁) − L_i(T₂)|.

The following argument is a standard one, see for instance Kozawa, Otachi, and Yamazaki (2010, Lemma 3.1). Assume that a is a balanced arc in T_M. Let |G_i| = (4k)² be the number of vertices of G_i. Without loss of generality, we assume that L_i(T₁) ≥ L_i(T₂). Because |G_i| = L_i(T₁) + L_i(T₂), this implies . Because |G_i| ≥ 16, we must have L_i(T₁) > 1, otherwise a would not be balanced. Then the root of T₁ must have two children, rooting subtrees and of T₁; see again Figure 13. We must have and , otherwise a would not be balanced. This in turn implies .To summarize these inequalities, we have

For each vertex i of G, we now choose a_i in the statement of the lemma to be a balanced arc for G_i. In the following discussion we focus on an arbitrary choice of i ∈ [n], and again we let T₁ and T₂ be the two subtrees of T_M obtained by removing a_i from T_M, with L_i(T₁) ≥ L_i(T₂), so that we can use Equation (6).

Let S be a grid of size 4k × 4k, that is, a square grid with |G_i| nodes. It is known that, for any choice of s nodes from S with , there are at least 4k edges of S connecting the chosen nodes to nodes in the complement set. For this claim, see for instance Ahlswede and Bezrukov (1995) or Rolim, Sýkora, and Vrt'o (1995). This claim must also be true for the edges internal to G_i, since G_i embeds the grid S. From Equation (6) we then have that at least 4k edges internal to G_i are routed through a_i. Furthermore, since T_M has carving width 4k, the width of a_i cannot exceed 4k. We then conclude that conditions (i) and (ii) are both satisfied.

Let be the set of vertices of M that are not vertices of G_i. It is easy to see from the specification in Section 4.2 that the sub-multigraph of M induced by is connected. (This is the very reason why we introduced the components X_i in our construction of M.) This implies that for any pair of vertices in , there exists a path in M connecting the two vertices that is entirely composed of edges that are not internal to G_i. If both T₁ and T₂ have some vertex from , there would be an edge not internal to G_i routed through a_i, in contrast to condition (ii). We therefore conclude that one among T₁ and T₂ contains only vertices internal to G_i, satisfying condition (iii). This subtree is called the subtree below a_i.

From Equation (6) and by our choice of T₁ , we have . Furthermore, , and hence both T₁ and T₂ have at least 4k² vertices from G_i. Regardless of which of T₁ or T₂ is below a_i, we conclude that condition (iv) is satisfied. ∎

Consider a square grid Q of size 4k × 4k, and let d be some fixed integer with d ∈ [k]. Assume that 2d vertices in the left-most column of Q and 2d vertices in the top-most row of Q have been selected as connector vertices. Observe that the number of connector vertices in Q is either 4d − 1 or 4d, depending on whether the vertex at the top-left corner of Q is selected twice or not. Let S be any subset of the vertices of Q. Recall from Section 3 that the edge boundary of S in Q, written ∂_Q(S), is the set of edges of Q that connect vertices in S and vertices in , the complement set of S.

Let T_M be any tree layout of M with carving width 4k. In Lemma 5 we associated each vertex i of G with an arc a_i of T_M having some specific properties. Similarly, our next lemma associates each edge (i, j) of G with a set of four paths in T_M. These paths are routed through a_i and through a_j, and do not share any of their edges. Using this property, we will later be able (in Lemma 8) to derive the carving width of G from the carving width of M. We use a simple example here to illustrate the special paths we are looking for.

The next lemma generalizes the previous example, showing that for any tree layout T_M we can associate each edge (i, j) of G with a set of four paths in M that are edge disjoint. We need to introduce some auxiliary notation. Let T_M and a_i, i ∈ [n], be defined as in Lemma 5. Consider an arc (i, j) in the source graph G. A path in M is called an (i, j)-path for T_M if it starts at a vertex inside the subtree below a_i (see Lemma 5(iii) for the definition of this subtree) and it ends at a vertex inside the subtree below a_j. We also say that two paths in M are edge disjoint if they do not share any edge (they might however share some vertex).

Proof

We call connector edges the edges of M that are not internal to any component G_i, i ∈ [n]. We call connector vertices those vertices of G_i, i ∈ [n], that have an impinging connector edge. In this way, each G_i has 4d_i − 1 connector vertices. These are placed in the left-most column of G_i and in the top-most row of G_i. We claim that, for each i ∈ [n], the connector vertices of G_i must all be outside of the subtree of T_M below a_i. To see this, let us assume that some connector vertex of G_i is found inside the tree below a_i. Because connector vertices are connected to vertices outside of G_i, there would be a vertex not internal to G_i inside the tree below a_i, against Lemma 5(iii), or else there would be a connector edge routed through a_i, against Lemma 5(ii). We therefore conclude that the connector vertices of G_i must all be outside of the tree below a_i.

The idea underlying the proof is to define each (i, j)-path in as the concatenation of three sub-paths, called segments, specified as follows.

• • 

  The first segment starts with an edge routed through arc a_i. By definition, one of the two end vertices of such an edge is inside the subtree below a_i, and this is also the starting vertex of the segment. Furthermore, the segment only uses edges that are internal to G_i, and ends with a connector vertex of G_i.

• • 

  The second segment only uses connector edges, and ends with a connector vertex of G_j.

• • 

  The third segment only uses edges internal to G_j, and ends with an edge routed through arc a_j. The end vertex of such an edge that is inside the subtree below a_j is the end vertex of the segment.

From now on, we focus on a specific vertex i of G and prove the existence of 4d_i edge disjoint (i, j)-paths for vertices j that are neighbors of i in G. We do this by separately specifying the three segments of the paths.

First segment. From Lemma 5(ii), there are 4k edges internal to G_i that are routed through arc a_i. We know that k ≥ d_i, since the carving width of a graph is always at least its degree. Thus we start our segments with any choice of 4d_i disjoint edges routed through arc a_i. We now show that these edges can be extended to 4d_i segments that reach the 4d_i − 1 connector vertices of G_i, without sharing any of their edges. We do this by reducing our problem to a network flow problem.

If we assume that edges of G_i have flow capacity of one, each segment corresponds to a flow through G_i having capacity one. The existence of 4d_i edge disjoint segments is then equivalent to the existence of a flow of capacity 4d_i from the subtree below a_i to the 4d_i − 1 connector vertices of G_i. Let V(G_i) be the set of vertices of G_i. The max-flow min-cut theorem states that this flow can be realized if and only if any cut of G_i with the source vertices on one side and the target vertices on the other side has capacity not less than 4d_i.

In our specific case, consider the square grid Q underlying component G_i. Let S ⊆ V(G_i) be a vertex set including all the vertices in the subtree below a_i and excluding all the 4d_i − 1 connector vertices of G_i. If we can show the relation

for any choice of S as before, then we have proved the existence of the desired 4d_i edge disjoint segments.

In order to prove Equation (7), we observe that the set of vertices in the subtree below a_i has size greater than or equal to 4k², by Lemma 5(iv). Furthermore, observe that there are 2d_i connector vertices of G_i in the left-most column of Q as well as 2d_i connector vertices in the top-most row. Under these conditions, we can apply Lemma 6 to the grid Q, with d = d_i ≤ k, and derive Equation (7). We then conclude that there exist 4d_i edge disjoint first segments starting at vertices in the subtree below a_i, as desired.

Second segment. As already observed in Section 4.2.2, for every connector vertex in the left-most column of G_i there is a red segment to some connector vertex in the left-most column of G_j , where j is some neighbor of i in G. Symmetrically, for every connector vertex in the top-most row of G_i there is a green segment to some connector vertex in the top-most row of some G_j . This provides a total of 4d_i segments. These segments are all edge disjoint, by construction of the components X_h, h ∈ [n], and by construction of the interconnection edges.

Third segment. We observe that the third segment of an (i, j)-path is the reversal of the first segment of a (j, i)-path. Therefore we can use our argument for the first segments to show the existence of 4d_i edge disjoint third segments ending at a vertex in the subtree below a_j.

To summarize, for each i ∈ [n] we have shown the existence of 4d_i edge disjoint (i, j)-paths for vertices j that are neighbors of i in G, to be added to set . To complete the proof, we observe that two paths in having disjoint start vertices must have edge disjoint first segments, since these segments are defined for edges that are internal to different components G_i. A similar argument applies to the third segments of paths having disjoint end vertices. ∎

We can now provide the main result of this section.

We can now present the main result on the optimization of the fan-out of parsing strategies for an SCFG rule.

In this section, we address the problem of finding the parsing strategy with the optimal time complexity. The time complexity of one combination step in a parsing strategy is determined by the total number of variables involved in the step, which is the total number of distinct endpoints of the spans being combined. Time complexity can differ from space complexity due to the fact that the time complexity takes into account which endpoints are shared in the two groups of spans being combined.

In general, more densely interconnected graphs have higher treewidth. Any tree has treewidth 1, a graph consisting of one large cycle has treewidth 2, and a fully connected graph of n vertices has treewidth n − 1. Low treewidth indicates some treelike structure in the graph, as shown by the example with treewidth 2 in Figure 16. As an example of the running intersection property, note that the vertex N appears in three adjacent nodes of the tree decomposition. Finding the treewidth of a graph is an NP-complete problem (Arnborg, Corneil, and Proskurowski 1987).

We use a representation known as a dependency graph to analyze the time complexity of parsing for a given SCFG rule. Dependency graphs (known under a large number of different names) are used to represent the interaction between variables in constraint satisfaction problems, and have one vertex for each variable, and an edge between vertices representing any two variables that appear together in a constraint. In parsing problems, the variables consist of indices into the string being parsed, and the constraints require that a nonterminal on the right-hand side of a rule has previously been identified. Thus, the graph has a vertex for each endpoint involved in the rule, and a clique for each nonterminal (including the left-hand side nonterminal) connecting all its endpoints.

A tree decomposition of the dependency graph corresponds directly to a parsing strategy, and the treewidth of the graph plus one is the exponent in the time complexity of the optimal parsing strategy (Gildea 2011). Each cluster of vertices in the tree decomposition corresponds to a combination step in a parsing strategy. The running intersection property of a tree decomposition ensures that each endpoint in the parsing rule has a consistent value at each step. Treewidth depends on the number of vertices in the largest cluster of the tree decomposition, which in turn determines the largest number of endpoints involved in any combination step of the parsing strategy.

Our hardness result in this section is a reduction from treewidth of general graphs. Given an input graph G to the treewidth problem, we construct an SCFG rule using techniques similar to those in the previous section. We then show that finding the parsing strategy with optimal time complexity for this rule would imply an approximation bound on the treewidth of the original graph G.

The precise construction proceeds as follows. Given an instance of the treewidth problem consisting of a graph G and integer k, we construct a cyclic permutation multigraph M as in the previous section, with the exception that the grid G_i will contain only 2d_i rows and 2d_i columns. This grid, shown in Figure 18, is a simplified version of the grid used in Figure 10. Furthermore, there are no gadgets X_i in M, but rather, for each edge (i, j) in G, there are a total of four edges in M connecting G_i directly to G_j. We refer to these edges as connector edges, and we refer to edges internal to some G_i as internal edges.

We construct a dependency graph D by replacing each edge (i, j) in M with a vertex v(i, j) in D, and adding edges (v(i, j), v(i, k)) to D connecting all vertices in D that derive from adjacent edges in M, as shown in Figure 19. As is required for the dependency graph representation, this gives us a graph with one vertex corresponding to each boundary between adjacent nonterminals in the SCFG rule and a clique connecting the vertices for the four boundary points of each nonterminal (two boundaries in English and two in Chinese).

We refer to vertices in D derived from connector edges in M as connector vertices, and we refer to vertices in D derived from internal edges in M as internal vertices. We refer to the subgraph of D derived from vertex i of G as D_i. To be precise, D_i consists of connector vertices and internal vertices derived from edges of M incident on vertices of G_i, and the edges of D connecting these vertices. We refer to the vertices of D_i as red or green, depending on whether they are derived from red or green edges in M.

We now show that any dependency graph D constructed from a cyclic permutation multigraph M is the dependency graph for some SCFG rule r. Interpret an arbitrary vertex in M as the left-hand side nonterminal. Each green edge represents a boundary in English shared between two nonterminals, including the boundary shared between the first right-hand side nonterminal and the left-hand side nonterminal, and the boundary shared between the last right-hand side nonterminal and the left-hand side nonterminal. Similarly, each red edge represents the boundary between two nonterminals in Chinese. We then replace each edge in M with a vertex in D, and replace each vertex in M with a clique in D. D is the dependency graph for an SCFG rule, because it has a vertex for each boundary, and a clique that connects the boundaries of each nonterminal (including the left-hand side nonterminal).

With this construction in place, we can prove the main result of this section.

Proof

Let T be a tree decomposition of G having treewidth k − 1, and let ∆(G) be the maximum degree of a vertex of G. Let M and D be the cyclic permutation multigraph and the dependency graph, respectively, constructed from G as previously specified. We can construct a tree decomposition T_D of D having treewidth 4k∆(G) − 1 by replacing each occurrence of vertex i in T with the 4d_i connector vertices in D that are derived from the 4d_i connector edges in M that are adjacent to G_i. From one of the nodes of T_D containing the connector vertices of i, we attach a subtree containing the remaining vertices of D_i. This subtree has treewidth 4d_i − 1, and can be constructed in a linear chain adding vertices from D_i one at a time. That is, we process the nodes in D_i in column-major order, at each step constructing a new node in the tree decomposition of D by adding one vertex of D_i, either red or green, and removing the corresponding vertex from the previous red or green column respectively. In sum, if k − 1 = OPT = tw(G), then

Suppose that we have an algorithm for finding the treewidth of dependency graphs derived from SCFG rules. Given a tree decomposition T_D of D of treewidth k′ − 1, a valid tree decomposition T_G of G having treewidth at most 2k′ − 1 can be constructed by replacing each connector vertex in T_D with the corresponding two vertices i and j of G, and replacing occurrences of internal vertices of D with the corresponding vertex i of G. This is a valid tree decomposition of G, because of the following.

• • 

  For any edge (i, j) in G, there is a node in T_D containing the corresponding connector vertex in D, and hence a node in T_G containing both i and j.

• • 

  Any two vertices u and v in D that are associated with i in G (either as connector vertices or as internal vertices) are connected by a path in D containing only vertices associated with i. Hence, the path between any two occurrences of i in T_G contains nodes that also contain i, guaranteeing the running intersection property of T_G.

Thus, by constructing D from G, finding an optimal tree decomposition of D, and then translating the result into a tree decomposition of G, we have a tree decomposition of G having width SOL where SOL ≤ 2tw(D). Combining the results, we have

Therefore, an algorithm for minimizing the treewidth of dependency graphs of SCFG rules would imply the existence of a constant-factor approximation algorithm for the treewidth of graphs of fixed degree. ∎

The existence of a polynomial time constant-factor approximation algorithm for the treewidth of graphs of fixed degree is a longstanding open problem. Therefore Theorem 3 suggests that designing a polynomial time algorithm for finding the parsing strategy of an SCFG rule having optimal time complexity is a difficult task.

In the other direction, a proof of hardness for finding the optimal time complexity for SCFG parsing would also require progress on a longstanding open problem, as stated by the following theorem.

In the context of machine translation, the problem of synchronous parsing with an SCFG corresponds to the problem of analyzing parallel text into grammar derivations, often as part of learning a translation model with Expectation Maximization or related algorithms. The synchronous parsing problem also applies to decoding with an SCFG and an integrated n-gram language model.

Parsing with an SCFG requires time (and space) polynomials in the sentence length, but with the degree of the polynomial depending on the specific grammar. A loose time upper bound obtained using dynamic programming techniques is , where n is the sentence length and r is the maximum rule length, that is, the maximum number of linked nonterminals in the right-hand side of a rule. Gildea and Štefankovič (2007) show a lower bound of Ω(n^cr) for some constant c, meaning that the exponent grows linearly with the rule length. In this article, we show that even finding this exponent is itself computationally difficult. For space complexity, finding the best parsing strategy is NP-hard. For time complexity, finding the best parsing strategy would require progress on approximation algorithms for the treewidth of general graphs.

To our knowledge, the use of the notion of carving width of a graph in connection with the space complexity of dynamic programming parsing is novel to this article. In previous work, Crescenzi et al. (2015) used the notion of graph cutwidth to investigate space complexity for SCFGs restricted to linear parsing strategies, that is, strategies that add only one nonterminal at a time. However, graph cutwidth does not extend to the general binary parsing strategies that we consider in this article.

The connection of space complexity to carving width means that we can use existing linear-time algorithms for computing the tree layout of graphs of bounded carving width (Thilikos, Serna, and Bodlaender 2000), and approximation algorithms for carving width such as the -factor approximation algorithm of Khuller, Raghavachari, and Young (1994). The connection of time complexity to treewidth means that we can apply algorithms for treewidth of general graphs that tend to find optimal results quickly in practice, such as the branch-and-bound algorithm of Gogate and Dechter (2004). We can also apply linear time algorithms for graphs of bounded treewidth (Bodlaender 1996), and approximation algorithms such as the -factor approximation algorithm of Feige, Hajiaghayi, and Lee (2005).

Because SCFG is an instance of the general class of linear context-free rewriting systems (LCFRSs) (Vijay-Shankar, Weir, and Joshi 1987), our hardness results also apply to the latter class. Therefore our results generalize those of Crescenzi et al. (2011), who show NP-hardness for optimizing space and time complexity for LCFRSs, again with the restriction for linear parsing strategies.

Viewing SCFG as an instance of LCFRS also allows connecting the parsing optimization problems investigated in this article to the rule factorization problem for SCFGs, investigated by Huang et al. (2009). Rule factorization is the problem of replacing an individual rule with a large size with several rules of smaller size, in a way that the language generated by the overall grammar is preserved. (The algorithm for casting a CFG in Chomsky normal form is an example of rule factorization.) As already mentioned in the Introduction, it is known that SCFGs do not admit any canonical binary form (Aho and Ullman 1972). This means that, when we attempt to factorize a SCFG rule into “smaller pieces,” some of the resulting pieces might no longer be SCFG rules, because they span more than two substrings; see again Figure 3. However, when viewing a SCFG as a LCFRS, factorization is always possible, and the parsing strategy trees defined in Section 2.3 directly provide a binary factorization of a SCFG into a LCFRS. Under this view, the problems investigated in this article are related to the problem of detecting rule factorizations for certain LCFRS, where we require that the factorized LCFRS are space or time optimal when used in standard algorithms for parsing based on dynamic programming. Rule factorization for general LCFRS has been investigated by Gómez-Rodríguez et al. (2009) and Gildea (2010).

As a final remark, if a strategy of space or time exists for a fixed k, it can be found in space or time , respectively, by parsing the SCFG rule's right-hand itself, represented as a string of nonterminals, with a grammar representing all possible parsing strategies. Such a grammar can be constructed by instantiating all possible LCFRS rules of space or time complexity . If a parse of the SCFG rule's right-hand side is found, the resulting parse tree can be used as a parsing strategy for the original rule.

We are grateful to Pierluigi Crescenzi for pointing us in the direction of carving width. D. Gildea has been partially funded by NSF award IIS-1446996. G. Satta has been partially supported by MIUR under project PRIN No. 2010LYA9RH 006.