Abstract
The syntactic structure of a sentence is often represented using syntactic dependency trees. The sum of the distances between syntactically related words has been in the limelight for the past decades. Research on dependency distances led to the formulation of the principle of dependency distance minimization whereby words in sentences are ordered so as to minimize that sum. Numerous random baselines have been defined to carry out related quantitative studies on lan- guages. The simplest random baseline is the expected value of the sum in unconstrained random permutations of the words in the sentence, namely, when all the shufflings of the words of a sentence are allowed and equally likely. Here we focus on a popular baseline: random projective per- mutations of the words of the sentence, that is, permutations where the syntactic dependency structure is projective, a formal constraint that sentences satisfy often in languages. Thus far, the expectation of the sum of dependency distances in random projective shufflings of a sentence has been estimated approximately with a Monte Carlo procedure whose cost is of the order of Rn, where n is the number of words of the sentence and R is the number of samples; it is well known that the larger R is, the lower the error of the estimation but the larger the time cost. Here we pre- sent formulae to compute that expectation without error in time of the order of n. Furthermore, we show that star trees maximize it, and provide an algorithm to retrieve the trees that minimize it.
1 Introduction
A successful way to represent the syntactic structure of a sentence is a dependency graph (Nivre 2006) that relates the words of a sentence by pairing them with syntactic links, as in Figure 1. Each link is directed and the arrow points from the head word to the dependent word (Figure 1). There are several conditions that are often imposed on the structure of dependency graphs (Nivre 2006). The first is well-formedness, namely, the graph is (weakly) connected. The second is single-headedness, that is, every word has at most one head. Another condition is acyclicity, that is, if two words, say wi and wj, are connected via following one or more directed links from wi to wj then there is no path of directed links from wj to wi. By definition, syntactic dependency trees always have a root vertex, that is, a vertex (word) with no head. The fourth condition is projectivity, often informally described as the situation where edges do not cross when drawn above the sentence and the root is not covered by any edge.
When a dependency graph is well-formed, single-headed, and acyclic, the graph is a directed tree, called a syntactic dependency tree (Kuhlmann and Nivre 2006; Gómez-Rodríguez, Carroll, and Weir 2011). In addition, a syntactic dependency structure is projective if, for every vertex v, all vertices reachable from v, that is, the yield of v, form a continuous substring within the linear ordering of the sentence (Kuhlmann and Nivre 2006). Equivalently, a syntactic dependency structure is projective if the yield of each vertex of the tree forms a contiguous interval of positions in the linear ordering of the vertices. Kuhlmann and Nivre (2006) define an interval (with endpoints i and j) as the set [i,j] = {k | i ≤ kand k ≤ j}.
A linear arrangement of a graph is planar if it does not have edge crossings (Sleator and Temperley 1993; Kuhlmann and Nivre 2006). Then projectivity can be characterized as a combination of two properties: planarity and the fact that the root is not covered (Mel’čuk 1988). Planarity was, to the best of our knowledge, first thought of as one-page embeddings of trees by Bernhart and Kainen (1979). Figure 1 shows an example of a projective tree 1(a), a planar tree 1(b), and a non-planar tree 1(c) (see Bodirsky, Kuhlmann, and Möhl [2005] for further characterizations of syntactic dependency structures).
A free tree T = (V,E) is an undirected acyclic graph (Figure 2(a)), where V is the set of vertices and E is the set of edges. Here we represent the syntactic dependency structure of a sentence as a pair consisting of a rooted tree and a linear arrangement of its vertices. A rooted tree Tr = (V,E;r) is a free tree T = (V,E) with one of its vertices, say r ∈ V, labeled as its root and with the edges oriented from r toward the leaves (Figure 2(b)). A linear arrangement π (also called embedding) of an n-vertex graph G = (V,E) is a (bijective) function that assigns every vertex u ∈ V to a position π(u). Throughout this article, we use the terms “linear arrangement,” “linear ordering,” “arrangement,” and “linearization” interchangeably. In addition, we assume that π(u) ∈ [n] = {1,⋯ ,n}. Linear arrangements are often seen as determined by the labeling of the vertices (Chung 1984; Kuhlmann and Nivre 2006), but here we consider that the labeling of a graph and π are independent. In order to clarify our notion of linear arrangement of a labeled graph G, we say that two linear arrangements π1 and π2 of G are equal if and only if, for every vertex u ∈ V, it holds that π1(u) = π2(u).
While there are constant-time formulae for the expectation of D(T) in unconstrained arrangements (Equation (4)), a procedure to calculate the expected value of D(T) under projectivity is not forthcoming. Our primary goal is to improve the calculation of the expected sum of edge lengths in uniformly random projective arrangements with respect to the Monte Carlo method or random sampling method put forward by Gildea and Temperley (2007). Such a widely used procedure (Park and Levy 2009; Futrell, Mahowald, and Gibson 2015; Kramer 2021) estimates the expectation of D of an n-vertex tree with an error that is negatively correlated with R, the amount of arrangements sampled, while its cost is directly proportional to that amount, that is, O(Rn). This raises the question of what the minimum value of R would be to obtain accurate-enough estimations of the expectation of D(T) in projective arrangements. In recent research (Futrell, Mahowald, and Gibson 2015; Kramer 2021), R = 10,R = 100 were used. Here we demonstrate that there is no need to answer this question since we provide formulae to calculate its exact value.
We improve upon these techniques by providing closed-form formulae for the expected value of D(Tr) in uniformly random projective arrangements of Tr that can be evaluated in O(n)-time. More formally, our goal in this article is to find closed-form formulae for , the expectation of the random variable D(Tr) conditioned to the set of projective arrangements, where the subscript “pr” indicates “projective linear arrangement.” Notice that in Equation (4) has no subscript to indicate unconstrained linear arrangement. An unconstrained linear arrangement is one of the n! possible orderings. is a widely used random baseline for research on DDm (Park and Levy 2009; Gildea and Temperley 2010; Futrell, Mahowald, and Gibson 2015; Kramer 2021).
The structure of this article is as follows. We first derive, in Section 2, an arithmetic expression for , given by
For any n-vertex rooted tree Tr, we have that with equality if, and only if, , where denotes the star tree of n vertices.
Then, a tight upper bound of is given by , as detailed in the next corollary.
Theorem 1 and Corollary 1 indicate that, for each n, a star tree rooted at its hub () maximizes , achieving (n2 − 1)/3. Section 3 also shows that the minima can be calculated with a dynamic programming algorithm.
Section 4 compares our new method to calculate exactly against the Monte Carlo estimation method using dependency treebanks and finds that commonly used values of R can yield a large relative error in the estimation on a single tree. This new method is available in the Linear Arrangement Library (Alemany-Puig, Esteban, and Ferrer-i-Cancho 2021). We finally present some conclusions and propose future work in Section 5.
2 Expected Sum of Edge Lengths
We devote this section to characterize projective arrangements (Section 2.1) and to derive an arithmetic expression to calculate the sum of expected edge lengths in said arrangements (Section 2.3). We end this section with some instantiations of said expression for particular classes of trees (Section 2.4).
2.1 The Number of Random Projective Arrangements
The number of unconstrained arrangements of an n-vertex tree T is N(T) = |P(T)| = n!, where P(T) denotes the set of all n! arrangements of T, hence N(T) is independent from the tree structure. The number of projective arrangements of a tree, however, depends on its structure, in particular on the out-degree sequence of the tree, as is shown later in this section. Counting the number of projective arrangements of a tree motivates a proper characterization that underpins the proof of Theorem 1. For this, we need to introduce some notation.
Henceforth we denote directed edges of a rooted tree Tr = (V,E;r) as uv =(u,v) ∈ E; all edges are oriented toward the leaves. We denote the set of children of a vertex v ∈ V as Γv, and thus the out-degree of v is dv = |Γv| in the rooted tree. In particular, we refer to the root’s children as . We denote the subtree of Tr rooted at u ∈ V as ; we denote its size (in vertices) as ; notice that nu ≥ 1. We say that is an immediate subtree of if uv is an edge of the tree. Figure 3 depicts a rooted tree and the immediate subtrees of Tr.
We provide a closed-form formula for the number of projective arrangements of a rooted tree, Npr(Tr). This result helps us characterize said arrangements.
The fact that subtrees span over intervals (Kuhlmann and Nivre 2006) is central to the proof of Proposition 1. Because intervals are associated with a fixed pair of starting and ending positions in a linear sequence, we use the term segment of a rooted tree to refer to a real segment within the linear ordering containing all vertices of (Figure 4); technically, that segment is an interval of length nu whose starting and ending positions are unknown until the whole tree is fully linearized. Thus, a segment is a movable set of vertices within the linear ordering. The concept of segment is equivalent to the notion of continuous constituent in headed phrase structure representations (Kuhlmann and Nivre 2006). Hereafter, for simplicity, we refer to the “segment of a tree in a linear arrangement” simply as “segment of a tree,” assuming that such a segment is defined with respect to a linear arrangement.
We can associate a set of segments to each vertex. The set of vertex u contains du + 1 segments: one segment corresponds to u (the only segment of length 1), and the remaining du segments correspond to the immediate subtrees of . We obtain a projective linear arrangement by permuting the elements of each set for all vertices. Therefore, a projective arrangement can be seen as being recursively composed of permutations of sets of segments. Such “recursion” starts at the permutation of the set of segments associated with r. Note, then, that there are (du + 1)! possible permutations of the segments associated to vertex u. For a fixed permutation of the segments associated with r, there are different projective arrangements of its immediate subtrees , hence the recurrence in Equation (8). Equation (9) follows upon unfolding the recurrence.
The proof of Proposition 1 can be used to devise a simple procedure to generate projective arrangements uniformly at random, and another to enumerate all projective arrangements, of a rooted tree. As explained in previous articles (Gildea and Temperley 2007; Futrell, Mahowald, and Gibson 2015), the former method consists of first generating a uniformly random permutation of the dv + 1 segments associated with every vertex v ∈ V and, afterward, constructing the arrangement using these permutations. When a tree is linearized using the permutations of the sets of segments, we say that each segment becomes an interval.
2.2 The Expected Sum of Edge Lengths in Random Arrangements
We first review the problem of computing —the expected value of D(T) in uniformly random unconstrained arrangements—so as to introduce the methodology applied for . The calculation requires two steps: first, the calculation of , the expected length of an arbitrary edge joining vertices u and v, and second, the calculation of ; henceforth we denote these values as and since they only depend on the size of T, not on its topology. For simplicity, we assume the definition of edge length in Equation (1).
2.3 The Expected Sum of Edge Lengths in Random Projective Arrangements
To obtain an arithmetic expression for , we follow again a two-step approach. First, we calculate the expected length of an edge in uniformly random projective arrangements. However, unlike the unconstrained case, the edge must be incident to the root. Second, we calculate the expected applying the result of the first step. Before we proceed, we need to introduce some notation.
An edge connecting the root of the tree (r) with one of its children (u) can be decomposed into two parts: its anchor (Shiloach 1979; Chung 1984) and its coanchor (Figure 4). Such decomposition is also found in Gildea and Temperley (2007) and Park and Levy (2009), but using different terminology. In the context of projective linear arrangements, we define αru(π) as the length of the anchor, that is, the number of positions of the linear arrangement covered by the edge ru in the segment of including the end of the edge π(u); similarly, we define βru(π) as the number of positions of the linear arrangement that are covered by that edge in segments other than that of and r. Put differently, αru(π) is the width of the part of covered by the edge ru including the end of the edge π(u); similarly, βru is the total width of over all r’s children v that fall between r and u. Then the length of an edge connecting r with u is δru(π) = |π(r) − π(u)| = αru(π) + βru(π).
The next lemma shows that the expected value of δru depends only on the size of the whole tree (nr) and the size of the subtree rooted at the child (nu).
Now we can derive an arithmetic expression for .
In the proof above we implicitly use our definition of edge length δuv(π) (Equation (1)). Nevertheless, the expression in Equation (6) can be easily adjusted to use different definitions of edge length, for example, (Equation (2)). It suffices to find an appropriate transformation of our definition of D(Tr) (Equation (3)) into the one desired, namely, D*(Tr). The next corollary gives the solution.
It is easy to see that Equations (6) and (31) can both be evaluated in O(n)-time and O(n)-space, where n is the number of vertices of the tree: One only needs to compute the values nv in O(n)-time, store them in O(n) space, and then evaluate the formula, also in O(n)-time using those values. In the analysis above, we are assuming that the values of dv are already computed; depending on the data structure used to represent the tree, the cost of computing dv might be relevant.
2.4 Formulae for Classes of Trees
Here we consider three kinds of free trees that are later transformed into rooted trees (Harary 1969; Valiente 2021). First, linear (or path) trees are trees in which the maximum degree is 2. Star trees consist of a vertex connected to n − 1 leaves; also, a complete bipartite graph K1,n−1. A quasi-star tree is a star tree in which one of its edges has been subdivided once with a vertex in the middle.1 For the following analyses, we define the hub of a rooted tree as the vertex of the underlying free tree that has the highest degree; we also use the term “leaf” to refer to a leaf in the underlying free tree. We now instantiate Equations (6) and (9) for several classes of trees (Table 1): Star trees, , rooted at the hub, , and at a leaf, ; Quasi-star trees, , rooted at the hub, , at a leaf adjacent to the hub, , at the leaf not adjacent to the hub , and at the only internal vertex that is not the hub, ; Linear trees when rooted at a vertex at distance k ≥ 0 from one of the leaves, . Each class of tree is depicted in Figure 5. These classes of trees are chosen for graph theoretic reasons. Linear trees minimize the variance of the degree (Ferrer-i-Cancho 2013); star graphs maximize it (Ferrer-i-Cancho 2013) and all their unconstrained linear arrangements are planar; concerning , all its linear arrangements are projective; quasi-star trees maximize the variance of the degree among trees whose set of unconstrained arrangements contains some non-planar arrangement (Ferrer-i-Cancho 2016).
Class of tree . | Tr . | Npr(Tr) . | . |
---|---|---|---|
Star | n! | (n2 − 1)/3 | |
2(n − 1)! | n(2n − 1)/6 | ||
Quasi-star (n ≥ 4) | 2(n − 1)! | (2n2 − 2n + 3)/6 | |
4(n − 2)! | (2n2 − 2n + 3)/6 | ||
6(n − 2)! | (2n2 − 3n + 7)/6 | ||
4(n − 2)! | (2n2 − 3n + 7)/6 | ||
Linear | 2n−1 | (n − 1)(n + 2)/4 | |
, (k > 0) | 3 · 2n−2 | [(n − 1)(3n + 10) + 6k(k + 1 − n)]/12 |
Class of tree . | Tr . | Npr(Tr) . | . |
---|---|---|---|
Star | n! | (n2 − 1)/3 | |
2(n − 1)! | n(2n − 1)/6 | ||
Quasi-star (n ≥ 4) | 2(n − 1)! | (2n2 − 2n + 3)/6 | |
4(n − 2)! | (2n2 − 2n + 3)/6 | ||
6(n − 2)! | (2n2 − 3n + 7)/6 | ||
4(n − 2)! | (2n2 − 3n + 7)/6 | ||
Linear | 2n−1 | (n − 1)(n + 2)/4 | |
, (k > 0) | 3 · 2n−2 | [(n − 1)(3n + 10) + 6k(k + 1 − n)]/12 |
3 Maxima and Minima
In this section, we tackle the problem of computing the minima and characterizing the maxima of , both over all n-vertex root trees (keeping n constant). In particular, we give a closed-form formula for the maximum value of and characterize the trees that maximize it, as well as outline a dynamic programming algorithm to compute the minima. Henceforth, we use to denote the set of n-vertex (unlabeled) rooted trees. Evidently, any tree that maximizes (respectively, minimizes) also maximizes (respectively, minimizes) ; thus we restrict our study to the former. Throughout this section, we use ni to refer to the size of the subtree rooted at the ith child of the root for 1 ≤ i ≤ dr.
The construction of projective minimum linear arrangements has optimal substructure: Optimal arrangements are composed of optimal arrangements of subtrees (Hochberg and Stallmann 2003; Gildea and Temperley 2007; Alemany-Puig, Esteban, and Ferrer-i-Cancho 2022). Similarly, the construction of n-vertex rooted trees that maximize (respectively, minimize) also has optimal substructure. The following lemma proves this claim.
We can construct an n-vertex optimal tree using optimal subtrees. We can obtain the right-hand side of Equation (40) using the recurrence in Equation (5). An optimal n-vertex tree is one whose cost value is optimal among all optimal n-vertex trees whose root has fixed degree dr ∈{1,⋯ ,n − 1}. Given a fixed root degree dr such that 1 ≤ dr ≤ n − 1, an optimal n-vertex rooted tree can be built by constructing an optimal (n − 1)-vertex forest of dr rooted trees, each of size ni vertices and optimal among the ni-vertex trees. Choosing the dr trees to be ni-optimal makes the sum of their costs, , optimal.
In subsequent paragraphs we use fM(n) to denote the maximization variant and fm(n) to denote the minimization variant of f(n), respectively. Theorem 2 characterizes the maxima of . Perhaps not so surprisingly, the only maximum of is obtained by .
We prove this by induction on n using the formalization of the optimum in Equation (40). The base cases can be easily obtained by an exhaustive enumeration of the (unlabeled) rooted trees of n vertices for some small n. Indeed, the only tree that maximizes for n ≤ 2 are the one-vertex tree and the two-vertex tree, which are both star trees.
It is easy to see that is bounded above by the expected value of D(T) as stated in Corollary 1 and justified in its proof below.
In this article, we do not characterize the minima of because, unlike the number of maxima, which shows a clear regularity, the number of minima varies with n in a non-monotonic fashion. Table 2 shows that the number of these minima oscillates between 1 and 2 for n ≤ 20 and Figure 6 shows the number of minimum trees in linear-log scale for n ≤ 178. Figure 7 suggests that the shape of the trees does not seem to fit into a simple class. Moreover, Figure 8 shows the values of fm(n) in log-linear scale. The straight line that is found in that scale for sufficiently large n suggests asymptotic behavior, to be confirmed in future research.
n . | # trees . | # opt trees . | . | n . | # trees . | # opt trees . | . |
---|---|---|---|---|---|---|---|
1 | 1 | 1 | 0 | 11 | 1,842 | 1 | 22 |
2 | 1 | 1 | 1 | 12 | 4,766 | 1 | |
3 | 1 | 1 | 2.5 | 13 | 12,486 | 2 | |
4 | 4 | 2 | 4.5 | 14 | 32,973 | 1 | 31.5 |
5 | 9 | 1 | 15 | 87,811 | 1 | ||
6 | 20 | 1 | 16 | 235,381 | 1 | 38 | |
7 | 48 | 1 | 11 | 17 | 634,847 | 1 | 41.5 |
8 | 115 | 2 | 18 | 1,721,159 | 2 | 45 | |
9 | 286 | 1 | 16.5 | 19 | 4,688,676 | 2 | 48.5 |
10 | 719 | 2 | 20 | 12,826,228 | 2 | 52 |
n . | # trees . | # opt trees . | . | n . | # trees . | # opt trees . | . |
---|---|---|---|---|---|---|---|
1 | 1 | 1 | 0 | 11 | 1,842 | 1 | 22 |
2 | 1 | 1 | 1 | 12 | 4,766 | 1 | |
3 | 1 | 1 | 2.5 | 13 | 12,486 | 2 | |
4 | 4 | 2 | 4.5 | 14 | 32,973 | 1 | 31.5 |
5 | 9 | 1 | 15 | 87,811 | 1 | ||
6 | 20 | 1 | 16 | 235,381 | 1 | 38 | |
7 | 48 | 1 | 11 | 17 | 634,847 | 1 | 41.5 |
8 | 115 | 2 | 18 | 1,721,159 | 2 | 45 | |
9 | 286 | 1 | 16.5 | 19 | 4,688,676 | 2 | 48.5 |
10 | 719 | 2 | 20 | 12,826,228 | 2 | 52 |
4 Exact versus Approximate Calculation of the Expected Sum of Dependency Distances
In previous research, the value of R is sometimes indicated, for example, R = 10 (Kramer 2021) and R = 100 (Futrell, Mahowald, and Gibson 2015), and sometimes not reported (Park and Levy 2009; Gildea and Temperley 2010). When R = 10, Figure 9 shows that the relative error peaks between n = 10 and n = 20 (close to the mean sentence length in English [Rudnicka 2018]) and tends to decay from then onward. However, the confidence interval of the relative error clearly broadens for sufficiently large n, for example, n = 30 or n = 40 onward. These behaviors smooth out for R = 100 while the range of variation and the confidence interval are narrower. As expected, the relative error reduces dramatically for larger R. In sum, Figure 9 indicates that a large R must be used to estimate with high numerical precision. However, that increases the computation time. An implication of that figure is that the best solution in terms of numerical precision and speed is provided by Equation (31).
5 Conclusions and Future Work
In this article, we have derived several simple closed-form formulae related to projective arrangements of rooted trees in Section 2. First, we have given a simple closed-form formula to calculate the number of different projective arrangements a rooted tree admits, Npr(Tr) (Proposition 1). The proof reveals a straightforward way of enumerating such arrangements for an input rooted tree, and of generating this kind of arrangement uniformly at random without a rejection method. Second, and more importantly, we have provided a way of calculating, for any given rooted tree, the expected value of the sum of edge lengths over the space of uniformly random different projective arrangements, (Theorem 1 and Corollary 2). This means that future studies in which such value is calculated approximately via random sampling of arrangements can now be calculated exactly and much faster. The O(Rn) Monte Carlo method to estimate true expectation with an error that tends to zero as R tends to infinity can now be replaced by our fast O(n) with zero error. Moreover, these formulae can be instantiated in particular classes of trees (as shown in Section 2.4). In Section 3, we have characterized the trees that maximize and proven that there exists a dynamic programming method to calculate the minima of . A precise characterization of the minima should be the subject of future research. Finally, in Section 4 we have highlighted the obvious advantages of an exact and fast calculation of for future quantitative dependency syntax research.
The present article is part of a research program on the calculation of random baselines for D via formulae or exact algorithms under formal constraints on linear arrangements that started about two decades ago with the unconstrained case, for example Equation (4) (Zörnig 1984; Ferrer-i-Cancho 2004). Here we have covered the projective case. In a forthcoming article, we will focus on planar linearizations of (free) trees—those in which there are no edge crossings—and obtain a closed-form formula, and a O(n)-time algorithm, to calculate , the expected value of D(T) in uniformly random planar arrangements of a given (labeled free) tree T (Alemany-Puig and Ferrer-i-Cancho 2022). In the future, the problem of the calculation of the variance of D(Tr) in random projective arrangements should also be considered. The analysis of the distribution of dependency distances—for example, their first and second moments of D(Tr), in random arrangements that are not uniformly random (but still projective and planar)—could benefit from applying general-purpose algorithmic frameworks (Eisner 2002; Li and Eisner 2009; Wang and Eisner 2018).
Here we have focused on , the expectation of D(Tr) on random arrangements of an individual tree. Finally, future research should consider the problem of , the expectation of on ensembles of random trees of a fixed size n. is indeed the average value of among all n-vertex rooted trees Tr. Although there are at least two ensembles possible, that is, uniformly random labeled trees and uniformly random unlabeled trees, a closed-form formula (or algorithm) to calculate seems easier to obtain in the former.
Acknowledgments
We are thankful to the anonymous reviewers for invaluable comments and suggestions. LAP is supported by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya and the Social European Fund. RFC is also supported by the recognition 2017SGR-856 (MACDA) from AGAUR (Generalitat de Catalunya). RFC and LAP are supported by the grant TIN2017-89244-R from MINECO (Ministerio de Economía, Industria y Competitividad).
Appendix A. The Standard Deviation of D(TR) in Projective Arrangements
Notes
Alternatively, an n-vertex quasi-star tree is obtained by joining to a 2-vertex complete graph, K2, a pendant vertex to one end and n − 3 pendant vertices to the other end of K2; a quasi-star tree is a particular case of bistar tree (San Diego and Gella 2014).
This Monte Carlo method consists of averaging the values of D obtained via random sampling of R random projective arrangements of a rooted tree. It is well known that the error of such methods decreases as the number of samples increases.
References
Author notes
Action Editor: Giorgo Satta