Abstract
A lexicalized tree-adjoining grammar is a tree-adjoining grammar where each elementary tree contains some overt lexical item. Such grammars are being used to give lexical accounts of syntactic phenomena, where an elementary tree defines the domain of locality of the syntactic and semantic dependencies of its lexical items. It has been claimed in the literature that for every tree-adjoining grammar, one can construct a strongly equivalent lexicalized version. We show that such a procedure does not exist: Tree-adjoining grammars are not closed under strong lexicalization.
1. Introduction
Many contemporary linguistic theories give lexical accounts of syntactic phenomena, where complex syntactic structures are analyzed as the combinations of elementary structures taken from a finite lexicon. In the computational linguistics community, this trend has been called lexicalization, and has been extensively investigated since the 1990s. From a mathematical perspective, the main question that arises in the context of lexicalization is whether the restriction of a given class of grammars to lexicalized form has any impact on the generative or computational properties of the formalism.
As a simple example, consider the class of context-free grammars (CFGs). Recall that a CFG is in Greibach normal form if the right-hand side of every rule in the grammar starts with a terminal symbol, representing an overt lexical item. Although several procedures for casting a CFG in Greibach normal form exist, all of them substantially alter the structure of the parse trees of the source grammar. In technical terms, these procedures provide a weak lexicalization of the source grammar (because the string language is preserved) but not a strong lexicalization (because the sets of parse trees that the two grammars assign to the common string language are not the same). Strong lexicalization is highly relevant for natural language processing, however, where the parse tree assigned by a grammar represents the syntactic analysis of interest, and is used by other modules such as semantic interpretation or translation. In this article, we investigate the problem of strong lexicalization.
Two important results about strong lexicalization have been obtained by Schabes (1990). The first result is that CFGs are not closed under strong lexicalization. (The author actually shows a stronger result involving a formalism called tree substitution grammar, as will be discussed in detail in Section 3.) Informally, this means that we cannot cast a CFG G in a special form in which each rule has an overt lexical item in its right-hand side, under the restriction that the new grammar generates exactly the same set of parse trees as G. As a special case, this entails that no procedure can cast a CFG in Greibach normal form, under the additional condition that the generated parse trees are preserved.
The second result obtained by Schabes concerns the relation between CFGs and the class of tree-adjoining grammars (TAGs) (Joshi, Levy, and Takahashi 1975; Joshi and Schabes 1997). A TAG consists of a finite set of elementary trees, which are phrase structure trees of unbounded depth, and allows for the combination of these trees by means of two operations called substitution and adjunction (described in more detail in the next section). A lexicalized TAG is one where each elementary tree contains at least one overt lexical item called the anchor of the tree; the elementary tree is intended to encapsulate the syntactic and semantic dependencies of its anchor. Because CFG rules can be viewed as elementary trees of depth one, and because context-free rewriting can be simulated by the substitution operation defined for TAGs, we can view any CFG as a special TAG. Under this view, one can ask whether lexicalized TAGs can provide a strong lexicalization of CFGs. Schabes' second result is that this is indeed the case. This means that, given a CFG G, one can always construct a lexicalized TAG generating the same set of parse trees as G, and consequently the same string language.
Following from this result, there arose the possibility of establishing a third result, stating that TAGs are closed under strong lexicalization. Schabes (1990) states that this is the case, and provides an informal argument to justify the claim. The same claim still appears in two subsequent publications (Joshi and Schabes 1992, 1997), but no precise proof of it has appeared until now. We speculate that the claim could be due to the fact that adjunction is more powerful than substitution with respect to weak generative capacity. It turns out, however, that when it comes to strong generative capacity, adjunction also shares some of the restrictions of substitution. This observation leads to the main result of this article: TAGs are not closed under strong lexicalization. In other words, there are TAGs that lack a strongly equivalent lexicalized version.
In the same line of investigation, Schabes and Waters (1995) introduce a restricted variant of TAG called tree insertion grammars (TIGs). This formalism severely restricts the adjunction operation originally defined for TAGs, in such a way that the class of generated string languages, as well as the class of generated parse trees, are the same as those of CFGs. Schabes and Waters then conjecture that TIGs are closed under strong lexicalization. In this article we also disprove their conjecture.
2. Preliminaries
We assume familiarity with the TAG formalism; for a survey, we refer the reader to Joshi and Schabes (1997). We briefly introduce here the basic terminology and notation for TAG that we use in this article.
2.1 Basic Definitions
Auxiliary trees are elementary trees in which a special node in the frontier has the same nonterminal label as the root node. This special node is called the foot node and is identified by an asterisk (*). Auxiliary trees may combine with other trees by means of an operation called adjunction. The adjunction operation entails splitting some target tree at an internal node with label A, and inserting an auxiliary tree whose root (and foot) node is labeled with A. The adjunction operation is illustrated in the right half of Figure 1.
A derivation in a TAG can be specified by a derivation treed; this is a rooted tree whose nodes are labeled with (instances of) elementary trees, and whose edges are labeled with (addresses of) nodes at which substitution or adjunction takes place. More specifically, an edge v →uv′ in d represents the information that the elementary tree at v′ is substituted at or adjoined into node u of the elementary tree at v. When we combine the elementary trees of our TAG as specified by d, we obtain a (unique) phrase structure tree called the derived tree associated with d, which we denote as t(d).
We use the symbol γ as a variable ranging over elementary trees, α as a variable ranging over initial trees, and β as a variable ranging over auxiliary trees. We also use the symbols u and v as variables ranging over nodes of generic trees (elementary, derived, or derivation trees). For an elementary tree γ, a derivation tree d is said to have type γ if the root node of d is labeled with γ. A derivation tree d is called sentential if d is of some type γ, and the root node of γ is labeled with the start symbol of the grammar, denoted as S.
A node u in an elementary tree γ may be annotated with an adjunction constraint, which for purposes here is a label in the set {NA, OA}. The label NA denotes Null Adjunction, forbidding adjunction at u; the label OA denotes Obligatory Adjunction, forcing adjunction at u. A derivation tree d is called saturated if, at each node v of d there is an arc v →uv′, for some v′, for every node u of the elementary tree at v that requires substitution or is annotated with an OA constraint.
An elementary tree γ of G is called useless if γ never occurs in a sentential and saturated derivation tree of G, that is, if no sentential and saturated derivation of G uses γ. A grammar G is called reduced if none of its elementary trees is useless. Throughout this article we shall assume that the grammars that we deal with are reduced.
2.2 Lexicalization
In a tree, a node labeled with a terminal symbol is called a lexical node. A TAG is called lexicalized if each of its elementary trees has at least one lexical node. Observe that a lexicalized grammar cannot generate the empty string, denoted by ε, because every derived tree yields at least one lexical element. Similarly, a lexicalized grammar is always finitely ambiguous, because the length of the generated strings provides an upper bound on the size of the associated derived trees. Let and
be two subclasses of the class of all TAGs. We say that
strongly lexicalizes
, if, for every grammar
that is finitely ambiguous and that satisfies
, there exists a lexicalized grammar
such that T(G′) = T(G). We also say that
is closed under strong lexicalization if the class
strongly lexicalizes itself.
Using this terminology, we can now restate the two main results obtained by Schabes (1990) about strong lexicalization for subclasses of TAGs, already mentioned in the Introduction. The first result states that the class of CFGs is not closed under strong lexicalization. Here we view a CFG as a special case of a TAG using only substitution and elementary trees of depth one. Informally, this means that we cannot cast a CFG G in a special form in which each rule has an overt lexical item in its right-hand side, under the restriction that the new grammar generates exactly the same tree set as G. The second result is that the class of TAGs strongly lexicalizes the class of tree substitution grammars (TSGs). The latter class is defined as the class of all TAGs that use substitution as the only tree combination operation, and thus includes all context-free grammars. This means that, given a TSG or a CFG G, we can always construct a TAG that is lexicalized and that generates exactly the same tree set as G.
3. Tree Substitution Grammars Are Not Closed Under Strong Lexicalization
Before turning to our main result in Section 4, we find it useful to technically revisit the related result for TSGs.
Theorem 1
Tree substitution grammars are not closed under strong lexicalization.

We use the following notation. For a derived tree t and a terminal symbol a, we write Nodes(a, t) to denote the set of all nodes in t that are labeled with a. Furthermore, for a node u of t we write depth(u, t) to denote the length of the unique path from the root node of t leading to u.
3.1 Intuition
In order to convey the basic idea behind Schabes's proof and our alternative version herein, we first consider a specific candidate grammar for the lexicalization of G1. For example, one might think that the following TSG G1′ lexicalizes G1:
This grammar is obtained from G1 by taking the lexicalized tree α1, as well as every elementary tree that can be obtained by substituting α1 into the non-lexicalized tree α2. The grammar G1′ only generates a subset of the trees generated by G1, however. The following tree, for example, cannot be generated by G1′:To see this, we reason as follows. Consider a lexical node v in an elementary tree γ of G′1, and let t be a tree obtained by substituting some elementary tree into γ. Because substitution takes place at the frontier of γ, depth(v, t) must be the same as depth(v, γ). More generally, the depth of a lexical node in an elementary tree γ is the same in all trees derived starting from γ. Because the maximal depth of a lexical node in an elementary tree of G′1 is 2, we deduce that every tree generated by G′1 contains a lexical node with depth at most 2. In contrast, all lexical nodes in the tree t1 have depth 3. Therefore the tree t1 is not generated by G′1.3.2 Main Part




Lemma 1
The grammar G1 has infinite d-index. Every lexicalized grammar in has finite d-index.
Proof
The first statement is easy to verify: Using longer and longer derivations, the minimal depth of an a-labeled node in the corresponding tree can be pushed beyond any bound.
To prove the second statement, let G be a lexicalized grammar in , and let t ∈ T(G). The tree t is derived starting from some initial tree; call this tree α. Because G is lexicalized, at least one of the a-labeled nodes in Nodes(a, t) is contributed by α. Let va be any such node in t, and let ua be the node of α that corresponds to va. Remember that the only tree combination operation allowed in a TSG derivation is substitution. Because substitution can only take place at the frontier of a derived tree, we must conclude that depth(va, t) = depth(ua, α). There are only finitely many initial trees in G, therefore depth(ua, α) must be upper bounded by some constant depending only on G, and the same must hold for depth(va, t). Lastly, because t has been arbitrarily chosen in T(G), we must conclude that d-index(G) is finite.▪
3.3 Lexicalization of Tree Substitution Grammars
4. Tree-Adjoining Grammars Are Not Closed Under Strong Lexicalization
In this section we develop the proof of the main result of this article.
Theorem 2
Tree-adjoining grammars are not closed under strong lexicalization.
4.1 Proof Idea
The basic idea underlying the proof of Theorem 2 is essentially the same as the one used in the proof of Theorem 1 in Section 3. Some discussion of this issue is in order at this point. In the previous section, we have seen that adjunction, in contrast to substitution, allows the insertion of additional structure at internal nodes of elementary trees, and enables TAGs to provide a strong lexicalization of TSGs. One might now be tempted to believe that, because the depth-based argument that we used in the proof of Lemma 1 can no longer be applied to TAGs, they might be closed under strong lexicalization.
There is a perspective under which adjunction quite closely resembles substitution, however. Let us first look at substitution as an operation on the yield of the derived tree. Under this view, substitution is essentially context-free rewriting: It replaces a nonterminal symbol in the yield of a derived tree with a new string consisting of terminals and nonterminals, representing the yield of the tree that is substituted. Under the same perspective, adjunction is more powerful than tree substitution, as is well known. But just as substitution can be seen as context-free rewriting on tree yields, adjunction can be seen as context-free rewriting on the paths of trees: It replaces a nonterminal symbol in some path of a derived tree with a string representing the spine of the tree that is adjoined—the unique path from the root node of the tree to the foot node.
A derived tree of G1, and the corresponding encoding, drawn from left to right. Every internal node of the original tree is represented by a pair of matching brackets [,
]. The correspondence is indicated by the numerical subscripts.
A derived tree of G1, and the corresponding encoding, drawn from left to right. Every internal node of the original tree is represented by a pair of matching brackets [,
]. The correspondence is indicated by the numerical subscripts.
4.2 Preliminaries



Grammar G2 derives trees that we call right spinal: Each node in such a tree has at most two children, and the left child of every node with two children is always a leaf node. The path from the root node of a right spinal tree t to the rightmost leaf of t is called spine. To save some space, in the following we write right spinal trees horizontally and from left to right, as already done in Figure 3. Thus the grammar G2 can alternatively be written as follows:




It is easy to prove by induction that, for each tree t ∈ T(G2), the excess of the sequence of nodes in the spine of t is always zero. Thus, we omit the proof of the following statement.
Lemma 2
Every derived tree t ∈ T(G2) is a right spinal tree, and excess(t) = 0.
In order to get a better understanding of the construction used in the following proofs, it is useful at this point to come back to our discussion of the relation between that construction and the construction presented in Section 3. We observe that for each tree t1 generated by G1 there is a tree t2 ∈ T(G2) such that the sequence of labels in t2's spine encodes t1, following the scheme exemplified in Figure 3. Using such encoding, we can establish a bijection between the a-labeled nodes in the frontier of t1 and the a-labeled nodes in the frontier of t2. Furthermore, if v1 in t1 and v2 in t2 are two nodes related by such a correspondence, then it is not difficult to see that depth(v1, t1) = excess(v2, t2).
4.3 Intuition
Before we give the actual proof of Theorem 2, let us attempt to get some intuition about why our counterexample grammar G2 cannot be strongly lexicalized by some other TAG. One might think that the following TAG G′2 is a lexicalized version of G2:
This grammar is obtained from G2 by taking the lexicalized tree β3 (repeated here as β5), as well as all trees that can be obtained by adjoining β3 into some non-lexicalized elementary tree. G′2 does not generate all trees generated by G2, however. The following tree t2 for example is not generated by G′2: Note that this tree is the encoded version of the counterexample tree t1 from the previous section (cf. Figure 3).To see that t2 is not generated by G2′, we reason as follows. Consider a lexical node u in an elementary tree γ of G2′, and let t be a tree obtained by adjoining some elementary tree into γ. Although this adjunction increases the depth of u, it does not increase its excess, as it adds a balanced sequence of parentheses into the spine of γ. More generally, the excess of a lexical node in an elementary γ is constant in all trees derived starting from γ. From this we conclude that every tree generated by G′2 contains a lexical node with excess at most 2; this is the maximal excess of a lexical node in an elementary tree of G′2. In contrast, all lexical nodes in the tree t2 have excess 3. This shows that t2 is not generated by G′2.
4.4 Main Part


There is one last obstacle that we need to overcome. For TSGs we noted (in the proof of Lemma 1) that the minimal depth of lexical nodes in a derived tree t is bounded by the minimal depth of lexical nodes in the elementary tree γ from which t was derived. For the TAG s in , the situation is not quite as simple, as an adjunction of an auxiliary tree β into an elementary tree γ might affect the excess of a lexical node of γ. It turns out, however, that this potential variation in the excess of a lexical node of γ is bounded by a grammar-specific constant. This observation is expressed in the following lemma. It is the correspondent of Lemma 4 in Knuth's paper on parenthesis languages (Knuth 1967), and is proved in essentially the same way. Recall that a derivation tree d is of type γ, γ some elementary tree, if d is derived starting from γ.
Lemma 3
Let . For each elementary tree γ of G, there exists a number e(γ) such that, for every saturated derivation tree d of type γ, excess(t(d)) = e(γ).
Proof
Because γ is not useless, we can find at least one sentential and saturated derivation tree of G that contains an occurrence of γ. Let d be any such derivation tree, and let v be any node of d labeled with γ. Let d1 be the subtree of d rooted at v. Observe that t(d1) must be a spinal tree. We then let e(γ) = excess(t(d)).

Using Lemma 3, we can now prove the following result.
Lemma 4
The grammar G2 has infinite e-index. Every lexicalized grammar in has finite e-index.
Proof

Let d be a sentential and saturated derivation tree of G. It has the following shape:
Here α is some initial tree, m ≥ 0, each ui is a node of α at which a tree combination operation takes place, each γi is an elementary tree, and each di is a derivation tree of type γi that is a subtree of d. According to this derivation tree, the derived tree t(d) is obtained by substituting or adjoining the derived trees t(di) at the respective nodes ui of α.

Two grammars in that have a different e-index cannot generate the same tree language, thus we have concluded the proof of Theorem 2.
5. Tree Insertion Grammars Are Not Closed Under Strong Lexicalization
As mentioned earlier Schabes and Waters (1995) introduce a restricted variant of TAG called TIG. The essential restriction in that formalism is the absence of wrapping trees, which are trees derived starting from auxiliary trees with overt lexical material on both sides of the foot node. Schabes and Waters Schabes and Waters (1995, Section 5.1.4) conjecture that the class of all TIGs is closed under strong lexicalization.
It is easy to see that the counterexample grammar G2 that we gave in Figure 4 does not derive wrapping trees; this means that G2 actually is a TIG. Using the proof of Section 4, we then obtain the following result.
Theorem 3
Tree insertion grammars are not closed under strong lexicalization.
In fact, we have even proved the stronger result that the class of TAGs does not lexicalize the class of TIGs.
6. Conclusion
We have shown that, in contrast to what has been claimed in the literature, TAGs are not closed under strong lexicalization: The restriction to lexicalized TAGs involves a loss in strong generative capacity.
In this article we have only considered TAGs with Null Adjunction and Obligatory Adjunction constraints. A third kind of adjunction constraint that has been used in the literature is Selective Adjunction, where a set of trees is provided that may be adjoined at some node. It is not difficult to see that the proofs of Lemma 3, Lemma 4, and Theorem 3 still hold if Selective Adjunction constraints are used.
Our result triggers a number of follow-up questions. First, are TAGs closed under weak lexicalization, defined in Section 1? We know that, in the case of CFGs, this question can be answered affirmatively, because Greibach normal form is a special case of lexicalized form, and for every CFG there is a weakly equivalent grammar in Greibach normal form. But to our knowledge, no comparable result exists for TAG. Second, if TAGs cannot strongly lexicalize themselves, what would a grammar formalism look like that is capable of providing strong lexicalization for TAGs?
Acknowledgements
We are grateful to Aravind Joshi for discussion on previous versions of this article and for helping us in shaping the text in the Introduction of the current version. We also acknowledge three anonymous reviewers for their helpful comments.
References
Author notes
Department of Linguistics and Philology, Box 635, 75126 Uppsala, Sweden. E-mail: [email protected].
Department of Information Engineering, via Gradenigo 6/A, 35131 Padova, Italy. E-mail: [email protected].