Hybrid Grammars for Parsing of Discontinuous Phrase Structures and Non-Projective Dependency Structures

We explore the concept of hybrid grammars, which formalize and generalize a range of existing frameworks for dealing with discontinuous syntactic structures. Covered are both discontinuous phrase structures and non-projective dependency structures. Technically, hybrid grammars are related to synchronous grammars, where one grammar component generates linear structures and another generates hierarchical structures. By coupling lexical elements of both components together, discontinuous structures result. Several types of hybrid grammars are characterized. We also discuss grammar induction from treebanks. The main advantage over existing frameworks is the ability of hybrid grammars to separate discontinuity of the desired structures from time complexity of parsing. This permits exploration of a large variety of parsing algorithms for discontinuous structures, with different properties. This is confirmed by the reported experimental results, which show a wide variety of running time, accuracy, and frequency of parse failures.

Much of the theory of parsing assumes syntactic structures that are trees, formalized such that the children of each node are ordered, and the yield of a tree, that is, the leaves read from left to right, is the sentence. In different terms, each node in the hierarchical syntactic structure of a sentence corresponds to a phrase that is a list of adjacent words, without any gaps. Such a structure is easy to represent in terms of bracketed notation, which is used, for instance, in the Penn Treebank (Marcus, Santorini, and Marcinkiewicz, 1993).

Describing syntax in terms of such narrowly defined trees seems most appropriate for relatively rigid word-order languages such as English. Nonetheless, the aforementioned Penn Treebank of English contains traces and other elements that encode additional structure next to the pure tree structure as indicated by the brackets. This is in keeping with observations that even English cannot be described adequately without a more general form of trees, allowing for so-called discontinuity (McCawley, 1982; Stucky, 1987). In a discontinuous structure, the set of leaves dominated by a node of the tree need not form a contiguous sequence of words, but may comprise one or more gaps. The need for discontinuous structures tends to be even greater for languages with relatively free word order (Kathol and Pollard 1995; Müller 2004).

In the context of dependency parsing (Kübler, McDonald, and Nivre 2009), the more specific term non-projectivity is used instead of, or next to, discontinuity. See Rambow (2010) for a discussion of the relation between constituent and dependency structures and see Maier and Lichte (2009) for a comparison of discontinuity and non-projectivity. As shown by, for example, Hockenmaier and Steedman (2007) and Evang and Kallmeyer (2011), discontinuity encoded using traces in the Penn Treebank can be rendered in alternative, and arguably more explicit, forms. In many modern treebanks, discontinuous structures have been given a prominent status (e.g., Böhmová et al. 2000). Figure 1 shows an example of a non-projective dependency structure.

The most established parsing algorithms are compiled out of context-free grammars (CFGs), or closely related formalisms such as tree substitution grammars (Sima’an et al. 1994) or regular tree grammars (Brainerd 1969; Gécseg and Steinby 1997). These parsers, which have a time complexity of $O(n3)$ for n being the length of the input string, operate by composing adjacent substrings of the input sentence into longer substrings. As a result, the structures they can build directly do not involve any discontinuity. The need for discontinuous syntactic structures thus poses a challenge to traditional parsing algorithms.

One possible solution is commonly referred to as pseudo-projectivity in the literature on dependency parsing (Kahane, Nasr, and Rambow, 1998; Nivre and Nilsson, 2005; McDonald and Pereira, 2006). A standard parsing system is trained on a corpus of projective dependency structures that was obtained by applying a lifting operation to non-projective structures. In a first pass, this system is applied to unlabeled sentences and produces projective dependencies. In a second pass, the lifting operation is reversed to introduce non-projectivity. A related idea for discontinuous phrase structures is the reversible splitting conversion of Boyd (2007). See also Johnson (2002), Campbell (2004), and Gabbard, Kulick, and Marcus (2006).

The two passes of pseudo-projective dependency parsing need not be strictly separated in time. For example, one way to characterize the algorithm by Nivre (2009) is that it combines the first pass with the second. Here the usual one-way input tape is replaced by a buffer. A non-topmost element from the parsing stack, which holds a word previously read from the input sentence, can be transferred back to the buffer, and thereby input positions can be effectively swapped. This then results in a non-projective dependency structure.

A second potential solution to obtain syntactic structures that go beyond context-free power is to use more expressive grammatical formalisms. One approach proposed by Reape (1989, 1994) is to separate linear order from the parent–child relation in syntactic structure, and to allow shuffling of the order of descendants of a node, which need not be its direct children. The set of possible orders is restricted by linear precedence constraints. A further restriction may be imposed by compaction (Kathol and Pollard, 1995). As discussed by Fouvry and Meurers (2000) and Daniels and Meurers (2002), this may lead to exponential parsing complexity; see also Daniels and Meurers (2004). Separating linear order from the parent–child relation is in the tradition of head-driven phrase structure grammar (HPSG), where grammars are commonly hand-written. This differs from our objectives to induce grammars automatically from training data, as will become clear in the following sections.

To stay within a polynomial time complexity, one may also consider tree adjoining grammars (TAGs), which can describe strictly larger classes of word order phenomena than CFGs (Rambow and Joshi, 1997). The resulting parsers have a time complexity of $O(n6)$ (Vijay-Shankar and Joshi, 1985). However, the derived trees they generate are still continuous. Although their derivation trees may be argued to be discontinuous, these by themselves are not normally the desired syntactic structures. Moreover, it was argued by Becker, Joshi, and Rambow (1991) that further additions to TAGs are needed to obtain adequate descriptions of certain non-context-free phenomena. These additions further increase the time complexity.

In order to obtain desired syntactic structures, one may combine TAG parsing with an idea that is related to that of pseudo-projectivity. For example, Kallmeyer and Kuhlmann (2012) propose a transformation that turns a derivation tree of a (lexicalized) TAG into a non-projective dependency structure. The same idea has been applied to derivation trees of other formalisms, in particular (lexicalized) linear context-free rewriting systems (LCFRSs) (Kuhlmann, 2013), whose weak generative power subsumes that of TAGs.

Parsers more powerful than those for CFGs often incur high time costs. In particular, LCFRS parsers have a time complexity that is polynomial in the sentence length, but with a degree that is determined by properties of the grammar. This degree typically increases with the amount of discontinuity in the desired structures. Difficulties in running LCFRS parsers for natural languages are described, for example, by Kallmeyer and Maier (2013).

In the architectures we have discussed, the common elements are:

• • 

  a grammar, in some fixed formalism, that determines the set of sentences that are accepted, and

• • 

  a procedure to build (discontinuous) structures, guided by the derivation of input sentences.

The purpose of this article is to explore a theoretical framework that allows us to capture a wide range of parsing architectures that all share these two common elements. At the core of this framework lies a formalism called hybrid grammar, introduced in Nederhof and Vogler (2014). Such a grammar consists of a string grammar and a tree grammar. Derivations are coupled, as in synchronous grammars (Shieber and Schabes, 1990; Satta and Peserico, 2005). In addition, each occurrence of a terminal symbol in the string grammar is coupled to an occurrence of a terminal symbol in the tree grammar. The string grammar defines the set of accepted sentences. The tree grammar, whose rules are tied to the rules of the string grammar for synchronous rewriting, determines the resulting syntactic structures, which may be discontinuous. The way the syntactic structures are obtained critically relies on the coupling of terminal symbols in the two component grammars, which is where our theory departs from that of synchronous grammars. A hybrid grammar generates a set of hybrid trees;¹Figure 2 shows an example of a hybrid tree, which corresponds to the non-projective dependency structure of Figure 1.

The general concept of hybrid grammars leaves open the choice of the string grammar formalism and that of the tree grammar formalism. In this article we consider simple macro grammars (Fischer, 1968) and LCFRSs as string grammar formalisms. The tree grammar formalisms we consider are simple context-free tree grammars (Rounds, 1970) and simple definite clause programs (sDCP), inspired by Deransart and Małuszynski (1985). This gives four combinations, each leading to one class of hybrid grammars. In addition, more fine-grained subclasses can be defined by placing further syntactic restrictions on the string and tree formalisms.

To place hybrid grammars in the context of existing parsing architectures, let us consider classical grammar induction from a treebank, for example, for context-free grammars (Charniak, 1996) or for LCFRSs (Maier and Søgaard, 2008; Kuhlmann and Satta, 2009). Rules are extracted directly from the trees in the training set, and unseen input strings are consequently parsed according to these structures. Grammars induced in this way can be seen as restricted hybrid grammars, in which no freedom exists in the relation between the string component and the tree component. In particular, the presence of discontinuous structures generally leads to high time complexity of string parsing. In contrast, the framework in this article detaches the string component of the grammar from the tree component. Thereby the parsing process of input strings is no longer bound to follow the tree structures, while the same tree structures as before can still be produced, provided the tree component is suitably chosen. This allows string parsing with low time complexity in combination with production of discontinuous trees.

Nederhof and Vogler (2014) presented experiments with various subclasses of hybrid grammars for the purpose of constituent parsing. Trade-offs between speed and accuracy were identified. In the present article, we extend our investigation to dependency parsing. This includes induction of a hybrid grammar from a dependency treebank. Before turning to the experiments, we present several completeness results about existence of hybrid grammars generating non-projective dependency structures.

This article is organized as follows. After preliminaries in Section 2, Section 3 defines hybrid trees. These are able to capture both discontinuous phrase structures and non-projective dependency structures. Thanks to the concept of hybrid trees, there will be no need, later in the article, to distinguish between hybrid grammars for constituent parsing and hybrid grammars for dependency parsing. To make this article self-contained, we define two existing string grammar formalisms and two tree grammar formalisms in Section 4. The four classes of hybrid grammars that result by combining these formalisms are presented in Section 5, which also discusses how to use them for parsing.

How to induce hybrid grammars from treebanks is discussed in Section 6. Section 7 reports on experiments that provide proof of concept. In particular, LCFRS/sDCP-hybrid grammars are induced from corpora of dependency structures and phrase structures and employed to predict the syntactic structure of unlabeled sentences. It is demonstrated that hybrid grammars allow a wide variety of results, in terms of time complexity, accuracy, and frequency of parse failures. How hybrid grammars relate to existing ideas is discussed in Section 8.

The text refers to a number of theoretical results that are not central to the main content of this article. In order to preserve the continuity of the discussion, we have deferred their proofs to appendices.

Let ℕ = {0,1,2, …} and ℕ ₊ = ℕ ∖{0}. For each n ∈ ℕ ₊, we let [n] stand for the set {1, … , n}, and we let [0] stand for ∅. We write [n]₀ to denote [n] ∪{0}. We fix an infinite list x₁,x₂, … of pairwise distinct variables. We let X = {x₁,x₂,x₃, …} and X_k = {x₁, … , x_k} for each k ∈ ℕ. For any set A, the power set of A is denoted by $P(A)$⁠.

A ranked set Δ is a set of symbols associated with a rank function assigning a number rk_Δ(δ) ∈ ℕ to each symbol δ ∈ Δ. A ranked alphabet is a ranked set with a finite number of symbols. We let Δ^(k) denote {δ ∈ Δ∣rk_Δ(δ) = k}.

The following definitions were inspired by Seki and Kato (2008). The sets of terms and sequence-terms (s-terms) over ranked set Δ, with variables in some set Y ⊆ X, are denoted by T_Δ(Y ) and $TΔ*(Y)$⁠, respectively, and defined inductively as follows:

• Y ⊆ T_Δ(Y ),

• if k ∈ ℕ, δ ∈ Δ^(k) and $si∈TΔ*(Y)$ for each i ∈ [k], then δ(s₁, … , s_k) ∈ T_Δ(Y), and

• if n ∈ ℕ and t_i ∈ T_Δ(Y ) for each i ∈ [n], then $〈t1,…,tn〉∈TΔ*(Y)$⁠.

We let $TΔ*$ and T_Δ stand for $TΔ*(∅)$ and T_Δ(∅), respectively. Throughout this article, we use variables such as s and s_i for s-terms and variables such as t and t_i for terms. The length |s| of a s-term s = 〈t₁, … , t_n〉 is n.

The justification for using s-terms as defined here is that they provide the required flexibility for dealing with both strings and unranked trees, in combination with derivational nonterminals in various kinds of grammar. By using an alphabet Δ =Δ⁽⁰⁾ one can represent strings. For instance, if Δ contains the symbols a and b, then the s-term 〈a(),b(),a()〉 denotes the string ab a. We will therefore refer to such s-terms simply as strings.

By using an alphabet Δ =Δ⁽¹⁾ one may represent trees without fixing the number of child nodes that a node with a certain label should have. Conceptually, one may think of such node labels as unranked, as is common in parsing theory of natural language. For instance, the s-term 〈a(〈b(〈〉),a(〈〉)〉)〉 denotes the unranked tree a(b,a). We will therefore refer to such s-terms simply as trees, and we will sometimes use familiar terminology, such as “node,” “parent,” and “sibling,” as well as common graphical representations of trees.

If theoretical frameworks require trees over ranked alphabets in the conventional sense (without s-terms), one may introduce a distinguished symbol cons of rank 2, replacing each s-term of length greater than 1 by an arrangement of subterms combined using occurrences of that symbol. Another symbol nil of rank 0 may be introduced to replace each s-term of length 0. Hence δ(〈α(〈〉),β(〈〉),γ(〈〉)〉) could be more conventionally written as δ(cons(α(nil),cons(β(nil),γ(nil)))).

Concatenation of s-terms is given by 〈t₁, … , t_n〉⋅〈t_n+1, … , t_n+m〉 = 〈t₁, … , t_{n +m}〉. Sequences such as s₁, … , s_k or x₁, … , x_k will typically be abbreviated to s_1,k or x_1,k, respectively. For δ ∈ Δ⁽⁰⁾ we sometimes abbreviate δ() to δ.

In examples we also abbreviate 〈t₁, … , t_n〉 to t₁⋯t_n—that is, omitting the angle brackets and commas. In particular, for n = 0, the s-term 〈〉 is abbreviated by ε. Moreover, we sometimes abbreviate δ(〈〉) to δ. Whether δ then stands for δ(〈〉) or for δ() depends on whether δ ∈ Δ⁽¹⁾ or δ ∈ Δ⁽⁰⁾, which will be clear from the context.

The subterm at position p in a term t (or a s-term s) is defined as follows. For any term t we have t|_ε = t. For a term t = δ(s_1,k), i ∈ [k],p ∈pos(s_i) we have t|_ip =s_i|_p. For a s-term s = 〈t_1,n〉, i ∈ [n],p ∈pos(t_i) we have s|_ip =t_i|_p.

The label at position p in a term t is denoted by t(p). In other words, if t|_p equals δ(s₁, … , s_k) or x ∈ X, then t(p) equals δ or x, respectively. Let Γ ⊆ Δ. The subset of pos(t) consisting of all positions where the label is in Γ is denoted by pos_Γ(t), or formally pos_Γ(t) = {p ∈ pos(t)∣t(p) ∈ Γ}. Analogously to t(p) and pos_Γ(t) for terms t, one may define s(p) and pos_Γ(s) for s-terms s.

The expression t[s′]_p (or s[s′]_p) denotes the s-term obtained from t (or from s) by replacing the subterm at position p by s-term s′. For any term t we have t[s′]_ε = s′. For a term t = δ(s_1,k), i ∈ [k],p ∈pos(s_i) we have t[s′]_ip =〈δ(s_1,i−1,s_i[s′]_p,s_{i +1,k})〉. For a s-term s = 〈t_1,n〉, i ∈ [n],p ∈pos(t_i) we have s[s′]_ip = 〈t_1,i−1〉⋅ t_i[s′]_p ⋅〈t_{i +1,n}〉.

For term t (or s-term s) with variables in X_k and s-terms s_i (i ∈ [k]), the first-order substitutiont[s_1,k] (or s[s_1,k], respectively) denotes the s-term obtained from t (or from s) by replacing each occurrence of any variable x_i by s_i, or formally:

• • 

  x_i[s_1,k] = s_i,

• • 

  δ(s′_1,n)[s_1,k] = 〈δ(s′₁[s_1,k], … , s′_n[s_1,k])〉,

• • 

  〈t_1,n〉[s_1,k] = t₁[s_1,k] ⋅… ⋅ t_n[s_1,k].

If $s∈TΔ*$⁠, s|_p = δ(s_1,k) and $s′∈TΔ*(Xk)$⁠, then the second-order substitution$s〚s′〛p$ denotes the s-term obtained from s by replacing the subterm at position p by s′, with the variables in s′ replaced by the corresponding s-terms found immediately below p, or formally $s〚s′〛p=s[s′[s1,k]]p$⁠.

The purpose of this section is to unify existing notions of non-projective dependency structures and discontinuous phrase structures, formalized using s-terms.

We fix a ranked alphabet Σ = Σ⁽¹⁾ and a subset Γ ⊆ Σ. A hybrid tree over (Γ,Σ) is a pair h = (s,≤_s), where $s∈TΣ*$ and ≤_s is a total order on pos_Γ(s). In words, a hybrid tree combines hierarchical structure, in the form of a s-term over the full alphabet Σ, with a linear structure, which can be seen as a string over Γ ⊆ Σ. This string will be denoted by str(h). Formally, let pos_Γ(s) = {p₁, … , p_n} with p_i ≤_sp_{i +1} (i ∈ [n − 1]). Then str(h) = s(p₁)⋯s(p_n). In order to avoid the treatment of pathological cases we assume that s≠〈〉 and pos_Γ(s)≠∅.

A hybrid tree (s,≤_s) is a phrase structure if ≤_s is a total order on the leaves of s. The elements of Γ would typically represent lexical items, and the elements of Σ ∖ Γ would typically represent syntactic categories. A hybrid tree (s,≤_s) is a dependency structure if Γ = Σ, whereby the linear structure of a hybrid tree involves all of its nodes, which represent lexical items. Our dependency structures generalize totally ordered trees (Kuhlmann and Niehren, 2008) by considering s-terms instead of usual terms over a ranked alphabet.

We say that a phrase structure (s,≤_s) over (Γ,Σ) is continuous if for each p ∈pos(s) the set pos_Γ(s|_p) is a complete span, that is, if the following condition holds: if p₁, p₂, p′ satisfy pp₁, p′, pp₂ ∈pos_Γ(s) and pp₁ ≤_sp′≤_spp₂, then p′ = pp₃ for some p₃. If the same condition holds for a dependency structure (s,≤_s), then we say that (s,≤_s) is projective. If the condition is not satisfied, then we call a phrase structure discontinuous and a dependency structure non-projective.

The concept of hybrid grammars is illustrated in Section 5, first on the basis of a coupling of linear context-free rewriting systems and simple definite clause programs, and then three more such couplings are introduced that further involve simple macro grammars and simple context-free tree grammars. In the current section we discuss these basic classes of grammars, starting with the simplest ones.

The definitions in this section are very close to those in Fischer (1968) with the difference that the notational framework of s-terms is used for strings, as in Seki and Kato (2008).

We write $⇒Gp,ρ$ for the “derives” relation, using rule $ρ=A(x1,k)→r$ at position p in a s-term. Formally, we write $s1⇒Gp,ρs2$ if $s1∈TN∪Γ*$⁠, s₁(p) = A and $s2=s1〚r〛p$⁠. We write s₁ ⇒_Gs₂ if $s1⇒Gp,ρs2$ for some p and ρ, and $⇒G*$ is the reflexive, transitive closure of ⇒_G. Derivation in i steps is denoted by $⇒Gi$⁠. The (string) language induced by macro grammar G is $[G]={s∈TΓ*∣〈S〉⇒G*s}$⁠.

In the sequel we will focus our attention on macro grammars with the property that for each rule $A(x1,k)→r$ and each i ∈ [k], variable x_i has exactly one occurrence in r. In this article, such grammars will be called simple macro grammars (sMGs).

The definitions in this section are a slight generalization of those in Rounds (1970) and Engelfriet and Schmidt (1977,1978), as here they involve s-terms. In Section 3 we already argued that the extra power due to s-terms can be modeled using fixed symbols cons and nil and is therefore not very significant in itself. The benefit of the generalization lies in the combination with macro grammars, as discussed in Section 5.

A (generalized) context-free tree grammar (CFTG) is a tuple G =(N,S,Σ,P), where Σ is a ranked alphabet with Σ = Σ⁽¹⁾ and N, S, and P are as for macro grammars except that Γ is replaced by Σ in the specification of the rules.

The “derives” relation $⇒Gp,ρ$ and other relevant notation are defined as for macro grammars. Note that the language $[G]={s∈TΣ*∣〈S〉⇒G*s}$ induced by a CFTG G is not a string language but a tree language, or more precisely, its elements are sequences of trees.

As for macro grammars, we will focus our attention on CFTGs with the property that for each rule $A(x1,k)→r$ and each i ∈ [k], variable x_i has exactly one occurrence in r. In this article, such grammars will be called simple context-free tree grammars (sCFTGs). Note that if N = N⁽⁰⁾, then a sCFTGs is a regular tree grammar (Brainerd 1969; Gécseg and Steinby 1997). sCFTGs are a natural generalization of the widely used TAGs; see Kepser and Rogers (2011), Maletti and Engelfriet (2012), and Gebhardt and Osterholzer (2015).

In Vijay-Shanker, Weir, and Joshi (1987), the semantics of LCFRS is introduced by distinguishing two phases. In the first phase, a tree over function symbols is generated by a regular tree grammar. In the second phrase, the function symbols are interpreted, each composing a sequence of tuples of strings into another tuple of strings. This formalism is equivalent to the multiple CFGs of Seki et al. (1991). We choose a notation similar to that of the formalisms discussed before, which will also enable us to couple these string-generating grammars to tree-generating grammars, as will be discussed later.

A derivation can be represented by a derivation treed (cf. Figure 6), which is obtained by glueing together the rules as they are used in the derivation. The backbone of d is a usual derivation tree of the CFG underlying the LCFRS (nonterminals and solid lines). Each argument of a nonterminal is represented as a box to the right of the nonterminal, and dashed arrows indicate dependencies of values. An x_i above a box specifies an argument of a nonterminal on the right-hand side of a rule, whereas the content of a box is an argument of the nonterminal on the left-hand side of a rule. The boxes taken as vertices and the dashed arrows taken as edges constitute a dependency graph. It can be evaluated by first sorting its vertices topologically and, according to this sorting, substituting in an obvious manner the relevant s-terms into each other. The final s-term that is evaluated in this way is an element of the language [G]. This s-term, denoted by ϕ(d), is called the evaluation of d. The notion of dependency graph originates from attribute grammars (Knuth, 1968; Paakki, 1995); it should not be confused with the linguistic concept of dependency.

Note that if the rules in a derivation are given, then the choice of r_i for each variable x_i in each rule instance is uniquely determined. For a given string s, the set of all LCFRS derivations (in compact tabular form) can be obtained in polynomial time in the length of s (Seki et al., 1991). See also Kallmeyer and Maier (2010, 2013) for the extension with probabilities.

There is a subclass of LCFRS called well-nested LCFRS. Its time complexity of parsing is lower than that of general LCFRS (Gómez-Rodríguez, Kuhlmann, and Satta 2010), and the class of languages it induces is strictly included in the class of languages induced by general LCFRSs (Kanazawa and Salvati, 2010). One can see sMGs as syntactic variants of well-nested LCFRSs (cf. footnote 3 of Kanazawa 2009), the former being more convenient for our purposes of constructing hybrid grammars, when the string component is to be explicitly restricted to have the power of sMG / well-nested LCFRS. The class of languages they induce also equals the class of string languages induced by sCFTGs.

In this section we describe a particular kind of definite clause program. Our definition is inspired by Deransart and Małuszynski (1985), who investigated the relation between logic programs and attribute grammars, together with the “syntactic single use requirement” from Giegerich (1988). The values produced are s-terms. The induced class of s-term languages is strictly larger than that of sCFTGs (cf. Appendix B).

As discussed subsequently, the class of string languages that results if we take the yields of those s-terms equals the class of string languages induced by LCFRSs. Thereby, our class of definite clause programs relates to LCFRSs much as the class of sCFTGs relates to sMGs.

A simple definite clause program (sDCP) is a tuple G =(N,S,Σ,P), where N is a ranked alphabet of nonterminals and Σ = Σ⁽¹⁾ is a ranked alphabet of terminals (as for CFTGs).² Moreover, each nonterminal A ∈ N has a number of arguments, each of which is either an inherited argument or a synthesized argument. The number of inherited arguments is the i-rank and the number of synthesized arguments is the s-rank of A; we let rk_N(A) =i-rk(A) +s-rk(A) denote the rank of A. The start symbolS has only one argument, which is synthesized—that is, rk_N(S) =s-rk(S) = 1 and i-rk(S) = 0.

The “derives” relation ⇒_G and other relevant notation are defined as for LCFRSs. Thus, in particular, the language induced by sDCP G is $[G]={s∈TΣ*∣〈S(s)〉⇒G*〈〉}$⁠, whose elements are now sequences of trees.

In the same way as for LCFRS we can represent a derivation of a sDCP G as derivation tree d. A box in its dependency graph is placed to the left of a nonterminal occurrence if it represents an inherited argument and to the right otherwise. As before, ϕ(d) denotes the evaluation of d, which is now a s-term over Σ.

If a sDCP is such that the dependency graph for any derivation contains no cycles, then we say the sDCP contains no cycles. In this case, if the rules in a derivation are given, then the choice of r_i for each variable x_i in each rule instance is uniquely determined, and can be computed in linear time in the size of the derivation. The existence of cycles is decidable, as we know from the literature on attribute grammars (Knuth, 1968). There are sufficient conditions for absence of cycles, such as the grammar being L-attributed (Bochmann, 1976; Deransart, Jourdan, and Lorho, 1988). In this article, we will assume that sDCPs contain no cycles. A one-pass computation model can be formalized as a bottom–up tree-generating tree-to-hypergraph transducer (Engelfriet and Vogler, 1998). One may alternatively evaluate sDCP arguments using the more general mechanism of unification, as it exists for example in HPSG (Pollard and Sag, 1994).

If the sDCP is single-synthesized—that is, each nonterminal has exactly one synthesized argument (and any number of inherited arguments)—then there is an equivalent sCFTG. The converse also holds. For proofs, see Appendix A.

On the basis of the flattening function, Appendix C proves the following result.

A hybrid grammar consists of a string grammar and a tree grammar. Intuitively, the string grammar is used for parsing a given string w, and the tree grammar simultaneously generates a hybrid tree h with str(h) = w. To synchronize these two processes, we couple derivations in the grammars in a way similar to how this is commonly done for synchronous grammars—namely, by indexed symbols. However, we apply the mechanism not only to derivational nonterminals but also to terminals.

Let Ω be a ranked alphabet. We define the ranked set , with . Let Δ be another ranked alphabet (Ω ∩ Δ = ∅) and Y ⊆ X. We let $IΩ,Δ*(Y)$ be the set of all s-terms $s∈TI(Ω)∪Δ*(Y)$ in which each index u occurs at most once.

For a s-term s, let ind(s) be the set of all indices occurring in s. The deindexing function $D$ removes all indices from a s-term $s∈IΩ,Δ*(Y)$ to obtain $D(s)∈TΩ∪Δ*(Y)$⁠. The set $IΩ,Δ(Y)$⊆$TI(Ω)∪Δ(Y)$ of terms with indexed symbols is defined much as above. We let $IΩ,Δ*$ =$IΩ,Δ*(∅)$ and $IΩ,Δ$ =$IΩ,Δ(∅)$⁠.

We first couple a LCFRS and a sDCP in order to describe a set of hybrid trees.

In order to define the “derives” relation $⇒Gu,ρ$⁠, for some index u and some rule ρ of the form of Equation (4), we need the additional notions of nonterminal reindexing and of terminal reindexing. The nonterminal reindexing is an injective function f_U that replaces each index at a nonterminal occurrence of the rule by one that does not clash with the indices in an existing set U ⊆ℕ ₊. We may, for example, define f_U such that it maps each v ∈ ℕ ₊ to the smallest v′ ∈ ℕ ₊ such that v′∉U ∪{f_U(1), … , f_U(v − 1)}. We extend f_U to apply to terms, s-terms, and rules in a natural way, to replace indices by other indices, but leaving all other symbols unaffected. A terminal reindexingg maps indices at occurrences of terminals in the rule to the indices of corresponding occurrences of terminals in the sentential form; we extend g to terms, s-terms, and rules in the same way as for f_U. The definition of f_U is fixed while an appropriate g needs to be chosen for each derivation step.

From a pair [s₁,s₂] ∈ [G], we can construct the hybrid tree (s,≤_s) over (Γ,Σ) by letting $s=D(s2)$ and, for each combination of positions p₁, p′₁, p₂, p′₂ such that $s1(p1)=s2(p2)∈I(Γ)$ and $s1(p1′)=s2(p2′)∈I(Γ)$⁠, we set p₂ ≤_sp′₂ if and only if p₁ ≤_ℓp′₁. (The lexicographical ordering ≤_ℓ on positions here simplifies to the linear ordering of integers, as positions in strings always have length 1.) In words, occurrences of terminals in s obtain a total order in accordance with the order in which corresponding terminals occur in s₁. The set of all such (s,≤_s) will be denoted by L(G).

A derivation of a LCFRS/sDCP hybrid grammar G can be represented by a derivation tree (cf. Figure 8). It combines a derivation tree of the first component of G and a derivation tree of its second component, letting them share the common parts. We make a graphical distinction between arguments of the first component (rectangles) and those of the second component (ovals). Implicit in a derivation tree is the reindexing of terminal indices. The evaluation ϕ₁ of the derivation tree d in Figure 8 yields the indexed string and the evaluation ϕ₂ of d yields the tree .

In order to illustrate the generality of our framework, we will sketch three more classes of hybrid grammars. In these three classes, the first component or the second component, or both, are less powerful than in the case of the LCFRS/sDCP hybrid grammars defined previously, and thereby the resulting hybrid grammars are less powerful, in the light of the observation that sMGs are syntactic variants of well-nested LCFRSs and sCFTGs are syntactic variants of sDCPs with s-rank restricted to 1. Noteworthy are the differences between the four classes of hybrid grammars in the formal definition of their derivations.

In the definition of the “derives” relation $⇒Gu,ρ$ we have to use a reindexing function. Because terminals are produced by a rule application (instead of being consumed as in the LCFRS/sDCP case), there is no need for a terminal reindexing that matches indices of terminal occurrences of the applied rule with those of the sentential form. Instead, terminal indices occurring in the rule have to be reindexed away from the sentential form in the same way as for nonterminal indices. Thus we use one reindexing function f_U that applies to nonterminal indices and terminal indices.

We define $[s1,s2]⇒Gu,ρ[s1′,s2′]$ for every $s1,s1′∈IN∪Γ,∅*$ and $s2,s2′∈IN∪Γ,Δ*$ if and only if:

• • 

  there are positions p₁ and p₂ such that and ,

• • 

  ρ ∈ P is $[A(x1,k)→r1,$$A(x1,k)→r2]$⁠,

• • 

  U = ind(s₁) ∖{u} = ind(s₂) ∖{u},

• • 

  $si′=si〚fU(ri)〛pi$ for i = 1,2.

In the usual way, we can extend LCFRS/sDCP hybrid grammars to become probabilistic LCFRS/sDCP hybrid grammars. For this we can assign a probability to each hybrid rule, under the constraint of a properness condition. More precisely, for each nonterminal A the probabilities of all hybrid rules with A in their left-hand sides sum up to 1. In this way a probabilistic LCFRS/sDCP hybrid grammar induces a distribution over hybrid trees. Thus it can be considered as a generative model. See also Nivre (2010) for a general survey of probabilistic parsing.

Algorithm 1 shows a parsing pipeline which takes as input a probabilistic LCFRS/sDCP hybrid grammar G and a sentence w ∈ Γ*. As output it computes the hybrid tree h ∈ L(G) that is derived by the most likely derivation whose first component derives w. In line 1 the first component of G is extracted. Because we later will restore the second component, we assume that a tag is attached to each LCFRS rule that uniquely identifies the original LCFRS/sDCP rule. This also means that two otherwise identical LCFRS rules are treated as distinct if they were taken from two different LCFRS/sDCP rules. In line 2 the string w is parsed. For this any standard LCFRS parser can be used. Such parsers, for instance those by Seki et al. (1991) and Kallmeyer (2010), typically run in polynomial time. The technical details of the chosen LCFRS parser (like the used form of items or the form of iteration over rules) are irrelevant, as for our framework only the functionality of the parsing component matters. The parsing algorithm builds a succinct representation of all derivations of w. In a second, linear-time phase the most likely derivation tree d for w is extracted. In line 3 the dependency graph of d is enriched by the inherited and synthesized arguments that correspond to the second components of the rules occurring in d; here, the identity of the original LCFRS/sDCP rule is needed. This results in an intermediate structure $d^$⁠. This is not yet a derivation tree of G, as the terminals still need to be assigned unique indices. This is done first by an indexing of the terminals from w in an arbitrary manner, and then by traversing the derivation to associate these indices with the appropriate terminal occurrences in the derivation, leading to a derivation tree d′ of the LCFRS/sDCP hybrid grammar G. Note that ϕ₁(d′) = s₁. Finally, in line 6 the derivation tree d′ is evaluated by ϕ₂ yielding a tree in $IΓ,Δ*$⁠.

For most practical applications, hybrid grammars would not be written by hand, but would be automatically extracted from finite corpora of hybrid trees, over which they should generalize. To be precise, the task is to construct a hybrid grammar G out of a corpus c such that c ⊆ L(G).

During grammar induction each hybrid tree h = (s,≤_s) of the corpus is decomposed. A decomposition will determine the structure of a derivation of h by the grammar. Classically, this decomposition and thereby the resulting derivations resemble the structure of s. This approach has been pursued, for instance, by Charniak (1996) for CFGs, and by Maier and Søgaard (2008) and Kuhlmann and Satta (2009) for LCFRSs. There is no guarantee, however, that this approach is optimal from the perspective of, for example, parsing efficiency, the grammar’s potential to generalize from training data, or the size of the grammar.

For a more general approach that offers additional degrees of freedom we extend the framework of Nederhof and Vogler (2014) and let grammar induction depend on a decomposition strategy of the string str(h), called recursive partitioning. One may choose one out of several such strategies. We consider four instances of induction of hybrid grammars, which vary in the type of hybrid trees that occur in the corpus, and the type of the resulting hybrid grammar:

A recursive partitioning of a string w of length n is a tree π whose nodes are labeled with subsets of [n]. The root of π is labeled with [n]. Each leaf of π is labeled with a singleton subset of [n]. Each non-leaf node has at least two children and is labeled with the union of the labels of its children, which furthermore must be disjoint. To fit recursive partitionings into our framework of s-terms, we regard $P([n])$ as ranked alphabet with $P([n])=P([n])(1)$⁠. We let $π∈TP([n])*$ with |π| = 1.

We say a set J ⊆ [n] has fanoutk if k is the smallest number such that J can be written as J = J₁ ∪… ∪ J_k, where:

• • 

  each J_ℓ (ℓ ∈ [k]) is of the form {i_ℓ,i_ℓ + 1, … , i_ℓ + m_ℓ}, for some i_ℓ and m_ℓ ≥ 0, and

• • 

  i ∈ J_ℓ, i′ ∈ J_ℓ′ (ℓ,ℓ′ ∈ [k]) and ℓ < ℓ′ imply i < i′.

Note that J₁, … , J_k are uniquely defined by this, and we write spans( J) = 〈J₁, … , J_k〉. The fanout of a recursive partitioning is defined as the maximal fanout of its nodes.

Figure 10 presents two recursive partitionings of a string with seven positions. The left one has fanout 3 (because of the node label {1,3,6,7} ={1}∪{3}∪{6,7}), whereas the right one has fanout 2.

Algorithm 2 constructs a LCFRS G out of a string w = α₁⋯α_n and a recursive partitioning π of w. The nonterminals are symbols of the form ⦇J⦈, where J is a node label from π. (In practice, the nonterminals ⦇J⦈ are replaced by other symbols, as will be explained in Section 6.8.) For each position p of π a rule is constructed. If p is a leaf labeled by {i}, then the constructed rule simply generates α_i where ⦇{i}⦈ has fanout 1 (lines 6 and 7). For each internal position labeled J₀ and its children labeled J₁, … , J_j we compute the spans of these labels (line 9). For each argument q of ⦇J₀⦈ a s-term s_q is constructed by analyzing the corresponding component J_0,q of spans( J₀) (lines 12–13). Here we exploit the fact that J_0,q can be decomposed in a unique way into a selection of sets that are among the spans of J₁, … , J_j. Each of these sets translates to one variable. The resulting grammar G allows for exactly one derivation tree that derives w and has the same structure as π. We say that G parses w according to π.

We observe that G as constructed in this way is in a particular normal form.³ To be precise, by the notation in Equation (2) each rule satisfies one of the following:

• • 

  n ≥ 2 and $s1,k0∈T∅*(Xmn)$⁠, (structural rule)

• • 

  n = 0, k₀ = 1, and s₁ = 〈α〉 for some α ∈ Γ. (terminal generating rule)

Conversely, each derivation tree d of a LCFRS in normal form can be translated to a recursive partitioning π_d, by processing d bottom–up as follows. Each leaf, that is, a rule A(〈α〉) →〈〉, is replaced by the singleton set {i} where i is the position of the corresponding occurrence of α in the accepted string. Each internal node is replaced by the union of the sets that were computed for its children. For the constructed LCFRS G that parses w according to π and for its only derivation tree d, we have π = π_d.

Observe that terminals are generated individually by the rules that were constructed from the leaves of the recursive partitioning but not by the structural rules obtained from internal nodes. Consequently, the LCFRS that we induce is in general not lexicalized. In order to obtain a lexicalized LCFRS, the notion of recursive partitioning would need to be generalized. We suspect that such a generalization is feasible, in conjunction with a corresponding generalization of the induction techniques presented in this article. This would be technically involved, however, and is therefore left for further research.

Figure 11 sketches three pipelines to induce a LCFRS from a hybrid tree, which differ in the way the recursive partitioning is constructed. The first way (cf. Figure 11a) is to extract a recursive partitioning directly from a hybrid tree (s,≤_s). This extraction is specified in Algorithm 3, which recursively traverses s. For each node in s, the gathered input positions consist of those obtained from the children (lines 6 and 7), plus possibly one input position from the node itself if its label is in Γ (line 5). The case distinction in line 8 and following is needed because every non-leaf in a recursive partitioning must have at least two children.

For a first example, consider the phrase structure in Figure 4a. The extracted recursive partitioning is given at the beginning of Example 16. For a second example, consider the dependency structure and the extracted recursive partitioning in Figure 12.

Note that if Algorithm 3 is applied to an arbitrary hybrid tree, then the accumulated set of input positions is empty for those positions of s that do not dominate elements with labels in Γ. The algorithm requires further (straightforward) refinements before it can be applied on hybrid trees with several roots, that is, if |s| > 1. In much of what follows, we ignore these special cases.

By this procedure, a recursive partitioning extracted from a discontinuous phrase structure or a non-projective dependency structure will have a fanout greater than 1. By then applying Algorithm 2, the resulting LCFRS will have fanout greater than 1. The more discontinuity exists in the input structures, the greater the fanout will be of the recursive partitioning and the resulting LCFRS, which in turn leads to greater parsing complexity. We therefore consider how to reduce the fanout of a recursive partitioning, before it is given as input to Algorithm 2. We aim to keep the structure of a given recursive partitioning largely unchanged, except where it exceeds a certain threshold on the fanout.

Algorithm 4 presents one possible such procedure. It starts at the root, which by definition has fanout 1. Assuming the fanout of the current node does not exceed k, then there are two cases to be distinguished. If the label J of the present node is a singleton, then the node is a leaf, and we can stop (lines 2–3). Otherwise, we search breadth-first through the subtree rooted in the present node to identify a descendant p such that both its label J′ and J ∖ J′ have fanout not exceeding k (line 4). It is easy to see that such a node always exists: Ultimately, breadth-first search will reach the leaves, which are each labeled with a single number. One of these numbers must match either the lowest or the highest element of some maximal subset of consecutive numbers from J, so that the fanout of J cannot increase if that number is removed.

The current node is now given two children π|_p and t. The first is the subtree rooted in the node labeled J′ that we identified earlier, and the second is a copy of the subtree rooted in the present node, but with J′ subtracted from the label of every node (lines 7–14). Nodes labeled with the empty set are removed (line 10), and if a node has the same label as its parent then the two are collapsed (line 13). As the two children each have fanout not exceeding k, we can apply the procedure recursively (line 6).

Algorithm 4 ensures that subsequent induction of an LCFRS (cf. Figure 11b) leads to a binary LCFRS. Note the difference between binarization algorithms such as those from Gómez-Rodríguez and Satta (2009) and Gómez-Rodríguez et al. (2009), which are applied on grammar rules, and our procedure, which is applied before any grammar is obtained. Unlike van Cranenburgh (2012), moreover, our objective is not to obtain a “coarse” grammar for the purpose of coarse-to-fine parsing.

Note that if k is chosen to be 1, then the resulting partitioning is consistent with derivations of a CFG. Even simpler partitionings exist. In particular, the left-branching partitioning has internal node labels that are {1,2, … , m}, each with children labeled {1, … , m − 1} and {m}. These are consistent with the computations of finite automata (FA) in reverse direction; see Figure 11c. Similarly, there is a right-branching recursive partitioning, reflecting finite-state processing in a forward direction. The use of branching partitionings completely detaches string parsing from the structure of the given hybrid tree.

The relation between recursive partitioning and worst-case parsing complexity of the induced LCFRS is summarized in Table 1. We remark that the parsing complexity for unrestricted LCFRSs would improve from $O(n(m+1)⋅k)$ to $O(n2⋅k+2)$ if we could ensure that the LCFRSs are well-nested (Gómez-Rodríguez, Kuhlmann, and Satta 2010), or in other words, if we replace LCFRSs by sMGs. How to refine pipeline (a) to achieve this is left for future investigation.

In the remainder of Section 6 we extend the induction pipelines for LCFRS to induction pipelines for LCFRS/sDCP hybrid grammars, as illustrated in Figure 14. Given a hybrid tree h = (s,≤_s), we choose a recursive partitioning π obtained in one of the ways discussed in the previous section. We apply Algorithm 2 to induce a LCFRS G₁ that generates str(h). Using the same recursive partitioning π, we induce a sDCP G₂ that generates s. For this we use either Algorithm 5 or 6, depending on whether h is a phrase structure or a dependency structure.

In preparation, we define several notions to relate labels of the recursive partitioning with subsets of the positions of s. Let pos_Γ(s) = {p₁, … , p_n} with p_i ≤_sp_{i +1} (i ∈ [n − 1]) and let J be a label of π. We define the set Π(J) ={p_i∣ i ∈ J}, which identifies the nodes of s corresponding to the elements in J. For any subset U ⊆pos(s), we construct the sets ⊤(U) and ⊥(U), which, intuitively, delimit U from the top and the bottom, respectively. Formally, p ∈⊤(U) if and only if p ∈ U and parent(p)∉U. We have p ∈⊥(U) if and only if parent(p) ∈ U and p∉U.

Grammar induction is relatively straightforward for a given hybrid tree h = (s,≤_s) that is a phrase structure, and a given recursive partitioning π. For each node of π, its label is a set of positions of str(h), and these positions must each correspond to a leaf node of s, by virtue of h being a phrase structure. We can apply a closure operation on this set of leaf nodes that includes a node if all of its children are included. Formally, let J be a label of π. Then C(J) is the smallest set U ⊆pos(s) satisfying (i) Π(J) ⊆ U and (ii) if p ∈pos(s), children(p)≠∅, and children(p) ⊆ U, then p ∈ U.

The set C(J) corresponds to a set of (maximal, disjoint) sub-s-terms of s, that is, C(J) can be partitioned such that each part contains the positions that correspond to one sub-s-term. These parts can be arranged according to the lexicographical ordering on positions in s. Formally, for each set U ⊆pos(s), we define s-rk(U) to be the maximal number k such that U can be partitioned into sets U₁, … , U_k where:

• • 

  for every i ∈ [k], p ∈ U_i, and p′ ∈ U if p′ = parent(p) or p′ = right-sibling(p), then p′ ∈ U_i, and

• • 

  for every i,j ∈ [k], p ∈ U_i, and p′ ∈ U_j we have p ≤_ℓp′ implies i ≤ j.

Note that U₁, … , U_k are uniquely defined by this, and we write gspans(U) = 〈U₁, … , U_k〉. The function “gspans” generalizes “spans,” which we defined for recursive partitioning earlier. Similarly, s-rk as used here generalizes the notion of fanout as used for node labels of a recursive partitioning.

These concepts allow us to define a mapping from a node J in π to the set gspans(C(J)) of positions of s. This mapping is such that if J₁, … , J_m are the child nodes of node J in π, then each set $Uj(i)$ in gspans(C(J_i)) (i ∈ [m], j ∈ [s-rk(J_i)]) is contained in some set $Uq(0)$ in gspans(C(J)) (q ∈ [s-rk(J)]). A different way of looking at this is that the image of J can be constructed out of the images of J_i (i ∈ [m]), possibly by adding further nodes linking existing sub-s-terms together.

Such composition of s-terms into larger s-terms is realized by a sDCP without inherited arguments, as constructed by Algorithm 5. It builds one production for each node J₀ with children J₁, … , J_m. J₀ has a synthesized attribute for each set $Uq(0)$ in gspans(C(J₀)). The corresponding s-term s_q is constructed by traversing the positions in $Uq(0)$ (lines 18–31). A sequence of consecutive positions that are also in some $Uj(i)$ in gspans(C(J_i)) is realized by a single variable $xj(i)$ (lines 21–24). Remaining positions become nodes of s_q (lines 26–30).

Figure 16 compares the two derivation trees of the LCFRS/sDCP hybrid grammars of Examples 19 (left) and 20 (right).

We observe the following general property of Algorithm 5: If the recursive partitioning extracted by Algorithm 3 is given as input, then each nonterminal of the induced sDCP G) has s-rank 1 (as in Example 19). This coincides with pipeline (a) of Figure 11, that is, the induction of a LCFRS of arbitrary fanout. However, if the recursive partitioning is transformed by Algorithm 4 or if the left-branching or right-branching recursive partitioning is used (as in Example 20 and as in pipelines (b) and (c) of Figure 11), then the fanout of the induced LCFRS decreases and its derivations are binarized. At the same time, the numbers of synthesized arguments in the induced sDCP may increase. In other words, we witness a trade-off between the degree of mild context-sensitivity of the LCFRS and the numbers of arguments of the sDCP.

We conclude:

Given a recursive partitioning π and a phrase structure h = (s,≤_s), the construction in Section 6.4 relied on a mapping from a node of π labeled J to sets of positions of maximal sub-s-terms in s whose yields together cover exactly the positions in J. We now say π is chunky with respect to h if, for each node J of π, we have s-rk(C(J)) = 1, that is, the nodes in its image under the mapping form a single sub-s-term. If π is chunky with respect to h, then each sDCP nonterminal in the construction from Section 6.4 will have a single synthesized argument. A sDCP with this property is equivalent to a sCFTG in which all nonterminals have rank 0, that is, a regular tree grammar. Therefore:

Let h = (s,≤_s) be a hybrid tree over (Σ,Σ), or in other words, a dependency structure, and let π be a recursive partitioning of str(h). The task is to construct a LCFRS/sDCP hybrid grammar that generates h and parses str(h) according to π.

In the case of phrase structures, we could identify entire subtrees of s whose yields corresponded to subsets of node labels of π. A sequence of consecutive such subtrees (i.e., whose roots were siblings) was then translated to a synthesized argument of a sDCP rule. With dependency structures, however, we need to allow for the scenario where we have a label J of a node in π and two positions p and p′ in s with parent(p′) = p, and p ∈ Π(J) whereas p′∉Π(J). Thus we will need to consider a subtree of s rooted in p, with a “gap” for child p′. This gap will be implemented by introducing an inherited argument in the sDCP, so that the gap can be filled by a structure built elsewhere.

Like the induction algorithms considered before, Algorithm 6 constructs a rule for each node p of π. If p is a leaf of π (line 6), labeled with a singleton J₀ = {i}, we construct the sDCP rule $⦇J0⦈(x1,〈α(〈x1〉)〉)→〈〉$⁠, where α is the i-th element of str(h), assuming the node labeled α is not a leaf in s (line 10). The i-rank of ⦇J₀⦈ is 1. Coupled to the corresponding LCFRS rule, this creates the hybrid rule . If the node labeled α is a leaf of s, we can dispense with the inherited argument of ⦇J₀⦈ in the second component and replace α(〈x₁〉) by α (line 8).

If p is an internal node of π, then we proceed as follows. We determine the set Π(J₀) of positions of s that correspond to the numbers in the label of p. We compute the sets of positions ⊤(Π(J₀)) and ⊥(Π(J₀)) that delimit Π(J₀) from the top and bottom, respectively. We bundle consecutive positions in ⊤(Π(J₀)) and ⊥(Π(J₀)) by applying gspans. For brevity, we write $⊤max(J0)$ instead of gspans(⊤(Π(J₀))) and $⊥max(J0)$ instead of gspans(⊥(Π(J₀))). The nonterminal ⦇J₀⦈ is given a synthesized attribute for each set $Iq(0)$ in $⊤max(J0)$ (line 12), and an inherited attribute for each set $Oj(0)$ in $⊥max(J0)$ (line 13).

Analogously, we determine sets $Oj(i)$ and $Iq(ℓ)$ of positions of s for each child of p labeled J_i (lines 14–15). Each set $Oj(i)$ corresponds to a distinct variable $xj(i)$ in the sDCP rule. Each set $Iq(ℓ)$ corresponds to a s-term $sq(ℓ)$ which combines variables. In particular, each set $Iq(ℓ)$ is the (disjoint) union of some of the sets $Oj(i)$⁠. Conversely, each set $Oj(i)$ is disjoint with all but one of the sets $Iq(ℓ)$⁠. This fact is used in lines 20–27 to construct the s-term $sq(ℓ)$⁠. Having specified the variables and s-terms, the construction of the sDCP rule is completed in line 18. Much as in Section 6.4, this sDCP rule can be coupled to the corresponding LCFRS rule to form a hybrid rule.