Skip to Main Content
Table 6: 
Perplexity (evaluated on the train split to avoid evaluating generalization) of a trigram language model trained (with add-0.001 smoothing) on different versions of rephrased training sentences. “Punctuation” only evaluates perplexity on the trigrams that have punctuation. “All” evaluates on all the trigrams. “Base” permutes all surface dependents including punctuation (Wang and Eisner, 2016). “Full” is our full approach: recover underlying punctuation, permute remaining dependents, regenerate surface punctuation. “Half” is like “Full” but it permutes the non-punctuation tokens identically to “Base.” The permutation model is trained on surface trees or recovered underlying trees T′, respectively. In each 3-way comparison, we boldface the best result (always significant under a paired permutation test over per-sentence log-probabilities, p < 0.05).
PunctuationAll
BaseHalfFullBaseHalfFull
Arabic 156.0 231.3 186.1 540.8 590.3 553.4 
Chinese 165.2 110.0 61.4 205.0 174.4 78.7 
English 98.4 74.5 51.0 140.9 131.4 75.4 
Hindi 10.8 11.0 9.7 118.4 118.8 91.8 
Spanish 266.2 259.2 194.5 346.3 343.4 239.3 
PunctuationAll
BaseHalfFullBaseHalfFull
Arabic 156.0 231.3 186.1 540.8 590.3 553.4 
Chinese 165.2 110.0 61.4 205.0 174.4 78.7 
English 98.4 74.5 51.0 140.9 131.4 75.4 
Hindi 10.8 11.0 9.7 118.4 118.8 91.8 
Spanish 266.2 259.2 194.5 346.3 343.4 239.3 
Close Modal

or Create an Account

Close Modal
Close Modal