Results of the conditional perplexity experiment (§4), reported as perplexity per punctuation slot, where an unpunctuated sentence of n words has n + 1 slots. Column “Attn.” is the BiGRU tagger with attention, and “CRF” stands for the BiLSTM-CRF tagger. “Attach” is the ablated version of our model where surface punctuation is directly attached to the nodes. Our full model “+NC” adds NoisyChannel to transduce the attached punctuation into surface punctuation. Dir is the learned direction (§2.2) of our full model’s noisy channel PFST: Left-to-right or Right-to-left. Our models are given oracle parse trees T. The best perplexity is boldfaced, along with all results that are not significantly worse (paired permutation test, p < 0.05).