MIT Press

Table 3:

Results on the StackOverflow dataset. The DeepRegex method totally fails even when the examples are generously rewritten to conform to the model’s “expected” style. Our GrammarSketch model can do significantly better, with or without manually labeled sketches.

Approach	Top-N Acc
Approach	top-1	top-5	top-25
ḎeepRegex+Filter
Transferred Model	0%	0%	0%
+Curated Language	0%	0%	6.6%

GrammarRegex+Filter	3.2%	9.7%	11.3%

Empty Sketch	4.8%	−	−

DeepSketch
Transferred Model	3.2%	3.2%	4.8%

GrammarSketch
Max Coverage	16.1%	34.4%	45.2%
MLE, manual sketches	34.4%	48.4%	53.2%
MML, no sketch sup	31.1%	54.1%	56.5%

Approach	Top-N Acc
Approach	top-1	top-5	top-25
ḎeepRegex+Filter
Transferred Model	0%	0%	0%
+Curated Language	0%	0%	6.6%

GrammarRegex+Filter	3.2%	9.7%	11.3%

Empty Sketch	4.8%	−	−

DeepSketch
Transferred Model	3.2%	3.2%	4.8%

GrammarSketch
Max Coverage	16.1%	34.4%	45.2%
MLE, manual sketches	34.4%	48.4%	53.2%
MML, no sketch sup	31.1%	54.1%	56.5%

View Large

This Feature Is Available To Subscribers Only