Skip to Main Content
Table 3: 
Results on the StackOverflow dataset. The DeepRegex method totally fails even when the examples are generously rewritten to conform to the model’s “expected” style. Our GrammarSketch model can do significantly better, with or without manually labeled sketches.
ApproachTop-N Acc
top-1top-5top-25
eepRegex+Filter 
 Transferred Model 0% 0% 0% 
 +Curated Language 0% 0% 6.6% 
 
GrammarRegex+Filter 3.2% 9.7% 11.3% 
 
Empty Sketch 4.8% − − 
 
DeepSketch 
 Transferred Model 3.2% 3.2% 4.8% 
 
GrammarSketch 
 Max Coverage 16.1% 34.4% 45.2% 
 MLE, manual sketches 34.4% 48.4% 53.2% 
 MML, no sketch sup 31.1% 54.1% 56.5% 
ApproachTop-N Acc
top-1top-5top-25
eepRegex+Filter 
 Transferred Model 0% 0% 0% 
 +Curated Language 0% 0% 6.6% 
 
GrammarRegex+Filter 3.2% 9.7% 11.3% 
 
Empty Sketch 4.8% − − 
 
DeepSketch 
 Transferred Model 3.2% 3.2% 4.8% 
 
GrammarSketch 
 Max Coverage 16.1% 34.4% 45.2% 
 MLE, manual sketches 34.4% 48.4% 53.2% 
 MML, no sketch sup 31.1% 54.1% 56.5% 
Close Modal

or Create an Account

Close Modal
Close Modal