Skip to Main Content
Table 16 
Results on the STS task. Top: results on the STS benchmark. Bottom: results on the SemEval-2017 data set. Reported results are Pearson correlation indices, measuring the agreement with human annotated data. In particular, we compare the Pearson scores obtained by the HCTI system using LessLex and GloVe vectors. With regard to the runs with GloVe vectors, we report results with no hand-crafted features (no HF), and without machine translation (no MT).
STS Benchmark (English)
TrackHCTI + LessLexHCTI + GloVe
(t-rank)(c-rank)(no HF)
dev .819 .823 .824 
test .772 .786 .783 
 SemEval 2017  
Track HCTI + LessLex HCTI + GloVe 
 (t-rank) (c-rank) (no MT) 
1. ara-ara .534 .618 .437 
2. ara-eng .310 .476 – 
3. spa-spa .800 .730 .671 
4a. spa-eng .576 .558 – 
4b. spa-eng .143 .009 – 
5. eng-eng .811 .708 .816 
6. tur-eng .400 .433 – 
STS Benchmark (English)
TrackHCTI + LessLexHCTI + GloVe
(t-rank)(c-rank)(no HF)
dev .819 .823 .824 
test .772 .786 .783 
 SemEval 2017  
Track HCTI + LessLex HCTI + GloVe 
 (t-rank) (c-rank) (no MT) 
1. ara-ara .534 .618 .437 
2. ara-eng .310 .476 – 
3. spa-spa .800 .730 .671 
4a. spa-eng .576 .558 – 
4b. spa-eng .143 .009 – 
5. eng-eng .811 .708 .816 
6. tur-eng .400 .433 – 
Close Modal

or Create an Account

Close Modal
Close Modal