Results of intrinsic evaluation via clustering. Score is the homogeneity scores. Dist. is the averaged cosine distance between idioms from different groups. Values are normalized (Norm.) using BART and Definition embeddings are used as lower and upper bound. Higher values are better.
| Method . | Score (Norm.) . | Dist. (Norm.) . |
|---|---|---|
| BART | 0.4546 (0.0) | 0.0379 (0.0) |
| BART-FT | 0.4659 (4.97) | 0.0681 (14.99) |
| ITI | 0.4597 (2.26) | 0.0397 (0.876) |
| ITI+SI | 0.4483 (−2.76) | 0.0514 (6.71) |
| ITI+SF | 0.4357 (−8.31) | 0.0411 (1.64) |
| ITI+SF+Copy | 0.5906 (59.92) | 0.1980 (79.47) |
| ITI+SF+SI | 0.6450 (83.86) | 0.2284 (94.54) |
| Definition | 0.6816 (100.0) | 0.2394 (100.0) |
| Method . | Score (Norm.) . | Dist. (Norm.) . |
|---|---|---|
| BART | 0.4546 (0.0) | 0.0379 (0.0) |
| BART-FT | 0.4659 (4.97) | 0.0681 (14.99) |
| ITI | 0.4597 (2.26) | 0.0397 (0.876) |
| ITI+SI | 0.4483 (−2.76) | 0.0514 (6.71) |
| ITI+SF | 0.4357 (−8.31) | 0.0411 (1.64) |
| ITI+SF+Copy | 0.5906 (59.92) | 0.1980 (79.47) |
| ITI+SF+SI | 0.6450 (83.86) | 0.2284 (94.54) |
| Definition | 0.6816 (100.0) | 0.2394 (100.0) |