Results for zero-shot evaluation on Flickr30k, MSCOCO, and CC. For Flickr30k and MSCOCO results we train on the respective other datasets. For CC results we train on Flickr30k. Joint+CoopIn-DomainOSCAR is the in-domain performance for the Joint+Coop approach and here represents the upper-bound.
| Loss . | Image Retrieval . | Text Retrieval . | Image Retrieval . | Text Retrieval . | Image Retrieval . | Text Retrieval . | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | |
| MSCOCO 5k . | Flickr30k 1k . | CC 14k . | ||||||||||||||||
| Joint+CoopIn-DomainOSCAR | 54.7 | 81.3 | 88.9 | 70.8 | 91.0 | 95.2 | 76.4 | 93.6 | 96.2 | 89.4 | 97.7 | 99.0 | – | – | – | – | – | – |
| CEUNITER | – | – | – | – | – | – | 66.2 | 88.4 | 92.9 | 80.7 | 95.7 | 98.0 | – | – | – | – | – | – |
| CEOSCAR | 47.8 | 75.7 | 84.6 | 61.8 | 86.2 | 92.0 | 67.2 | 88.5 | 92.7 | 81.0 | 95.5 | 97.8 | – | – | – | – | – | – |
| CLIP | 30.4 | 56.1 | 66.9 | 50.1 | 74.8 | 83.6 | 61.1 | 85.9 | 91.8 | 81.9 | 95.0 | 97.5 | 30.8 | 52.7 | 61.3 | 32.1 | 53.9 | 63.0 |
| BEOSCAR | 37.6 | 64.4 | 75.0 | 52.0 | 78.1 | 86.3 | 63.3 | 86.4 | 91.6 | 78.2 | 94.0 | 97.3 | 13.8 | 29.4 | 37.9 | 14.4 | 29.6 | 37.6 |
| Sep+CoopOSCAR | 47.6 | 73.9 | 81.2 | 62.8 | 83.8 | 88.7 | 67.6 | 89.0 | 93.1 | 82.4 | 96.3 | 98.2 | 16.8 | 34.3 | 41.9 | 17.0 | 33.5 | 41.5 |
| Joint+CoopOSCAR | 47.6 | 74.5 | 82.6 | 63.9 | 85.7 | 91.0 | 70.0 | 90.2 | 94.1 | 83.7 | 96.8 | 97.9 | 16.7 | 34.7 | 43.6 | 17.5 | 34.6 | 43.5 |
| Loss . | Image Retrieval . | Text Retrieval . | Image Retrieval . | Text Retrieval . | Image Retrieval . | Text Retrieval . | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | R@1 . | R@5 . | R@10 . | |
| MSCOCO 5k . | Flickr30k 1k . | CC 14k . | ||||||||||||||||
| Joint+CoopIn-DomainOSCAR | 54.7 | 81.3 | 88.9 | 70.8 | 91.0 | 95.2 | 76.4 | 93.6 | 96.2 | 89.4 | 97.7 | 99.0 | – | – | – | – | – | – |
| CEUNITER | – | – | – | – | – | – | 66.2 | 88.4 | 92.9 | 80.7 | 95.7 | 98.0 | – | – | – | – | – | – |
| CEOSCAR | 47.8 | 75.7 | 84.6 | 61.8 | 86.2 | 92.0 | 67.2 | 88.5 | 92.7 | 81.0 | 95.5 | 97.8 | – | – | – | – | – | – |
| CLIP | 30.4 | 56.1 | 66.9 | 50.1 | 74.8 | 83.6 | 61.1 | 85.9 | 91.8 | 81.9 | 95.0 | 97.5 | 30.8 | 52.7 | 61.3 | 32.1 | 53.9 | 63.0 |
| BEOSCAR | 37.6 | 64.4 | 75.0 | 52.0 | 78.1 | 86.3 | 63.3 | 86.4 | 91.6 | 78.2 | 94.0 | 97.3 | 13.8 | 29.4 | 37.9 | 14.4 | 29.6 | 37.6 |
| Sep+CoopOSCAR | 47.6 | 73.9 | 81.2 | 62.8 | 83.8 | 88.7 | 67.6 | 89.0 | 93.1 | 82.4 | 96.3 | 98.2 | 16.8 | 34.3 | 41.9 | 17.0 | 33.5 | 41.5 |
| Joint+CoopOSCAR | 47.6 | 74.5 | 82.6 | 63.9 | 85.7 | 91.0 | 70.0 | 90.2 | 94.1 | 83.7 | 96.8 | 97.9 | 16.7 | 34.7 | 43.6 | 17.5 | 34.6 | 43.5 |