MIT Press

Table 6:

Accuracies (%) on the FewRel dataset. N-K indicates the N-way K-shot setting. MTB uses the LARGE size and all the other models use the BASE size. ^† indicates oracle models which may have seen facts in the FewRel 1.0 test set during pre-training.

Model	FewRel 1.0				FewRel 2.0
Model	5-1	5-5	10-1	10-5	5-1	5-5	10-1	10-5
MTB (BERT_LARGE)^†	93.86	97.06	89.20	94.27	−	−	−	−

Proto (BERT)	80.68	89.60	71.48	82.89	40.12	51.50	26.45	36.93
Proto (MTB)	81.39	91.05	71.55	83.47	52.13	76.67	48.28	69.75
Proto (ERNIE_BERT)^†	89.43	94.66	84.23	90.83	49.40	65.55	34.99	49.68
Proto (KnowBert_BERT)^†	86.64	93.22	79.52	88.35	64.40	79.87	51.66	69.71
Proto (RoBERTa)	85.78	95.78	77.65	92.26	64.65	82.76	50.80	71.84
Proto (Our RoBERTa)	84.42	95.30	76.43	91.74	61.98	83.11	48.56	72.19
Proto (ERNIE_RoBERTa)^†	87.76	95.62	80.14	91.47	54.43	80.48	37.97	66.26
Proto (KnowBert_RoBERTa)^†	82.39	93.62	76.21	88.57	55.68	71.82	41.90	58.55
Proto (KEPLER-Wiki)	88.30	95.94	81.10	92.67	66.41	84.02	51.85	73.60

PAIR (BERT)	88.32	93.22	80.63	87.02	67.41	78.57	54.89	66.85
PAIR (MTB)	83.01	87.64	73.42	78.47	46.18	70.50	36.92	55.17
PAIR (ERNIE_BERT)^†	92.53	94.27	87.08	89.13	56.18	68.97	43.40	54.35
PAIR (KnowBert_BERT)^†	88.48	92.75	82.57	86.18	66.05	77.88	50.86	67.19
PAIR (RoBERTa)	89.32	93.70	82.49	88.43	66.78	81.84	53.99	70.85
PAIR (Our RoBERTa)	89.26	93.71	83.32	89.02	63.22	77.66	49.28	65.97
PAIR (ERNIE_RoBERTa)^†	87.46	94.11	81.68	87.83	59.29	72.91	48.51	60.26
PAIR (KnowBert_RoBERTa)^†	85.05	91.34	76.04	85.25	50.68	66.04	37.10	51.13
PAIR (KEPLER-Wiki)	90.31	94.28	85.48	90.51	67.23	82.09	54.32	71.01

Model	FewRel 1.0				FewRel 2.0
Model	5-1	5-5	10-1	10-5	5-1	5-5	10-1	10-5
MTB (BERT_LARGE)^†	93.86	97.06	89.20	94.27	−	−	−	−

Proto (BERT)	80.68	89.60	71.48	82.89	40.12	51.50	26.45	36.93
Proto (MTB)	81.39	91.05	71.55	83.47	52.13	76.67	48.28	69.75
Proto (ERNIE_BERT)^†	89.43	94.66	84.23	90.83	49.40	65.55	34.99	49.68
Proto (KnowBert_BERT)^†	86.64	93.22	79.52	88.35	64.40	79.87	51.66	69.71
Proto (RoBERTa)	85.78	95.78	77.65	92.26	64.65	82.76	50.80	71.84
Proto (Our RoBERTa)	84.42	95.30	76.43	91.74	61.98	83.11	48.56	72.19
Proto (ERNIE_RoBERTa)^†	87.76	95.62	80.14	91.47	54.43	80.48	37.97	66.26
Proto (KnowBert_RoBERTa)^†	82.39	93.62	76.21	88.57	55.68	71.82	41.90	58.55
Proto (KEPLER-Wiki)	88.30	95.94	81.10	92.67	66.41	84.02	51.85	73.60

PAIR (BERT)	88.32	93.22	80.63	87.02	67.41	78.57	54.89	66.85
PAIR (MTB)	83.01	87.64	73.42	78.47	46.18	70.50	36.92	55.17
PAIR (ERNIE_BERT)^†	92.53	94.27	87.08	89.13	56.18	68.97	43.40	54.35
PAIR (KnowBert_BERT)^†	88.48	92.75	82.57	86.18	66.05	77.88	50.86	67.19
PAIR (RoBERTa)	89.32	93.70	82.49	88.43	66.78	81.84	53.99	70.85
PAIR (Our RoBERTa)	89.26	93.71	83.32	89.02	63.22	77.66	49.28	65.97
PAIR (ERNIE_RoBERTa)^†	87.46	94.11	81.68	87.83	59.29	72.91	48.51	60.26
PAIR (KnowBert_RoBERTa)^†	85.05	91.34	76.04	85.25	50.68	66.04	37.10	51.13
PAIR (KEPLER-Wiki)	90.31	94.28	85.48	90.51	67.23	82.09	54.32	71.01

View Large

This Feature Is Available To Subscribers Only