MIT Press

Table 10

Emotion recognition performance (Precision, Recall, F₁) of the models based only on text (⁠ $T \to E$ ⁠) or both text and appraisals (⁠ $TA \to E$ ⁠). Standard deviations are reported in Table 19 in the Appendix. Delta values show the differences between the Macro-F₁ scores of the indexed models.

	$\binom{T \to E}{human}$			$\binom{T \to E}{model}$			$Δ_{(a)}^{(b)}$	$\binom{{TA}^{Gold} \to E}{model}$			$Δ_{(b)}^{(c)}$	$\binom{{TA}^{Pred} \to E}{model}$			$Δ_{(c)}^{(d)}$	$Δ_{(b)}^{(d)}$
Emotion	(a)			(b)				(c)				(d)
	P	R	F₁	P	R	F₁	F₁	P	R	F₁	F₁	P	R	F₁	F₁	F₁
Anger	.50	.66	.57	.57	.52	.53	− .04	.56	.58	.57	+.04	.56	.58	.57	.00	+.04
Boredom	.78	.69	.73	.81	.87	.84	+.11	.83	.84	.83	− .01	.83	.83	.83	.00	− .01
Disgust	.85	.53	.65	.74	.59	.66	+.01	.70	.63	.66	.00	.70	.63	.66	.00	.00
Fear	.66	.83	.73	.65	.66	.65	− .08	.69	.66	.67	+.02	.69	.66	.67	.00	+.02
Guilt	.48	.58	.53	.63	.39	.48	− .05	.64	.54	.58	+.10	.63	.52	.56	− .02	+.08
Joy	.41	.62	.49	.53	.40	.45	− .04	.49	.48	.48	+.03	.49	.46	.47	− .01	+.02
No emo.	.72	.21	.33	.66	.50	.55	+.22	.61	.54	.56	+.01	.62	.53	.56	.00	+.01
Pride	.52	.69	.59	.48	.64	.54	− .05	.51	.61	.55	+.01	.50	.62	.55	.00	+.01
Relief	.56	.74	.64	.65	.63	.63	− .01	.58	.67	.62	− .01	.58	.68	.62	.00	− .01
Sadness	.54	.76	.63	.52	.68	.59	− .04	.61	.69	.65	+.06	.59	.69	.63	− .02	+.04
Shame	.48	.48	.48	.53	.50	.51	+.03	.55	.47	.50	− .01	.55	.45	.49	− .01	− .02
Surprise	.57	.33	.42	.53	.54	.53	+.11	.58	.44	.49	− .04	.58	.44	.50	+.01	− .03
Trust	.95	.36	.52	.73	.75	.74	+.22	.76	.71	.73	− .01	.76	.70	.72	− .01	− .02

Macro avg.	.62	.58	.56	.62	.59	.59	+.03	.62	.60	.61	+.02	.62	.60	.60	− .01	+.01

	$\binom{T \to E}{human}$			$\binom{T \to E}{model}$			$Δ_{(a)}^{(b)}$	$\binom{{TA}^{Gold} \to E}{model}$			$Δ_{(b)}^{(c)}$	$\binom{{TA}^{Pred} \to E}{model}$			$Δ_{(c)}^{(d)}$	$Δ_{(b)}^{(d)}$
Emotion	(a)			(b)				(c)				(d)
	P	R	F₁	P	R	F₁	F₁	P	R	F₁	F₁	P	R	F₁	F₁	F₁
Anger	.50	.66	.57	.57	.52	.53	− .04	.56	.58	.57	+.04	.56	.58	.57	.00	+.04
Boredom	.78	.69	.73	.81	.87	.84	+.11	.83	.84	.83	− .01	.83	.83	.83	.00	− .01
Disgust	.85	.53	.65	.74	.59	.66	+.01	.70	.63	.66	.00	.70	.63	.66	.00	.00
Fear	.66	.83	.73	.65	.66	.65	− .08	.69	.66	.67	+.02	.69	.66	.67	.00	+.02
Guilt	.48	.58	.53	.63	.39	.48	− .05	.64	.54	.58	+.10	.63	.52	.56	− .02	+.08
Joy	.41	.62	.49	.53	.40	.45	− .04	.49	.48	.48	+.03	.49	.46	.47	− .01	+.02
No emo.	.72	.21	.33	.66	.50	.55	+.22	.61	.54	.56	+.01	.62	.53	.56	.00	+.01
Pride	.52	.69	.59	.48	.64	.54	− .05	.51	.61	.55	+.01	.50	.62	.55	.00	+.01
Relief	.56	.74	.64	.65	.63	.63	− .01	.58	.67	.62	− .01	.58	.68	.62	.00	− .01
Sadness	.54	.76	.63	.52	.68	.59	− .04	.61	.69	.65	+.06	.59	.69	.63	− .02	+.04
Shame	.48	.48	.48	.53	.50	.51	+.03	.55	.47	.50	− .01	.55	.45	.49	− .01	− .02
Surprise	.57	.33	.42	.53	.54	.53	+.11	.58	.44	.49	− .04	.58	.44	.50	+.01	− .03
Trust	.95	.36	.52	.73	.75	.74	+.22	.76	.71	.73	− .01	.76	.70	.72	− .01	− .02

Macro avg.	.62	.58	.56	.62	.59	.59	+.03	.62	.60	.61	+.02	.62	.60	.60	− .01	+.01

View Large

This Feature Is Available To Subscribers Only