Skip to Main Content
Table 3: 

Rounded hyperparameter values.

hyperparameterAAECCDCPAbstRCT
batch size  4  
MLP dropout, dim  0.1, 768  
λc 0.18 0.057 0.035 
λℓ 1.05 0.82 0.58 
λr 0.21 0.15 0.17 
LR 9.1e-5 5.6e-5 8.1e-5 
LR (multi-task 
pre-training) 1.7e-5 2.5e-5 1.9e-5 
epochs (single-task 
training)  20  
epochs (multi-task 
pre-training) 2 4 4 
epochs (target corpus 
fine-tuning) 18 16 16 
auxiliary weight 0.24 0.66 0.76 
Adam beta1, beta2  0.9, 0.998  
hyperparameterAAECCDCPAbstRCT
batch size  4  
MLP dropout, dim  0.1, 768  
λc 0.18 0.057 0.035 
λℓ 1.05 0.82 0.58 
λr 0.21 0.15 0.17 
LR 9.1e-5 5.6e-5 8.1e-5 
LR (multi-task 
pre-training) 1.7e-5 2.5e-5 1.9e-5 
epochs (single-task 
training)  20  
epochs (multi-task 
pre-training) 2 4 4 
epochs (target corpus 
fine-tuning) 18 16 16 
auxiliary weight 0.24 0.66 0.76 
Adam beta1, beta2  0.9, 0.998  
View Large
Close Modal
This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal
This site uses cookies. By continuing to use our website, you are agreeing to our privacy policy.
Accept