Text Attribute Control via Closed-Loop Disentanglement

Changing an attribute of a text without changing the content usually requires first disentangling the text into irrelevant attributes and content representations. After that, in the inference phase, the representation of one attribute is tuned to a different value, expecting that the corresponding attribute of the text can also be changed accordingly. The usual way of disentanglement is to add some constraints on the latent space of an encoder-decoder architecture, including adversarial-based constraints and mutual-information-based constraints. However, previous semi-supervised processes of attribute change are usually not enough to guarantee the success of attribute change and content preservation. In this paper, we propose a novel approach to achieve a robust control of attributes while enhancing content preservation. In this approach, we use a semi-supervised contrastive learning method to encourage the disentanglement of attributes in latent spaces. Differently from previous works, we re-disentangle the reconstructed sentence and compare the re-disentangled latent space with the original latent space, which makes a closed-loop disentanglement process. This also helps content preservation. In addition, the contrastive learning method is also able to replace the role of minimizing mutual information and adversarial training in the disentanglement process, which alleviates the computation cost. We conducted experiments on three text datasets, including the Yelp Service review dataset, the Amazon Product review dataset, and the GoEmotions dataset. The experimental results show the effectiveness of our model.


Introduction
Controlling the attributes of a text is an important application of interpretable natural language mod- Figure 1: Attribute control: a sentence is disentangled into separate attributes.Each dashed circle represents an attribute.After one of the attributes was changed to another value (here, attribute 3 was changed from a circle to a triangle), the corresponding attribute of the reconstructed sentence was changed accordingly.els.The term "control" usually means to take attributes as a handle, and pulling the handle causes corresponding changes in the text.The control process should not change the content of the text.Usually, this is realized by disentangling the text into multiple irrelevant latent spaces for content and multiple attributes (Sha and Lukasiewicz, 2021).
Previous works mainly use two methods for disentangling the attributes: adversarial learning (Chen et al., 2016;John et al., 2019) and mutual information minimization (Moyer et al., 2018;Sha and Lukasiewicz, 2021).For each latent space (corresponding to the content or attributes), the former (John et al., 2019) applies adversarial training to reduce the information that should not be contained in that space.Also, Logeswaran et al. (2018) uses an adversarial method to encourage the generated text to be compatible with the tuned attributes.To alleviate the training cost and the instability of adversarial methods, Moyer et al. (2018) and Sha and Lukasiewicz (2021) proposed to minimize the mutual information between different latent spaces.
When changing attributes, previous methods change the representation of an attribute in the la-tent space, expecting the generated text to satisfy the changed attribute.However, the generated text does not necessarily do so and preserve the content as well as other attributes, if this is not explicitly encouraged in the training process.
In this paper, we propose a novel attribute control model, which uses contrastive learning to make the latent representation of attributes irrelevant to each other, while encouraging the content to be unchanged during attribute control.We still use an autoencoder architecture to disentangle the text into latent spaces.Inspired by closedloop control systems (Di Stefano et al., 1967) and closed-loop data transcription (Dai et al., 2022), we utilize the encoder once more to disentangle the generated text into re-disentangled latent spaces.This enables the disentanglement process to operate in a closed-loop manner, resulting in greater stability.Then, we use contrastive learning to reduce the difference of unchanged attributes between the original and the re-disentangled latent spaces, while enlarging the difference between changed attributes.The contrastive learning method thus provides an alternative way for disentanglement, since it directly encourages content preservation and non-target attribute preservation when changing the targeted attribute.
Our contributions are briefly summarized as follows: • We propose a new approach to disentanglement based on contrastive learning, where we re-disentangle the reconstructed sentence and compare the re-disentangled latent space with the original latent space to make a closedloop control.
• We propose several contrastive learning loss functions to disentangle the text into irrelevant latent spaces as a replacement for adversarial learning or mutual information minimization.
• We conduct extensive experiments on three text datasets (Yelp Service review, Amazon Product review, and GoEmotions dataset) to show the disentanglement effectiveness of our method.

Related Works
Disentanglement for Attribute Control.For a natural text, if we want to change one of its at-tributes while keeping all its other attributes unchanged, a promising way is to disentangle the attributes from the text.Then, changing one attribute is not expected to affect other attributes.
Techniques for disentangling attributes can be divided into two different types: explicit disentanglement (Chen et al., 2016;John et al., 2019;Sha and Lukasiewicz, 2021) and implicit disentanglement (Higgins et al., 2017;Chen et al., 2018).Explicit disentanglement requires the training dataset to contain attribute annotations, which may help to separate the latent space into interpretable components for each attribute.For example, Chen et al. (2016) and John et al. (2019) used adversarial methods to reduce the influence between latent spaces.To overcome the training difficulties and resource-consuming problems of adversarial methods, mutual information minimization methods (Moyer et al., 2018;Sha and Lukasiewicz, 2021) have been proposed to conduct disentanglement in a non-adversarial way.The explicit disentanglement method is easier for attribute control, because it is easy to tell the model which part of the latent space represents which attribute.
Implicit disentanglement does not use the attribute annotations in the training dataset, so for each disentangled component, it is hard to tell exactly which attribute it corresponds to.Implicit disentanglement includes β-VAE (Higgins et al., 2017), β-TCVAE (Chen et al., 2018), and many derivatives (Mathieu et al., 2018;Kumar et al., 2017;Esmaeili et al., 2018;Hoffman and Johnson, 2016;Narayanaswamy et al., 2017;Kim and Mnih, 2018;Shao et al., 2020).The basic principle of implicit disentanglement is to capture the internal relationship between input examples.For example, Chen et al. (2018) break the evidence lower bound (ELBO) into several parts and proposed the Total Correlation, which encourages the different attributes to be statistically independent.Total Correlation is also the cornerstone for MTDNA (Sha and Lukasiewicz, 2021).Esmaeili et al. (2018) further break the ELBO into more segments and discussed the effect of each segment toward implicit disentanglement.However, without the help of annotation, it is difficult for implicit disentanglement to obtain better disentangled latent spaces than explicit disentanglement.

Attribute Control without Disentanglement.
Although disentanglement is a general way to perform attribute control, there are also methods that     Reconstruct < l a t e x i t s h a 1 _ b a s e 6 4 = " M A 4 H u q I j 5 F p / H 2 q m q 8 Q n < l a t e x i t s h a 1 _ b a s e 6 4 = " 2 C F r 9 o l j D e y 1 p q v r 9 S K t 0 E I 5 e 4 g = " > A A A B 7 3 i c b V D L S s N A F J 3 4 r P V V d e l m s A i u S l J q T < l a t e x i t s h a 1 _ b a s e 6 4 = " n T D 8 Z 9 q P / P M M F s 8 y H q i 2 i  2019) use a back translation method to model the attribute control process.Similar methods are also applied by Luo et al. (2019), Artetxe et al. (2018), and Artetxe et al. (2019).Other methods also tried some other task formulations, like probabilistic inference by HMM (He et al., 2019) and paraphrase generation (Krishna et al., 2020).
Contrastive Learning.Contrastive learning has been proposed by Hadsell et al. (2006), and has witnessed a series of developments in recent years.The goal of contrastive learning can be seen as training an encoder for a dictionary look-up task (He et al., 2020).Triplet loss (Chechik et al., 2010;Hoffer and Ailon, 2015;Wang and Gupta, 2015;Sermanet et al., 2018) has originally been proposed to achieve this, which reduces the distance between the example and a positive example and enlarges the distance between the example and a negative example.Noise contrastive estimation (NCE) loss (Gutmann andHyvärinen, 2010, 2012) uses a probabilistic model to discriminate the positive and negative examples.Based on NCE, InfoNCE (Oord et al., 2018;Hjelm et al., 2018;Anand et al., 2019;Bachman et al., 2019;Gordon et al., 2020;Hjelm and Bachman, 2020;Zhuang et al., 2019;Xie et al., 2020;Khosla et al., 2020) has a similar form of classification-based Npair loss (Le-Khac et al., 2020), and it has proved that the minimization of InfoNCE also maximises the lower bound of the mutual information between the input and the representation (Oord et al., 2018).Similar mutual-information-based losses include DIM (Hjelm et al., 2018), PCL (Li et al., 2020), and SwAV (Caron et al., 2020).Also, MoCo (He et al., 2020;Chen et al., 2020cChen et al., , 2021) ) uses a dynamic memory queue for building large and consistent dictionaries for unsupervised learning with InfoNCE loss.SimCLR (Chen et al., 2020a,b) uses a large batch size in an instance discrimination task.
In contrast to the above, instead of on the input examples, we apply contrastive learning in the original and re-disentangled latent spaces to encourage that attributes can be robustly controlled, which thus makes the latent space disentangled.To our knowledge, this is the first work of using contrastive learning in such a way to conduct disentanglement.
The difference between our approach and other disentanglement methods.Our CLD exploits the essence of attribute disentanglement.We now compare it with two previous methods of disentanglement.
Adversarial disentanglement (Chen et al., 2016;John et al., 2019) naturally uses adversarial methods to eliminate the information of other attributes from the representation of one attribute.However, if there are multiple style types, then we need one discriminator for each of the style types, which is a massive cost of resources.Also, adversarial methods can only be taken as constraints on the latent space, since they do not directly encourage the other attributes not being affected by the changed attribute.
Another method is mutual information minimization (Moyer et al., 2018;Sha and Lukasiewicz, 2021), which is more efficient and elegant.However, it still does not directly encourage that the change in the style's latent space can be perfectly reflected in the output sentence.On the other hand, it is based on some strong assumptions like that the content vector should also follow a Gaussian distribution.But in our CLD, the contrastive-learning-based method does not require any of these assumptions.Moreover, CLD directly models the attribute control process in an easier and more natural way, which is more flexible to be generalized to more complex attributes and latent spaces.

Approach
In this section, we introduce the design of our model for contrastive learning disentanglement (CLD).Differently from previous works, our proposed model is very simple, as it only contains the basic encoder-decoder architecture and three contrastive learning loss functions.The architecture of our model is shown in Figure 2.

Basic Architecture for Disentanglement
Like previous disentanglement methods ( Higgins et al., 2017;John et al., 2019;Sha and Lukasiewicz, 2021), we use an autoencoder as our basic architecture.Autoencoders are able to map the input text into a latent space, while encouraging the latent vector to contain the complete information of the input.So, disentanglement is usually achieved by adding constraints to the latent space to split it into irrelevant segments.Then, each segment represents an isolated feature of the input, and once changed the reconstructed text should also be changed correspondingly.
For explicit disentanglement (with annotated attributes for training), we use two kinds of autoencoders: vanilla autoencoders (Hinton and Zemel, 1994) and variational autoencoders (VAEs) (Kingma and Welling, 2014).Given a text dataset S X = {X 1 , . . ., X N }, the loss functions of these two autoencoders are defined as follows: (1) where f (•) and q E (z|X) are the encoders in the vanilla and the variational autoencoders, respectively, p(X|z) is the decoder, and p(z) is a prior distribution (usually, N (0, 1)).The detailed architecture is given in the appendix.Note that J VAE has the name "VAE" because the latent space is calculated using the same method as a variational autoencoder (VAE).Specifically, a VAE uses an encoder to generate a distribution over the latent space, and then samples a vector z from this distribution, and then feeds z to a decoder.Sampling from a distribution results in a continuous latent space (Bowman et al., 2016).

Contrastive Learning for Explicit Disentanglement
Contrastive learning is originally proposed to learn such an embedding space in which similar sample pairs stay close to each other, while dissimilar ones are far apart.So, for disentangled representations, we can re-disentangle the reconstructed input and conduct contrastive learning between the disentangled representations and re-disentangled representations.Intuitively, after one disentangled feature is changed, the corresponding redisentangled feature should also be changed, and the other re-disentangled features should remain unchanged.
Basics for Explicit Disentanglement.In explicit disentanglement, the most typical way is to separate the latent space into two irrelevant latent spaces, one for the style (s) and one for the content (c) (John et al., 2019;Sha and Lukasiewicz, 2021).The style1 vector here is the representation of one of the attributes of the text, including sentiment, tense, and tone for text.In this paper, we define a new symbol to represent the disentanglement: "↠".Then, X ↠ [s, c] represents that the representations of s and c are obtained by directly splitting the latent vector z (in Eq. ( 2)) into s and c.On the other hand, we define "↣" for generating text according to the disentangled attributes.By Eq. ( 2), the distribution of the generated text is calculated by p(X|s, c).So, we can take a sample text from this distribution as X ′ ∼ p(X|s, c), which is denoted by [s, c] ↣ X ′ in this paper.
Then, the disentanglement process and the reconstruction process are written as: where X ′ represents the reconstructed text.
Re-disentanglement for Style Transfer.Following the unified distribution-control (UDC) method in (Sha and Lukasiewicz, 2021), we also predefine a Gaussian distribution N i for the i-th style type value.To give a specific example, there are two values for text sentiment (positive and negative), each corresponds to a Gaussian distribution.
To directly model the style transfer process, we first change the style vector s to the vector of a different style, which is sampled from the unified style distribution defined by the UDC method.In the training phase, this sampling process can be conducted by the reparameterization trick as shown in (Kingma and Welling, 2014).Then, we reconstruct the text and disentangle the text for a second time (namely, re-disentangle) into style vector and content vector.
In detail, assuming that there are V possible style values for s, we sample v style values s1 , . . ., sv that are different from s's original style value.Then, we replace s with s2 and generate the text X.After that, we re-disentangle the generated text X ′ (in Eq. ( 3)) and X, and compare the re-disentangled representation of style and content with the original representation of style and content.
So, the generation and re-disentanglement process can be described as follows: Contrastive Learning.First, under the UDC setting, assume that the predefined trainable distributions for each style value are N 1 , . . ., N V .The disentangled style vector s is expected to be close to the corresponding style value's representation 2 The subscript is omitted, since we do the same operation for each style type value sample.
When we re-disentangle the reconstructed text as X ′ ↣ [s ′ , c ′ ], the representation for style s ′ should be close to the original style value, and far away from all the other style value's representations.The corresponding InfoNCE (Oord et al., 2018) loss is as follows: On the other hand, when the style transfer process is conducted as Eq. ( 5), ideally, the redisentangled style representation s′ should be far from the original style s and close to the transferred style vector s.So, the InfoNCE (Oord et al., 2018) loss function for each of the sampled style values, namely, sk (k = 1, . . ., v), is as follows: Similarly, the contrastive learning constraint for c′ is L c (c ′ ) as follows.
where c (i) is the disentangled content representation of the i-th example in the current batch, M represents the batch size.Finally, if we are using a vanilla autoencoder as the basic architecture, the total loss function of contrastive-learning-based explicit disentanglement is shown in Eq. ( 12).
where λ ori , λ re , λ k , and λ c are hyperparameters.When we are using a VAE as the basic architecture, we only need to replace J AE with J VAE in Eq. ( 12).L c is obtained by summing up the two contrastive learning losses for content preservation as shown in Eq. ( 13).The coefficients of the three items are set to the same, because they are expected to provide an equal effect on the three latent spaces: the original latent space, the redisentangled latent space, and the style-transferred re-disentangled latent space.

Data
Consistent with previous works, we use Yelp Service Reviews4 (Shen et al., 2017), Amazon Product Reviews5 (Fu et al., 2018), and the GoEmotions dataset 6 (Demszky et al., 2020) as the datasets for explicit disentanglement.In the Yelp dataset, there are 444k, 63k, and 126k reviews in the train, valid, and test sets, while the Amazon dataset contains 559k, 2k, and 2k, respectively.Both datasets contain sentiment labels with two possible values ("pos" and "neg").Besides, the tense label is also available in the Amazon dataset, which contains three possible values ("past", "now", and "future").

Evaluation Metrics
We borrow the metric mutual information gap (MIG) in (Chen et al., 2018) for evaluating the disentanglement performance.MIG was originally proposed for implicit disentanglement, which takes each single dimension (a scalar latent variable) of the latent vector as an attribute.In the original design, MIG measures the difference of two mutual information values, one of them is the mutual information between the ground truth factor v k and latent variable z * (z * is the best fit latent variable for v k with the largest mutual information), the other is the mutual information between the ground truth factor v k and latent variable z * * (z * * is the second best fit latent variable for v k ).MIG is defined as follows (Chen et al., 2018): where the subscript "im" stands for implicit disentanglement, and the mutual information I(z; v k ) is defined by: where K is the latent vector's dimension, H(v k ) is the entropy of v k , and χ v k is the support of p(X|v k ).
When computing MIG in explicit disentanglement, we replace the latent variables z * and z * * by s and c: where the subscript "ex" stands for explicit disentanglement.
When evaluating the attribute control performance, we have 4 metrics for the NLP tasks.
• Attribute transfer accuracy (TA): Following previous works (John et al., 2019;Sha and Lukasiewicz, 2021), we use an external sentence classifier (TextCNN (Kim, 2014)) to measure the sentiment accuracy after the attribute change.The external sentence classifiers are trained separately for the Yelp and the Amazon dataset, and achieved an acceptable accuracy on the validation set (Yelp: 97.68%, Amazon: 82.32%).
• Content preservation BLEU (CBLEU-1 & CBLEU-4): This metric is proposed in (Logeswaran et al., 2018), which transfers the attribute-transferred sentence back to the original attribute, and then computes the BLEU score with the original sentence.
• Perplexity (PPL): Perplexity is used for evaluating the fluency of the generated sentences.We use a third-party language model (Kneser and Ney, 1995, KenLM) as the evaluator.Two separate KenLM's are trained and used for evaluation on the two datasets.
• Transfer BLEU (TBLEU): The BLEU score is calculated between the original sentence and the attribute-transferred sentence.We delete the sentiment words before evaluation according to a sentiment word list.7 • Geometric mean (GM): We use the geometric mean of TA, 1/PPL, and TBLEU as an aggregated score, which considers attribute control performance and fluency simultaneously.

Disentanglement Performance
We have visualized the latent space of attributes and contents in Figures 3 and 4. To generate this visualization, we perform dimension reduction on the hidden attribute representations in the latent space.Specifically, we use t-SNE (van der Maaten and Hinton, 2008) to reduce the high-dimensional attribute representations to 2D embeddings that can be plotted.We see that with contrastive learning, both the vanilla and the variational autoencoder have separated different labels of sentiment (or tense) into different latent spaces successfully.
In comparison, the different labels are mixed together in the content's latent space according to Figure 4, which means that the content space does not contain information of the sentiment attribute.Note that we do not use any resource-consuming traditional disentanglement methods like adversarial methods or mutual information minimization, simply re-disentangling the generated sentence and using contrastive learning can lead to such a good disentanglement performance.
For datasets with more granular emotion categories, we also visualize the attribute latent space of the GoEmotions dataset.We again use t-SNE to reduce the high-dimensional attribute representations to 2D embeddings that can be plotted.As shown in Fig. 5, the 2D latent space naturally separates into three distinct clusters corresponding to the semantic-level taxonomy of positive, negative, and neutral emotions.Furthermore, within the positive and negative regions, the space separates into smaller sub-clusters representing each of the six Ekman emotions.This demonstrates that our model has learned a disentangled latent space where proximity aligns with annotated emotion similarities.By visualizing the latent space in 2D, we can better understand the relationships learned between different emotion categories.
Also, the comparison of the MIG value is shown in Figure 6.We reimplemented the previous works of explicit disentanglement, (John et al., 2019) and MTDNA (Sha and Lukasiewicz, 2021), based on their released code, the hyperparameters of the encoder and the decoder are all set to the same.Dif 3: Overall attribute control performance of GoEmotions dataset.For the sentiment taxonomy, the transfer direction is "Negative→Positive", and "Positive→ambiguous".For the Ekman taxonomy, the transfer direction is "joy→fear", "fear→sadness", "sadness→disgust", "disgust→anger", "anger→surprise", "surprise→joy".TA(Sentiment) is the TA metric for sentiment, while TA(Ekman) is for Ekman taxonomy.All the advantages of our results compared to the previous best results are statistically significant, as confirmed by the Wilcoxon signed-rank test.(p < 0.05).The state-of-the-art results made by pretrained language models are underlined.
dom initialization.So, we draw box plots to show the statistical comparison of MIG values in 40 experiments.In both datasets for explicit disentanglement, our method CLD achieves a better MIG value and is more robust (has smaller variance) than the other two methods.
Besides, due to the computational efficiency of contrastive learning losses, our proposed method takes less time for each epoch compared to adversarial-based and mutual-information-based methods.On Yelp, it takes CLD 20.93 min (Vanilla) and 21.56 min (VAE) for one epoch, while (John et al., 2019) requires 46.36 min (Vanilla) and 44.59 min (VAE) for one epoch, MTDNA (Sha and Lukasiewicz, 2021) requires 42.74 min (Vanilla) and 43.62 min (VAE) for one epoch.

Performance of Attribute Control
We compare our method CLD with multiple previous attribute control methods: (Logeswaran et al., 2018) and (Lample et al., 2019) as nondisentanglement methods, and (John et al., 2019) and MTDNA (Sha and Lukasiewicz, 2021) as explicit disentanglement methods.We also compared our approach with the prefix-tuning-based method by Qian et al. (2022) for controlling the attribute of generated text.However, we note that their method was not specifically designed to maintain the text content while modifying attributes.Therefore, we limited our comparison to the TA and PPL metrics.The overall performances of the Yelp and Amazon datasets are listed in Table 2.The overall performance of GoEmotions dataset is listed in Table 3.We can see that our proposed method CLD outperforms all the previous works in the transfer accuracy metric (TA), perplexity, and TBLEU score.Compared with the baseline methods without contrastive learning, our approach shows great advantages over the MTDNA (Sha and Lukasiewicz, 2021) models in the CBLEU metrics.This fact shows that the content of a sentence is much easier to be preserved (the attribute control process is more robust) when we are using contrastive learning to keep the con-tent vector before and after re-disentanglement to be as close as possible.Moreover, when we added back-translation loss as is conducted by Logeswaran et al. (2018) and Lample et al. (2019), our method CLD achieved an even higher score in the CBLEU-1 and CBLEU-4 metric, and this score has outperformed the state-of-the-art CBLEU score.This again proved that backtranslation loss will become more powerful in content preservation when used together with contrastive learning.According to the aggregated performance (GM) listed in Table 2, CLD also outperforms the baseline methods, and CLD(VAE) with back-translation loss achieved state-of-the-art results.We have observed similar results in the tense attribute, which is shown in the column "TA(T)" in Table 2.
We also conducted a comparison between our method and the prompt-tuning-based approach proposed by Qian et al. (2022).However, it is important to note that the prompt-tuning-based method only focuses on controlling the attribute of the generated text, without ensuring content preservation.Therefore, we limited our comparison to the TA and PPL metrics.To evaluate their work, we applied Qian et al. ( 2022)'s method on our datasets and assessed the results based on our metrics.As demonstrated in Table 2, our method still has a clear advantage over the prompt-tuningbased approach, as the latter sacrifices some attribute accuracy in order to achieve controllable text generation.
Our method is very easy to be merged with pretrained language models in encoder-decoder architectures (like T5 (Raffel et al., 2020)).We merged our method with T5 and report the results in Table 2. Due to the large storage of text corpus and common sense knowledge in the pretrained language model, the result achieved a much better level in style transfer accuracy, content preservation, and fluency metrics.

Ablation Test
Effect of Re-disentanglement Process.To prove that the re-disentanglement process is necessary, we remove all the contrastive losses related to the re-disentanglement process.The visualization of the latent spaces for vanilla and VAE are shown on Figure 8.We can see that the latent space became partly mixed up, which shows that the re-disentanglement process is indispensable.
Effect of Contrastive Loss Functions.To study the effect of each contrastive learning loss, we remove the loss functions one by one to check the difference of the evaluation metrics.The results are shown in Table 4.We found that after the content contrastive loss L c is removed, the style transfer accuracy has been improved, which shows that the constraint on the content vector would negatively affect the style information in the generated sentences.Also, the CBLEU-4 and TBLEU scores largely dropped, which shows that L c is very important for content preservation.Then, after Lk is removed, the TA metric dropped about 3 percentage points, while the CBLEU-4 and TBLEU scores did not have any significant change.Since Lk is a constraint for the re-disentangled style vector of the style-transferred sentence, it does not have too much effect on the content of the sentence.A similar phenomenon is observed when we remove the loss L re : the TA metric significantly decreased again, and the BLEU scores slightly decreased.
Besides, we also remove the three contrastive learning losses for the content preservation (L c (c ′ ), L c (c ′ )) to study their effect on the results.The scores are also listed in Table 5.We can see that removing any one of the two losses would cause an increase in the TA score, which means all of the content preservation losses are limitations on the style latent space.Both the CBLEU-4 and TBLEU scores decrease a lot after removing the two content preservation losses.In particular, it seems that L c (c ′ ) has the largest effect on the scores, which is sensible, because a more distinguishable content space is easier for content preservation intuitively.
We also conducted experiments about changing the content's contrastive learning loss L c to meansquare error (MSE) loss to check whether contrastive learning is necessary.In this experiment, we replace L c with the following loss L mse : where ∥ • ∥ 2 represents the 2-norm.The results are also shown in line CLD (Vanilla) (MSE) and CLD (VAE) (MSE) of Table 4.We can see that, the score of CBLEU-4 and TBLEU dropped a lot compared to CLD (Vanilla) and CLD (VAE) after we replaced L c with L mse .The intrinsic difference between L c and L mse is that L mse only encourages c ′ and c from the same case to be close, while  L c also requires the content vectors from different cases to be far away form each other.The latter alleviates the possibility of the content space to collapse.This result proved that the contrastive learning loss is inevitable for content preservation.
Effect of τ .To investigate the effect of the temperature hyperparameter τ , we run the model sev-eral times with different values of τ , and visualize the latent space in Figure 7.According to Figure 7, when τ has a small value, the latent spaces for the different style values tend to be connected in some area.In contrast, the latent spaces get separated when the value of τ increases.The reason is that when the temperature τ is getting large, the distinction between the positive and negative examples in the contrastive losses tends to be underestimated.Hence, the model needs to work harder to make the distinction large, and thus the latent spaces are getting more separated.

Case Study
We sampled some generated text when we are transferring the sentiment attribute from one to another, the results are shown in Table 6.According to the results, the content of text almost remains unchanged, while the target attribute was changed to what we expected.Furthermore, we evaluated more complex emotion attribute transfer cases from the GoEmotions dataset.We transformed the emotions according to the Ekman taxonomy and presented the results produced by CLD using both the vanilla and VAE architectures.These results are tabulated in Table 8.

Human Evaluation
We also conducted a human evaluation for the attribute control results.
We sampled 1,000 examples from each of Yelp and Amazon, and changed their attribute value to the opposite value ("Positive"→"Negative", "Negati- every one is so nice , and the food is amazing ! the servant is rude and the food is terrible .
every one is so tepid , and the food is awful.
an excellent dining experience .the dining feels bad .an awful dining experience .
yesterday i went to this location and the staff was very informative and personable .
yesterday i went to this location and found the staff so rude and angry .
yesterday i went here and the staff was very tepid , not a good choice .
Original (Neg) Vanilla Transferred (Pos) VAE Transferred (Pos) crap service with mediocre food is not a good business model to live by .
good service and the food is delicious .good service with delicious food , good business model to live by .
this is a horrible representation of a deli .
this is a great place to go in this area .this is a good place of a deli .
the staff does a horrible job with my teenagers .
the staff works well with my teenagers .
the staff does a great job working with my teenagers .this machine was exactly a speller .
The machine was a speller, just as its name indicated .
it's so small (of course) and it's really only good for nuts .
it was so small and only good for nuts .it was very small and only useful for nuts in the past , just as it is now .
Original (Past) Vanilla Transferred (Future) VAE Transferred (Future) i did not like the taste of this at all.i will never like this taste .i will never like this taste any more .
i was not impressed, but at least i tried.I will never be impressed .I will not be impressed, but at least I will try.
Original (Future) Vanilla Transferred (Past) VAE Transferred (Past) i'm going to e-mail the company but in the meantime, if you drink this tea, stop.
I emailed the company .I emailed the company, stop drinking this tea .
i'm probably going to end up throwing all of these out .
I threw all this out probably .I probably ended up throwing all of these out.ve"→"Positive").Then, we collected the generated sentences and asked 3 data graders to give a score to the sentences on 3 metrics (transfer accuracy (TA), content preservation (CP), and language quality (LQ)).Among them, TA is a percentage, CP and LQ are scored between 1 ∼ 5.
The detailed questions are listed in the appendix.
We randomly shuffled the sentences to remove the ordering hint.The final result of human evaluation is shown in Table 9.The inter-rater agreements (the Krippendorff's alpha values ( 2004)) of the three metrics are 0.84, 0.89, and 0.92, all of them are acceptable due to Krippendorff's principle (2004).We can see that our proposed method CLD outperforms the baseline in each of the human evaluation metrics.We also listed some gen-  Table 9: Human evaluation results on Yelp and Amazon.

Discussion
Recent work has explored utilizing large language models (LLMs) like ChatGPT and GPT-4 for controllable text generation.For example, Reif et al. (2021) have proposed methods to steer text style transfer in these LLMs by conditioning on discrete attributes or continuous latent representations.Compared to our approach, a key difference is that we train our model end-to-end to disentan-gle latent attributes, while LLMs rely on prompting or fine-tuning approaches applied post-hoc.While promising, utilizing LLMs for attributecontrolled generation remains challenging.The discrete prompting approach can yield brittle or superficial style changes, as the models' understanding of prompted attributes is imperfect and limited to correlation patterns in the pretraining data (Reif et al., 2021;Luo et al., 2023).Latent space steering has shown more coherent style transfer, but current methods rely on complex optimization schemes or assume access to an attribute classifier (John et al., 2019;Sha and Lukasiewicz, 2021).In contrast, our model learns disentangled representations directly from data through closedloop contrastive training.

Limitations
Controlling the attribute's intensity Our model is not designed to control the intensity of an attribute, like generating some neutral sentence instead of "pos" or "neg".If we want to generate a neutral sentence anyway, we just need to take the average vector of the mean value of the "pos" and "neg", and replace the original semantic style vector.Then, the decoder will generate a neutral sentence.However, this method will not always be successful, because there is no guarantee that these latent spaces are smoothly distributed with overlapping regions, and the decoder may not have been required to generate such texts with novel style features during training.To better control the attribute's intensity, it is required to design some special mechanics in a supervised manner.
Difficult attributes Apart from the simple text attributes, there are also some complex attributes like some specific author's style of writing, which are usually intertwined together in the latent space.Discrete categorical style types are hard to design for such kind of complex attributes.Whether disentanglement can be used for controlling complex attributes requires further research.

Conclusion
In this paper, we proposed a novel explicit disentanglement method, called contrastive learning disentanglement (CLD), which uses contrastive learning as the core method.Differently from previous works, we re-disentangle the reconstructed sentences, and conduct contrastive learning between the disentangled vectors and the redisentangled vectors.To encourage the disentanglement of the attributes' latent space, we propose the re-disentangled contrastive loss L re and the transferred re-disentangled contrastive loss Lk .The latter fully imitates the attribute control process.To encourage content preservation, we proposed the content contrastive loss L c , which contains three sub-losses.These sub-losses make the content space more distinguishable and encourage the content keep unchanged during attribute control.Our proposed method is not only much easier in the mathematical derivations, it also outperforms all the compared methods in the evaluation metrics according to our experimental results.
t e x i t s h a 1 _ b a s e 6 4 = " f S 9 s n e P 0 / 6 W 0 o D j 2 h Z + P G u D d e j N d Z 6 5 I x n z l A P 2 C 8 f Q E M p p S c < / l a t e x i t > t e x i t s h a 1 _ b a s e 6 4 = " l 6 6 3 g x 7 o w n 4 9 V 4 m 4 9 m j M X O I f y A 8 f 4 F 8 D i S W w = = < / l a t e x i t > p(X|s, c) < l a t e x i t s h a 1 _ b a s e 6 4 = " z o + t s P a O w S R s S 0 V D S 5 B A 1 y D J m g B D B L w C J 7 B i 3 F v P B m v x t u 8 d c V Y z B y C H z D e v w B m 9 J V g < / l a t e x i t > p(X|s, c) t e x i t s h a 1 _ b a s e 6 4 = " z o + t s P a O w S R s S 0 V D S 5 B A 1 y D J m g B D B L w C J 7 B i 3 F v P B m v x t u 8 d c V Y z B y C H z D e v w B m 9 J V g < / l a t e x i t > e B p z s D p I b y t 5 e I / 3 m d W P m O O 6 E 8 i h X h e L 7 I j 5 m p Q j O J b f a p I F i x s S Y I C 6 p g w c A e e w L N x Y 9 w b L 8 b r r H X J m M 8 c g B 8 w 3 r 4 A 4 6 u U g Q = = < / l a t e x i t > X Encoder < l a t e x i t s h a 1 _ b a s e 6 4 = "

Figure 2 :
Figure 2: Complete architecture of our proposed model CLD.The upper row (a) represents the normal disentanglement process.The lower row (b) imitates the style/attribute transfer process.In both processes, we conduct re-disentanglement and use contrastive learning to encourage the content vector (c) to stay unchanged, while the style vectors (s, s) change to the desired values.
away from other style values' representation.Consistent with previous works(He et al., 2020), we use the dot product to measure the similarity and the InfoNCE(Oord et al., 2018) loss as the contrastive learning loss function as follows: For the re-disentangled content representations c ′ and c′ , it should be close to the original content representation c and far from the content representation disentangled from other examples.The InfoNCE loss for content representation is L c (c ′ ).

Figure 4 :Figure 5 :Figure 6 :
Figure 3: Visualization of the disentangled latent space for the two style types: sentiment and tense.(a), (b), and (c) are created by a vanilla autoencoder, while (d), (e), and (f) are created by a VAE.All results are obtained when τ is set to 100.

Figure 7 :
Figure7: Change of the latent space when the temperature hyperparameter τ is getting larger.We show four different τ values (namely, 0.5, 1.0, 10.0, 100.0) for the two possible architectures.The first row is from the vanilla autoencoder architecture, while the second row is from the VAE architecture.

Table 1 :
Mapping of Emotion Categories to Sentiment and Ekman Taxonomy in GoEmotions Dataset

Table 2 :
ferent experiments for a model would have multiple different MIG values due to different ran-Overall attribute control performance.For the sentiment type, the transfer direction is "Neg→Pos", and "Pos→Neg".For the tense type, the transfer direction is "Past→Now", "Now→Future" and "Future→Past".TA(S) is the TA metric for sentiment, while TA(T) is for tense.All the advantages of our results compared to the previous best results are statistically significant, as confirmed by the Wilcoxon signed-rank test.(p < 0.05).The state-of-the-art results made by pretrained language models are underlined.

Table 4 :
Ablation test results.We select three metrics (TA, CBLEU-4, and TBLEU) in this experiment, because they are closely related to the contrastive losses L re , Lk , and L c .

Table 5 :
Ablation test results w.r.t.different components in L c .

Table 6 :
Examples of sentiment polarity control.

Table 7 :
Examples of tense control.

Table 8 :
Examples of Ekman control in GoEmotions dataset.