Text Attribute Control via Closed-Loop Disentanglement

Controlling the attributes of a text is an important application of interpretable natural language models. The term “control” usually means to take attributes as a handle, and pulling the handle causes corresponding changes in the text. The control process should not change the content of the text as is shown in Figure 1. Usually, this is realized by disentangling the text into multiple irrelevant latent spaces for content and multiple attributes (Sha and Lukasiewicz, 2021).

Previous work mainly use two methods for disentangling the attributes: adversarial learning (Chen et al., 2016; John et al., 2019) and mutual information minimization (Moyer et al., 2018; Sha and Lukasiewicz, 2021). For each latent space (corresponding to the content or attributes), the former (John et al., 2019) applies adversarial training to reduce the information that should not be contained in that space. Also, Logeswaran et al. (2018) uses an adversarial method to encourage the generated text to be compatible with the tuned attributes. To alleviate the training cost and the instability of adversarial methods, Moyer et al. (2018) and Sha and Lukasiewicz (2021) proposed to minimize the mutual information between different latent spaces.

When changing attributes, previous methods change the representation of an attribute in the latent space, expecting the generated text to satisfy the changed attribute. However, the generated text does not necessarily do so and preserve the content as well as other attributes, if this is not explicitly encouraged in the training process.

In this paper, we propose a novel attribute control model, which uses contrastive learning to make the latent representation of attributes irrelevant to each other, while encouraging the content to be unchanged during attribute control. We still use an autoencoder architecture to disentangle the text into latent spaces. Inspired by closed-loop control systems (Di Stefano et al., 1967) and closed-loop data transcription (Dai et al., 2022), we utilize the encoder once more to disentangle the generated text into re-disentangled latent spaces. This enables the disentanglement process to operate in a closed-loop manner, resulting in greater stability. Then, we use contrastive learning to reduce the difference of unchanged attributes between the original and the re-disentangled latent spaces, while enlarging the difference between changed attributes. The contrastive learning method thus provides an alternative way for disentanglement, since it directly encourages content preservation and non-target attribute preservation when changing the targeted attribute.

Our contributions are briefly summarized as follows:

• We propose a new approach to disentanglement based on contrastive learning, where we re-disentangle the reconstructed sentence and compare the re-disentangled latent space with the original latent space to make a closed-loop control.

• We propose several contrastive learning loss functions to disentangle the text into irrelevant latent spaces as a replacement for adversarial learning or mutual information minimization.

• We conduct extensive experiments on three text datasets (Yelp Service review, Amazon Product review, and GoEmotions dataset) to show the disentanglement effectiveness of our method.

For a natural text, if we want to change one of its attributes while keeping all its other attributes unchanged, a promising way is to disentangle the attributes from the text. Then, changing one attribute is not expected to affect other attributes.

Techniques for disentangling attributes can be divided into two different types: explicit disentanglement (Chen et al., 2016; John et al., 2019; Sha and Lukasiewicz, 2021) and implicit disentanglement (Higgins et al., 2017; Chen et al., 2018). Explicit disentanglement requires the training dataset to contain attribute annotations, which may help to separate the latent space into interpretable components for each attribute. For example, Chen et al. (2016) and John et al. (2019) used adversarial methods to reduce the influence between latent spaces. To overcome the training difficulties and resource-consuming problems of adversarial methods, mutual information minimization methods (Moyer et al., 2018; Sha and Lukasiewicz, 2021) have been proposed to conduct disentanglement in a non-adversarial way. The explicit disentanglement method is easier for attribute control, because it is easy to tell the model which part of the latent space represents which attribute.

Implicit disentanglement does not use the attribute annotations in the training dataset, so for each disentangled component, it is hard to tell exactly which attribute it corresponds to. Implicit disentanglement includes β-VAE (Higgins et al., 2017), β-TCVAE (Chen et al., 2018), and many derivatives (Mathieu et al., 2018; Kumar et al., 2017; Esmaeili et al., 2018; Hoffman and Johnson, 2016; Narayanaswamy et al., 2017; Kim and Mnih, 2018; Shao et al., 2020). The basic principle of implicit disentanglement is to capture the internal relationship between input examples. For example, Chen et al. (2018) break the evidence lower bound (ELBO) into several parts and proposed the Total Correlation, which encourages the different attributes to be statistically independent. Total Correlation is also the cornerstone for MTDNA (Sha and Lukasiewicz, 2021). Esmaeili et al. (2018) further break the ELBO into more segments and discussed the effect of each segment toward implicit disentanglement. However, without the help of annotation, it is difficult for implicit disentanglement to obtain better disentangled latent spaces than explicit disentanglement.

Although disentanglement is a general way to perform attribute control, there are also methods that control attributes without disentanglement. For example, Logeswaran et al. (2018) use adversarial training to judge whether the generated sentence is compatible with the target attribute label. Lample et al. (2019) use a back translation method to model the attribute control process. Similar methods are also applied by Luo et al. (2019), Artetxe et al. (2018), and Artetxe et al. (2019). Other methods also tried some other task formulations, like probabilistic inference by HMM (He et al., 2019) and paraphrase generation (Krishna et al., 2020).

Contrastive learning has been proposed by Hadsell et al. (2006), and has witnessed a series of developments in recent years. The goal of contrastive learning can be seen as training an encoder for a dictionary look-up task (He et al., 2020). Triplet loss (Chechik et al., 2010; Hoffer and Ailon, 2015; Wang and Gupta, 2015; Sermanet et al., 2018) has originally been proposed to achieve this, which reduces the distance between the example and a positive example and enlarges the distance between the example and a negative example. Noise contrastive estimation (NCE) loss (Gutmann and Hyvärinen, 2010, 2012) uses a probabilistic model to discriminate the positive and negative examples. Based on NCE, InfoNCE (Oord et al., 2018; Hjelm et al., 2018; Anand et al., 2019; Bachman et al., 2019; Gordon et al., 2020; Hjelm and Bachman, 2020; Zhuang et al., 2019; Xie et al., 2020; Khosla et al., 2020) has a similar form of classification-based N-pair loss (Le-Khac et al., 2020), and it has proved that the minimization of InfoNCE also maximizes the lower bound of the mutual information between the input and the representation (Oord et al., 2018). Similar mutual-information-based losses include DIM (Hjelm et al., 2018), PCL (Li et al., 2020), and SwAV (Caron et al., 2020). Also, MoCo (He et al., 2020; Chen et al., 2020c, 2021) uses a dynamic memory queue for building large and consistent dictionaries for unsupervised learning with InfoNCE loss. SimCLR (Chen et al., 2020a, b) uses a large batch size in an instance discrimination task.

In contrast to the above, instead of on the input examples, we apply contrastive learning in the original and re-disentangled latent spaces to encourage that attributes can be robustly controlled, which thus makes the latent space disentangled. To our knowledge, this is the first work of using contrastive learning in such a way to conduct disentanglement.

Our CLD exploits the essence of attribute disentanglement. We now compare it with two previous methods of disentanglement.

Adversarial disentanglement (Chen et al., 2016; John et al., 2019) naturally uses adversarial methods to eliminate the information of other attributes from the representation of one attribute. However, if there are multiple style types, then we need one discriminator for each of the style types, which is a massive cost of resources. Also, adversarial methods can only be taken as constraints on the latent space, since they do not directly encourage the other attributes not being affected by the changed attribute.

Another method is mutual information minimization (Moyer et al., 2018; Sha and Lukasiewicz, 2021), which is more efficient and elegant. However, it still does not directly encourage that the change in the style’s latent space can be perfectly reflected in the output sentence. On the other hand, it is based on some strong assumptions like that the content vector should also follow a Gaussian distribution. But in our CLD, the contrastive-learning-based method does not require any of these assumptions. Moreover, CLD directly models the attribute control process in an easier and more natural way, which is more flexible to be generalized to more complex attributes and latent spaces.

In this section, we introduce the design of our model for contrastive learning disentanglement (CLD). Differently from previous work, our proposed model is very simple, as it only contains the basic encoder-decoder architecture and three contrastive learning loss functions. The architecture of our model is shown in Figure 2.

Like previous disentanglement methods (Higgins et al., 2017; John et al., 2019; Sha and Lukasiewicz, 2021), we use an autoencoder as our basic architecture. Autoencoders are able to map the input text into a latent space, while encouraging the latent vector to contain the complete information of the input. So, disentanglement is usually achieved by adding constraints to the latent space to split it into irrelevant segments. Then, each segment represents an isolated feature of the input, and once changed the reconstructed text should also be changed correspondingly.

Note that J_VAE has the name “VAE” because the latent space is calculated using the same method as a variational autoencoder (VAE). Specifically, a VAE uses an encoder to generate a distribution over the latent space, and then samples a vector z from this distribution, and then feeds z to a decoder. Sampling from a distribution results in a continuous latent space (Bowman et al., 2016).

Contrastive learning is originally proposed to learn such an embedding space in which similar sample pairs stay close to each other, while dissimilar ones are far apart. So, for disentangled representations, we can re-disentangle the reconstructed input and conduct contrastive learning between the disentangled representations and re-disentangled representations. Intuitively, after one disentangled feature is changed, the corresponding re-disentangled feature should also be changed, and the other re-disentangled features should remain unchanged.

Following the unified distribution-control (UDC) method in Sha and Lukasiewicz (2021), we also predefine a Gaussian distribution $Ni$ for the i-th style type value. To give a specific example, there are two values for text sentiment (positive and negative), each corresponds to a Gaussian distribution.

To directly model the style transfer process, we first change the style vector s to the vector of a different style, which is sampled from the unified style distribution defined by the UDC method. In the training phase, this sampling process can be conducted by the reparameterization trick as shown in Kingma and Welling (2014). Then, we reconstruct the text and disentangle the text for a second time (namely, re-disentangle) into style vector and content vector.

In detail, assuming that there are V possible style values for s, we sample v style values $s~1,…,s~v$ that are different from s’s original style value. Then, we replace s with $s~$² and generate the text $X~$⁠. After that, we re-disentangle the generated text X′ (in Eq. (3)) and $X~$⁠, and compare the re-disentangled representation of style and content with the original representation of style and content.

Consistent with previous work, we use Yelp Service Reviews⁴ (Shen et al., 2017), Amazon Product Reviews⁵ (Fu et al., 2018), and the GoEmotions dataset⁶ (Demszky et al., 2020) as the datasets for explicit disentanglement. In the Yelp dataset, there are 444k, 63k, and 126k reviews in the train, valid, and test sets, while the Amazon dataset contains 559k, 2k, and 2k, respectively. Both datasets contain sentiment labels with two possible values (“pos” and “neg”). Additionally, the tense label is also available in the Amazon dataset, which contains three possible values (“past”, “now”, and “future”).

GoEmotions dataset contains 58,009 examples with the train, test, and validation sets split as 43,410, 5,427, and 5,426 examples, respectively. GoEmotions annotations categorize the examples into 27 distinct emotion labels. These emotion labels are further grouped in two ways: First, by sentiment into positive, negative, and ambiguous classes. Second, by Ekman’s emotion taxonomy which divides the emotions into 6 broad categories: anger (including anger, annoyance, disapproval), disgust, fear (including fear and nervousness), joy (covering all positive emotions), sadness (including sadness, disappointment, embarrassment, grief, and remorse), and surprise (spanning all ambiguous emotions). The mapping relations are shown in Table 1.

When evaluating the attribute control performance, we have 4 metrics for the NLP tasks.

• Attribute transfer accuracy (TA): Following previous works (John et al., 2019; Sha and Lukasiewicz, 2021), we use an external sentence classifier (TextCNN [Kim, 2014]) to measure the sentiment accuracy after the attribute change. The external sentence classifiers are trained separately for the Yelp and the Amazon dataset, and achieved an acceptable accuracy on the validation set (Yelp: 97.68%, Amazon: 82.32%).

• Content preservation BLEU (CBLEU-1 & CBLEU-4): This metric is proposed in Logeswaran et al. (2018), which transfers the attribute-transferred sentence back to the original attribute, and then computes the BLEU score with the original sentence.

• Perplexity (PPL): Perplexity is used for evaluating the fluency of the generated sentences. We use a third-party language model (Kneser and Ney, 1995, KenLM) as the evaluator. Two separate KenLMs are trained and used for evaluation on the two datasets.

• Transfer BLEU (TBLEU): The BLEU score is calculated between the original sentence and the attribute-transferred sentence. We delete the sentiment words before evaluation according to a sentiment word list.⁷

• Geometric mean (GM): We use the geometric mean of TA, 1/PPL, and TBLEU as an aggregated score, which considers attribute control performance and fluency simultaneously.

We have visualized the latent space of attributes and contents in Figures 3 and 4. To generate this visualization, we perform dimension reduction on the hidden attribute representations in the latent space. Specifically, we use t-SNE (van der Maaten and Hinton, 2008) to reduce the high-dimensional attribute representations to 2D embeddings that can be plotted. We see that with contrastive learning, both the vanilla and the variational autoencoder have separated different labels of sentiment (or tense) into different latent spaces successfully. In comparison, the different labels are mixed together in the content’s latent space according to Figure 4, which means that the content space does not contain information of the sentiment attribute. Note that we do not use any resource-consuming traditional disentanglement methods like adversarial methods or mutual information minimization, simply re-disentangling the generated sentence and using contrastive learning can lead to such a good disentanglement performance.

For datasets with more granular emotion categories, we also visualize the attribute latent space of the GoEmotions dataset. We again use t-SNE to reduce the high-dimensional attribute representations to 2D embeddings that can be plotted. As shown in Figure 5, the 2D latent space naturally separates into three distinct clusters corresponding to the semantic-level taxonomy of positive, negative, and neutral emotions. Furthermore, within the positive and negative regions, the space separates into smaller sub-clusters representing each of the six Ekman emotions. This demonstrates that our model has learned a disentangled latent space where proximity aligns with annotated emotion similarities. By visualizing the latent space in 2D, we can better understand the relationships learned between different emotion categories.

Also, the comparison of the MIG value is shown in Figure 6. We reimplemented the previous works of explicit disentanglement (John et al., 2019) and MTDNA (Sha and Lukasiewicz, 2021), based on their released code, the hyperparameters of the encoder and the decoder are all set to the same. Different experiments for a model would have multiple different MIG values due to different random initialization. So, we draw box plots to show the statistical comparison of MIG values in 40 experiments. In both datasets for explicit disentanglement, our method CLD achieves a better MIG value and is more robust (has smaller variance) than the other two methods.

Additionally, due to the computational efficiency of contrastive learning losses, our proposed method takes less time for each epoch compared to adversarial-based and mutual-information-based methods. On Yelp, it takes CLD 20.93 min (Vanilla) and 21.56 min (VAE) for one epoch, while John et al. (2019) requires 46.36 min (Vanilla) and 44.59 min (VAE) for one epoch, and MTDNA (Sha and Lukasiewicz, 2021) requires 42.74 min (Vanilla) and 43.62 min (VAE) for one epoch.

We compare our method CLD with multiple previous attribute control methods: Logeswaran et al. (2018) and Lample et al. (2019) as non-disentanglement methods, and John et al. (2019) and MTDNA (Sha and Lukasiewicz, 2021) as explicit disentanglement methods. We also compared our approach with the prefix-tuning-based method by Qian et al. (2022) for controlling the attribute of generated text. However, we note that their method was not specifically designed to maintain the text content while modifying attributes. Therefore, we limited our comparison to the TA and PPL metrics.

The overall performances of the Yelp and Amazon datasets are listed in Table 2. The overall performance of GoEmotions dataset is listed in Table 3. We can see that our proposed method CLD outperforms all the previous works in the TA metric, perplexity, and TBLEU score. Compared with the baseline methods without contrastive learning, our approach shows great advantages over the MTDNA (Sha and Lukasiewicz, 2021) models in the CBLEU metrics. This fact shows that the content of a sentence is much easier to be preserved (the attribute control process is more robust) when we are using contrastive learning to keep the content vector before and after re-disentanglement to be as close as possible. Moreover, when we added back-translation loss as is conducted by Logeswaran et al. (2018) and Lample et al. (2019), our method CLD achieved an even higher score in the CBLEU-1 and CBLEU-4 metric, and this score has outperformed the state-of-the-art CBLEU score. This again proved that back-translation loss will become more powerful in content preservation when used together with contrastive learning. According to the aggregated performance (GM) listed in Table 2, CLD also outperforms the baseline methods, and CLD(VAE) with back-translation loss achieved state-of-the-art results. We have observed similar results in the tense attribute, which is shown in the column “TA(T)” in Table 2.

We also conducted a comparison between our method and the prompt-tuning-based approach proposed by Qian et al. (2022). However, it is important to note that the prompt-tuning-based method only focuses on controlling the attribute of the generated text, without ensuring content preservation. Therefore, we limited our comparison to the TA and PPL metrics. To evaluate their work, we applied Qian et al.’s (2022) method on our datasets and assessed the results based on our metrics. As demonstrated in Table 2, our method still has a clear advantage over the prompt-tuning-based approach, as the latter sacrifices some attribute accuracy in order to achieve controllable text generation.

Our method is very easy to be merged with pretrained language models in encoder-decoder architectures (like T5 [Raffel et al., 2020]). We merged our method with T5 and report the results in Table 2. Due to the large storage of text corpus and common sense knowledge in the pretrained language model, the result achieved a much better level in style transfer accuracy, content preservation, and fluency metrics.

To prove that the re-disentanglement process is necessary, we remove all the contrastive losses related to the re-disentanglement process. The visualization of the latent spaces for vanilla and VAE are shown in Figure 8. We can see that the latent space became partly mixed up, which shows that the re-disentanglement process is indispensable.

To study the effect of each contrastive learning loss, we remove the loss functions one by one to check the difference of the evaluation metrics. The results are shown in Table 4. We found that after the content contrastive loss L_c is removed, the style transfer accuracy is improved, which shows that the constraint on the content vector would negatively affect the style information in the generated sentences. Also, the CBLEU-4 and TBLEU scores largely dropped, which shows that L_c is very important for content preservation. Then, after $L~k$ is removed, the TA metric dropped about 3 percentage points, while the CBLEU-4 and TBLEU scores did not have any significant change. Since $L~k$ is a constraint for the re-disentangled style vector of the style-transferred sentence, it does not have too much effect on the content of the sentence. A similar phenomenon is observed when we remove the loss L_re: The TA metric significantly decreased again, and the BLEU scores slightly decreased.

Additionally, we also remove the three contrastive learning losses for the content preservation (L_c(c′), $Lc(c~′)$⁠) to study their effect on the results. The scores are also listed in Table 5. We can see that removing any one of the two losses would cause an increase in the TA score, which means all of the content preservation losses are limitations on the style latent space. Both the CBLEU-4 and TBLEU scores decrease a lot after removing the two content preservation losses. In particular, it seems that L_c(c′) has the largest effect on the scores, which is sensible, because a more distinguishable content space is easier for content preservation intuitively.

To investigate the effect of the temperature hyperparameter τ, we run the model several times with different values of τ, and visualize the latent space in Figure 7. According to Figure 7, when τ has a small value, the latent spaces for the different style values tend to be connected in some area. In contrast, the latent spaces become separated when the value of τ increases. The reason is that when the temperature τ is getting large, the distinction between the positive and negative examples in the contrastive losses tends to be underestimated. Hence, the model needs to work harder to make the distinction large, and thus the latent spaces are getting more separated.

We sampled some generated text when we are transferring the sentiment attribute from one to another, the results are shown in Table 6. The corresponding results for tense are shown in Table 7. According to the results, the content of text almost remains unchanged, while the target attribute was changed to what we expected.

Furthermore, we evaluated more complex emotion attribute transfer cases from the GoEmotions dataset. We transformed the emotions according to the Ekman taxonomy and presented the results produced by CLD using both the vanilla and VAE architectures. These results are tabulated in Table 8.

We also conducted a human evaluation for the attribute control results. We sampled 1,000 examples from each of Yelp and Amazon, and changed their attribute value to the opposite value (“Positive”→“Negative”, “Negative”→“Positive”). Then, we collected the generated sentences and asked 3 data graders to give a score to the sentences on 3 metrics (transfer accuracy (TA), content preservation (CP), and language quality (LQ)). Among them, TA is a percentage, CP and LQ are scored between 1 ∼ 5. The detailed questions are listed in the appendix. We randomly shuffled the sentences to remove the ordering hint. The final result of human evaluation is shown in Table 9. The inter-rater agreements (the Krippendorff alpha values) of the three metrics are 0.84, 0.89, and 0.92, all of them are acceptable due to Krippendorff’s principle (2004). We can see that our proposed method CLD outperforms the baseline in each of the human evaluation metrics. We also listed some generated cases in Appendix 4.6.

Recent work has explored utilizing large language models (LLMs) like ChatGPT and GPT-4 for controllable text generation. For example, Reif et al. (2021) have proposed methods to steer text style transfer in these LLMs by conditioning on discrete attributes or continuous latent representations. Compared to our approach, a key difference is that we train our model end-to-end to disentangle latent attributes, while LLMs rely on prompting or fine-tuning approaches applied post-hoc.

While promising, utilizing LLMs for attribute-controlled generation remains challenging. The discrete prompting approach can yield brittle or superficial style changes, as the models’ understanding of prompted attributes is imperfect and limited to correlation patterns in the pretraining data (Reif et al., 2021; Luo et al., 2023). Latent space steering has shown more coherent style transfer, but current methods rely on complex optimization schemes or assume access to an attribute classifier (John et al., 2019; Sha and Lukasiewicz, 2021). In contrast, our model learns disentangled representations directly from data through closed-loop contrastive training.

Our model is not designed to control the intensity of an attribute, like generating some neutral sentence instead of “pos” or “neg”. If we want to generate a neutral sentence anyway, we just need to take the average vector of the mean value of the “pos” and “neg”, and replace the original semantic style vector. Then, the decoder will generate a neutral sentence. However, this method will not always be successful, because there is no guarantee that these latent spaces are smoothly distributed with overlapping regions, and the decoder may not have been required to generate such texts with novel style features during training. To better control the attribute’s intensity, it is required to design some special mechanics in a supervised manner.

Apart from the simple text attributes, there are also some complex attributes like some specific author’s style of writing, which are usually intertwined together in the latent space. Discrete categorical style types are hard to design for such kind of complex attributes. Whether disentanglement can be used for controlling complex attributes requires further research.

In this paper, we proposed a novel explicit disentanglement method, called contrastive learning disentanglement (CLD), which uses contrastive learning as the core method. Differently from previous works, we re-disentangle the reconstructed sentences, and conduct contrastive learning between the disentangled vectors and the re-disentangled vectors. To encourage the disentanglement of the attributes’ latent space, we propose the re-disentangled contrastive loss L_re and the transferred re-disentangled contrastive loss $L~k$⁠. The latter fully imitates the attribute control process. To encourage content preservation, we proposed the content contrastive loss L_c, which contains three sub-losses. These sub-losses make the content space more distinguishable and encourage the content keep unchanged during attribute control. Our proposed method is not only much easier in the mathematical derivations, it also outperforms all the compared methods in the evaluation metrics according to our experimental results.

This work was supported by the ESRC grant ES/S010424/1 “Unlocking the Potential of AI for English Law”, by the National Natural Science Foundation of China under grants KZ37117501, ZG216S23E8, and 62306024; by the Alan Turing Institute under the EPSRC grant EP/N510129/1; and by the AXA Research Fund. We also acknowledge the use of Oxford’s Advanced Research Computing (ARC) facility, of the EPSRC-funded Tier 2 facility JADE (EP/P020275/1), and of GPU computing support by Scan Computers International Ltd.

Q: Do you think the given sentence belongs to positive sentiment or negative sentiment?

• A: Positive.

• B: Negative.

Q: Do you think the generated sentence has the same content with the original sentence, although the sentiment/tense is different?

Please choose a score according to the following description. Note that the score is not necessary to be integer, you can give scores like 3.2 or 4.9 by your feeling.

• 5: Exactly. The contents are exactly the same.

• 4: Highly identical. Most of the content are identical.

• 3: Half. Half of the content is identical.

• 2: Almost Not the same.

• 1: Totally different.

Q: How fluent do you think the generated text is? Give a score based on your feeling.

Please choose a score according to the following description. Note that the score is not necessary to be integer, you can give scores like 3.2 or 4.9 by your feeling.

• 5: Very fluent.

• 4: Highly fluent.

• 3: Partial fluent.

• 2: Very unfluent.

• 1: Nonsense.