MACSum: Controllable Summarization with Mixed Attributes Open Access

Text summarization is the task of compressing the input text into a concise and coherent version by preserving salient information. There has been substantial progress in generic summarization by generating one overall summary for each input (McKeown and Radev, 1995; Erkan and Radev, 2004; Rush et al., 2015; Cheng and Lapata, 2016; See et al., 2017; Paulus et al., 2018). However, readers have diverse preferences when summarizing the same article, such as topics, speakers, or lengths of the summary (Fan et al., 2018; Zhong et al., 2021; Goyal et al., 2022b). Therefore, generating customized summaries to meet different preferences is a natural capability of summarization systems.

Due to the lack of a human-annotated controllable summarization benchmark, existing research has to adapt generic datasets to create pseudo-controllable summaries (Fan et al., 2018; He et al., 2020; Zhong et al., 2021; Goyal et al., 2022b; Chan et al., 2021). He et al. (2020), for example, extract topics from a generic summary by assuming the summary is controlled by the extracted topics to evaluate summarization over topics. However, this adaptation-based setting raises three issues. First, the adapted summaries are not really written with the guidance of being controlled by the designed attributes. Second, this conversion can only build one target summary for each source, while it is preferable to have summaries with different control attributes for the same input article. Third, for attributes like Extractiveness or Specificity, there are no straightforward adaptation methods.

Meanwhile, previous studies mostly focus on controlling a single attribute, e.g., generating a short summary or a highly abstractive summary. However, mixing different control attributes is more challenging and underexplored (Russo et al., 2020). For example, Figure 1 shows a case of mixed-attribute control by requiring a short summary regarding “Pope Francis”, or a highly extractive and highly specific summary on the topic “blood moon”. Users can simultaneously control different attributes in the generated summary.

In this paper, we propose MACSum, a human-annotated benchmark for controllable summarization with mixed attributes. In MACSum, source texts are collected from both news and dialogue domains. We define five control attributes of summarization by synthesizing previous studies (Chan et al., 2021; Liu et al., 2018; Fan et al., 2018), including Length (Len), Extractiveness (Ext), summarization Specificity (Spe), Topic (Tpc), and Speaker (Spk).¹ For each input source, we sample a set of different combinations of these attributes for human annotations. The resulting MACSum dataset contains a rich set of annotations of human-written summaries for the same input with different mixtures of control attributes. Table 1 compares MACSum with previous work, and MACSum is the first to mix these five attributes with human annotations, covering both dialogue and document source texts.

Furthermore, to establish a baseline of mixed-attribute control, we design two simple yet effective frameworks that can steer a summarization model by either hard prompt (HP) or soft prefix (SP) inspired by prompt learning (Raffel et al., 2020; Li and Liang, 2021). For each value of a control attribute (e.g., long length), in the HP framework, we prepend the description of control attributes (e.g., “Length: Long”) to the input source as hard prompts; in the SP framework, we assign a set of external parameters, called soft prefixes, to the model. The summarization model is trained to summarize an input with hard prompt/soft prefixes of different control signals. We evaluate these baseline models on MACSum with proposed two automatic evaluation metrics measuring the quality of control. Our results and analysis in two domains reveal that the HP framework yields the best performance on all automatic metrics and human evaluations, although mixed-attribute control is still challenging.

Previous work on controllable text summarization focuses on length (Fan et al., 2018; Liu et al., 2018; Makino et al., 2019; Saito et al., 2020; Liu et al., 2022; He et al., 2022; Goyal et al., 2022a), entity (He et al., 2020; Narayan et al., 2021; Maddela et al., 2022; Hofmann-Coyle et al., 2022), aspect (Tan et al., 2020; Amplayo et al., 2021), content (Dou et al., 2021; Xiao et al., 2022), style (Cao and Wang, 2021), granularity (Zhong et al., 2022), and abstractiveness (Song et al., 2020). Query-focused summarization (Dang, 2005; Fisher and Roark, 2006; Daumé III and Marcu, 2006) generates summaries for specific user information requests, but it does not explicitly control the output style. Furthermore, interactive summarization (Bornstein et al., 1999; Leuski et al., 2003) and reinforcement learning guided summarization (Paulus et al., 2018; Böhm et al., 2019; Stiennon et al., 2020) have been used to incorporate human preferences and feedback, yet the human feedback explored so far is largely limited to the generic quality of summaries instead of fine-grained attributes. Notably, Chan et al. (2021) propose a constrained Markov Decision Process for controllable summarization for different attributes, but it is unclear if it can perform multi-attribute control. Goyal et al. (2022b) investigate multi-feature control by mixing multiple decoders, yet their solution is only based on decoding improvements that yield suboptimal controlling performance. Therefore, most previous work is over-specialized for controlling particular attributes, while controlling multiple attributes is still underexplored. Further more, existing works are mostly evaluated on pseudo datasets adapted from generic summarization datasets.

Prompt learning is first proposed in GPT-3 (Brown et al., 2020), where large pretrained language models can perform desired tasks with the guidance of instructions and examples. Some efforts explore prompt-tuning using natural language by converting original inputs into cloze-style questions and then tuning language models (Shin et al., 2020; Schick and Schütze, 2021; Chen et al., 2022; Min et al., 2022). However, most of them focus on natural language understanding tasks and usually need a careful selection of prompts. Instead of using human-crafted tokens, other work explores using continuous vectors as prompts (Lester et al., 2021; Qin and Eisner, 2021; Liu et al., 2021; Li and Liang, 2021). Among them, prefix-tuning is particularly designed for text generation (Li and Liang, 2021). Prefix-tuning prepends trainable vectors to each layer of language models as prefixes and keeps other parameters frozen during training. In this work, we propose two methods for mixed attribute controllable summarization based on prompt-tuning and prefix-tuning, respectively.

To provide a benchmark for controllable summarization, we propose MACSum, a high-quality human-annotated mixed-attribute controlled summarization dataset. Inspired by several previous studies on controllable generation (Chan et al., 2021; Liu et al., 2018; Fan et al., 2018), MACSum is annotated with 5 types of control attributes, including Topic, Speaker, Length, Extractiveness, and Specificity (Section 3.1).

As shown in Figure 1, these five attributes can be combined together in various designs (Section 3.1). Additionally, Topic and Speaker can have multiple values as well, i.e., more than one speaker or topic to focus on. In annotation, we require the corresponding summary to fulfill all attributes together.

The data annotation pipeline is divided into four steps (Figure 2). First, we carefully select the source texts from several widely used summarization datasets in news or dialogue domains. Second, some automatic tools are leveraged to form a pool of candidate attributes as guidance for the next step. Third, the annotators manually label five attributes to form a control attribute set and repeat the process multiple times. Finally, the annotators write down the summary that meets each control attribute set.

MACSum provides five attributes to control the summary generation.

indicates certain contents of the source that users are particularly interested in. The summary should only contain contents that are related to the given topics. We provide multiple keywords such as “remote control, financial information” for annotators as candidate topics.

indicates certain speakers in a dialogue whose content is preferred by the user. In MACSum (MAC-Dial only), this is specified by giving the name of certain speakers, such as “Program Manager”.

indicates the number of words of the summary, serving the time budget for users to read this summary. In MACSum, Len is controlled by [short, normal, long], three values of this attribute.² Our annotation guideline provides a reference range of compression ratio and word count for each length value.

describes the proportion of the summary extracted from the source text. This is useful when users sometimes want content directly extracted from the source, while sometimes they want more abstractive and more readable results. In MACSum, Ext can take values of [normal, high, full].

means how many details or descriptive contents we need to include in the summary. Referring to Louis and Nenkova (2011), different users can prefer more general summaries or more specific summaries. MACSum contains two levels of Spe control, namely [normal, high], where normal is the natural specificity and high requires more specific contents.

Specificity differs from Length. Length is the number of words, while Specificity is the density of descriptive contents (e.g., numbers, entities, and names). Thus, a short summary can have a higher Specificity than a long one.

MACSum supports Mixed-Attribute Control because it is a natural need for users to control multiple aspects at the same time, e.g., wanting the summary to be short, highly extractive, and only talking about some topics. To this end, as shown in Figure 1, the samples in MACSum can control multiple attributes simultaneously. We require the annotated summaries to meet all requirements at the same time. If some combinations are considered too difficult to fulfill, we allow annotators to skip them in rare cases. We provide detailed distribution of attributes in Figure 3.

MACSum covers source text from both document and dialogue summarization tasks. We pick CNNDM (Hermann et al., 2015) as the document dataset and QMSum (Zhong et al., 2021) as the dialogue dataset. CNNDM is a large-scale document summarization dataset containing news stories along with their corresponding highlights, collected from CNN and Daily Mail websites. QMSum is a popular query-based meeting summarization dataset. It contains the transcripts of three domains, including AMI, ICSI, and committee meetings of the Welsh Parliament and the Parliament of Canada. For CNNDM, we randomly pick 10k documents in the test set for the annotation. For QMSum, we first split each meeting into shorter units according to the topic partition and discard the units that are longer than 5000 tokens.

For Topic, we first use a keyword extraction tool (Boudin, 2016) to extract the top 20 keywords from the source text as candidates. For Speaker, we collect all speakers in the source text to form a candidate set. For the remaining Length, Extractiveness, and Specificity attributes, we generate their values and combination randomly from a uniform distribution, mimicking the behavior of users with diverse needs for customized summaries.

We hire 4 native English speakers as annotators. The annotators can either freely choose topics from the candidate topics or write the keywords by themselves. As for the Speaker attribute, we ask the annotators to pick one or more names from the candidate set. Besides, Length, Extractiveness, and Specificity are automatically filled with randomly generated values.

Attribute generation repeats several times for each source to form various attribute combinations, so-called samples. Overall, each source text contains eight samples for every 2000 words.

We first ask all annotators to read our annotation guideline and 10 annotated examples. Afterward, given several combinations of control requirements, i.e., the control attribute sets, the annotators follow our guidance and write a summary for each control combination.

We also ask them to annotate the related text spans for use in future work, such as retrieval-based methods. Related text spans are the turns/sentences in the source that are most relevant to the golden summary. These spans are the minimum necessary turns/sentences the annotators need to produce the complete summary.

Finally, the annotators read the summary again for quality assurance, and we ask them to write a short title for this summary, e.g., “discussion of remote control style”. This is helpful for future work such as title generation, and it also provides us with a quick way to verify whether the annotators read their generated summaries.

First, we control the annotation quality through a careful pilot test. Before the annotation process starts, annotators are carefully selected via a pilot test. We assign each annotator the same three input texts with various mixed attributes, and we choose the qualified annotators according to annotation results.

Second, we conduct a weekly sampling inspection. We frequently monitor the quality of annotations. We collect the results weekly and provide feedback to the annotators to ensure quality.

Along with the annotated benchmark, we also design a system of automatic metrics for evaluating the model’s capability to generate controllable summaries. For each attribute, we define its own attribute metric function to represent the degree of control. We then propose Control Error Rate (CER) and Control Correlation (CC). CER measures the distance between the generated and golden summary in terms of their degrees of control using attribute metric functions. A good model should have smaller CER ↓. CC measures the distribution of attribute metric functions among generated summaries with different attribute values, representing the model’s capability to correlate to the definition of the control attribute. A good model should have a CC distribution that is similar to that of the golden summary ↕. In addition, we also report F-1 of ROUGE-1/2/L (Lin, 2004) for the general quality of the summary ↑.

For each of the five control attributes, we define its own attribute metric f_r which maps the summary to a real number that represents the degree of control. Topicf_Tpc is the proportion of topic keywords shown in the summary. Speakerf_Spk is the number of tokens spoken by the selected speakers divided by the total number of tokens in the summary. Lengthf_Len is the number of tokens in the summary. Extractivenessf_Ext is the average of ROUGE-2 precision and ROUGE-3 precision (Lin, 2004) of the generated summary against the input. For Specificity, inspired by previous studies (Resnik, 1995; Amplayo et al., 2021), we find that verb, noun, numeral, and the total number of tokens show the most significant information about specificity. Thus, we define f_Spe = (0.1 × vb + 0.2 × tok + 0.3 × nn + 0.4 × cd)/n_s, where vb, tok, nn, cd, and n_s represent the number of verbs, tokens, nouns, numeral tokens, and the number of sentences in the summary.

Table 2 shows the statistics. MACSum covers two domains (MAC-Doc for news and MAC-Dial for dialogue) with 8333 annotated summaries (5379 in MAC-Doc and 2954 in MAC-Dial), paired with 1353 source inputs (943 in MAC-Doc and 410 in MAC-Dial). The averaged number of tokens in sources of MAC-Doc is shorter than that in the original QMSum dataset since we truncate the input into segments. We split the source text randomly into training/valid/test sets with 80%/10%/10%.

With definitions from Section 3.3, Table 3 calculates automatic attribute metrics for all 5 control attributes. As presented, the annotated summaries with different control attribute values can distinguish from each other by a large margin. For example, samples with Len: long have a much longer input, and samples with Ext: full have a higher extractiveness metric. This verifies the high annotation quality of MACSum and also proves that our proposed attribute metrics are consistent with the control objective of each control attribute.

Figure 3 shows the ratio of different combinations of the control attributes. This illustrates diverse combinations of mixed-attributes summaries by controlling Len, Ext, and Spe together in one sample.

For setting baseline results on MACSum, we propose three models following previous research on controllable text generation using prompt learning. With the same input and different prompts, the large pretrained model is able to generate different results for different tasks, such as summarization and translation (He et al., 2020; Fan et al., 2018; Raffel et al., 2020). As shown in Figure 4, we leverage two types of prompt learning approaches to control the attributes of summaries, namely, hard prompt (HP) and soft prefix tuning (SP). We also test the combination of them, HP+SP.

uses the description of control attributes as the hard prompt. Each attribute is formed as “Attribute: Value”, where “Attribute” can be “Topic, Speaker, Length, Extractiveness, Specificity”, and “Value” is the corresponding value (e.g., High or Normal) of the attribute. We concatenate 5 control attributes using “;” and prepend it to the input source.

follows Li and Liang (2021). We prepend external trainable parameters to both the encoder and decoder to control the summarization model. For controlling Len, Ext, and Spe, we assign m prefix embeddings for each attribute value where m is a hyper-parameter meaning the length of prefix, i.e., prefix length. Readers can refer to Li and Liang (2021) for implementation details. For example, for Len: Long, we assign $ELen:long=[eLen:long1,⋯,eLen:longm]$ where $eij$ is a vector with dimension of word embedding. And for controlling an input case with a set $V$ of mixed requirements, we sum the embeddings of all control attribute values: $E=[∑v∈Vev1,⋯,∑v∈Vevm]$⁠. And for controlling Tpc and Spk, we use the embeddings of input topics words E_Tpc and input speaker names E_Spk. This list of embedding vectors E is then prepended to each layer of the Transformer-based summarization model as external key/value vectors in its self-attention operations. E_Tpc and E_Spk are prepended only to the input layer.

combines both approaches by prepending the hard prompt of five attributes in HP and using prefix tuning in SP.

In this section, we present the implementation details, experimental results, and human evaluation of models on MACSum dataset.

We use PyTorch and the HuggingFace library (Wolf et al., 2019) to implement our model. The experiments are conducted on 8 A100 GPUs.

We use BART (Lewis et al., 2020) as the backbone model. We also use a vanilla BART trained without control attribute input as a weak baseline (Appendix A). If not mentioned, we initialize the backbone using BART-large-cnn and then finetune it on the MACSum dataset. We pick the 3e – 5 learning rate searching from {1e – 5, 3e – 5, 1e – 4}. Additionally, n-gram blocking is set to 3, and we use the AdamW optimizer with 500 warmup steps. Dialogue inputs are flattened by separating turns with “<∖s>” which we find yields better results.

As mentioned in Section 3.3, we calculate Control Error Rate (CER) and Control Correlation (CC) metrics for evaluating control quality, and we also report ROUGE scores for evaluating summarization quality. For a model, its performance is better when the CER value is lower↓, ROUGE is higher↑, and its CC is closer to the golden summary↕.

Table 4 shows the results of MAC-Doc. The HP model obtains the highest performance on both CER and CC across all 5 control attributes. Compared with the HP model, the SP model has similar control ability on Ext and Spe. However, it does not perform well on Len and Tpc. This could be the result of using the pretraining checkpoint that has learned some knowledge about the length-related hard prompt before training (Section 6.3).

Table 5 displays the results of MAC-Dial. Similar to the MAC-Doc dataset, the HP model obtains the highest scores on most of the metrics. However, the overall performance of length decreases because using the pretrained CNNDM checkpoint does not lead to performance gain in the dialogue domain (Section 6.3).

It is worth noting that the CER should not be compared across datasets, because its scale is different from different datasets. For example, random uncontrolled BART in MAC-Doc obtains 1.177 CER for Ext while it is 0.544 in MAC-Dial.

Although automatic metrics usually provide a speedy comparison, these metrics cannot easily evaluate the quality of the control, especially mixed-attribute control. Thus, we also conduct a human evaluation for the controlled summaries.

We hire two evaluators with expertise in English and text summarization. We show them randomly selected summaries generated by different systems with the source text and control attributes. The evaluators answer a yes/no question: “For the given summary, does it follow the control requirement of this attribute?” Specifically, we select golden summaries, summaries generated by HP model, and summaries generated by HP+SP model. For each model, we pick 30 samples from MAC-Doc and MAC-Dial separately, resulting in 180 summaries in total. Furthermore, we compute Cohen’s kappa (Cohen, 1960) to measure the agreement between evaluators.

Table 6 shows the human evaluation results. Each number (except for kappa) is calculated by the count of yes answers divided by the total count of questions, indicating the control ability of the model. As shown, the HP model performs better than HP+SP on most of the attributes. This result confirms the consistency of our proposed CER and CC with human evaluation.

Besides, golden summaries always rank first, and the kappa score of the two evaluators is over 0.8. These two results also verify the high annotation quality of MACSum, because human evaluators agreed that the golden summaries followed the control requirements most of the time.

For a deeper understanding of the task of mixed-attribute controllable summarization on MACSum, we conduct analysis including attribute difficulty, attribute dependency, model pretraining, and present several example outputs for case studies.

Models have different difficulties in controlling certain attribute values, as some attribute values can be easier or harder to be controlled. We analyze this by comparing CER for different attribute values of the HP model’s outputs. As shown in Figure 5, for MAC-Doc, the system obtains a higher CER on Len: normal samples compared with the other two values of Len, showing that normal is more difficult to control, and the hardest values in controlling Ext and Spe are both high. For MAC-Dial, the hardest values in controlling Len, Ext, and Spe are short, normal, and high, respectively.

In mixed-attribute controllable summarization, we notice interesting dependencies among attributes, as changing one attribute influences the other one. To analyze this, we randomly select 200 samples from the test set for each attribute, and randomly change this attribute to another value to form a new sample (e.g., from Len: long to Len: short). Then, the same HP model, without further training, is used to generate summaries on these new samples. We evaluate the performance difference between the newly predicted summaries $ŷ′$ and the originally predicted summaries $ŷ$ via $CER(ŷ′,ŷ)$⁠.

Figure 6 shows the performance change. As can be seen, for MAC-Doc, Len has the highest dependency toward other attributes, while Spe has the lowest. For MAC-Dial, Ext has the highest dependency, while Spe has the lowest. We believe this is because the model in MAC-Doc has a strong control ability towards Len. Thus, the value change of Len will influence more on other attributes.

We investigate the effect of pretraining on the control ability of summarization models. For two HP models initialized by BART-large and BART-large-cnn separately, we compare their results after finetuning them on both MAC-Doc and MAC-Dial.

As shown in Table 7, for MAC-Doc, the BART-large-cnn initialized model is able to control the length substantially better than the vanilla BART-large initialized model. On the contrary, for MAC-Dial, the advantage of the BART-large-cnn checkpoint is negligible. Using BART-large-cnn or not only slightly influences the control ability of all attributes in MAC-Dial. We believe the reason for this is that the CNNDM pretraining provides certain useful information for the model to learn the ability to control attributes on news articles.

We show three case studies in Table 8, discussing three typical phenomena in mixed-attribute controllable summarization, namely, Topic Defocus, Length against Specificity, and Extractiveness against Readability.

In Table 8 Case 1, MACSum asks for a summary focusing on the topic of “education”. Although the human-annotated summary does not contain the topic word, its contents are still highly related to “education”. This shows that human annotators have the flexibility of conducting high-level summarization of the topic. In contrast, although the model-generated summary contains the topic word, its content is poorly structured. This shows the challenge of topic defocus, a phenomenon where models rely too much on explicitly containing the topic words when generating topic-controlled summaries.

Another challenge is the contradiction between long length and low specificity. Long summaries contain more tokens and inevitably invite more specific information. On the contrary, short summaries only describe core events using a few words and are naturally biased towards low specificity. As shown in Table 8 Case 2, when Len is short and Spe is high, both HP and HP+SP generated summaries are longer compared with the human-annotated summary.

As shown in Table 8 Case 3, when Ext is full, the model-generated summaries are choppy and unnatural, in particular for dialogues. When humans are asked to annotate fully extractive summaries, they may have to write unnatural sentences, and this phenomenon is amplified by a trained summarization system. As shown in the table, the HP+SP generated summary is not grammatical and consists of short phrases instead of complete sentences. This can be explained by the fact that the complicated dialogue discourse structures and frequent interactions between different interlocutors make salient information sparse.

We propose MACSum, a high-quality human-annotated benchmark for mixed-attribute controllable summarization. It contains 5 types of control attributes, including Topic, Speaker, Length, Extractiveness, and Specificity. To the best of our knowledge, MACSum is the first dataset with mixed attributes as well as human annotations. We explore the hard prompt and soft prefix models and evaluate them on MACSum. Results and analysis demonstrate that hard prompt models yield the best performance and also show this is a challenging task as a large gap between machine learning models and human still exists.

Future work can design more effective models for the mixed-attribute controllable summarization task, or explore mixed-attribute control on other generation tasks.

We thank Ming Zhong, Tao Yu, Haoran Zhang, Sarkar Das, and Nan Zhang for their valuable feedback and suggestions. We also would like to thank the reviewers and action editor for their helpful comments and reviews.

We list the implementation details for the models.

For HP+SP on MAC-Doc, we load the HP trained model first and set different learning rates for the language model and prefixes, i.e., 3e −5,1e −6 separately, and remove the Len prefix from the model. This is because we find that the HP model obtains high performance with Len related attributes very well, due to the pretrained BART-large-CNN checkpoint. Using prefix tuning or tuning the language model with a large learning rate will hurt the performance (Section 6.3). For HP+PE on MAC-Dial, we only set the different learning rates, but we do not load the checkpoint or remove the Len prefix. This is because the CNN pretrained checkpoint is not significantly beneficial for MAC-Dial (Section 6.3).

The model is a pretrained BART model which only prepends the hard prompt of topic and speaker to the input, which means it does not control the rest of the attributes. This is the baseline to justify if we control these three attributes.

We have four annotators with native English background. Before the pilot test, we also supply annotators with professional training for high-quality annotation and provide annotation visualization tools for the annotators to regularize the annotation process. For each sample, we ask the annotators to inspect the quality and decide to keep the annotation or discard it due to difficulty or errors. We combine the annotations of each week to form the MACSum dataset with careful processing: we discard the invalid samples reported by the annotators and use a program to filter out the other invalid samples with empty or wrong text.

We write annotation guidelines of MACSum for two purposes. First, the guidelines are used as our criteria to evaluate annotators during the pilot test. Second, during annotation, we provide annotation guidelines to the annotators and ask them to carefully follow them. For both purposes, the guidelines are a key step to ensure the quality of the whole annotation process. Thus, we pick out some of the details in the guidelines. Note that the following paragraphs are directly copied from the guideline document and shared across all four annotators.

A dialogue may contain multiple speakers. if we specify certain speaker names as the control attribute, it means we only care about what these speakers say in the dialogue. So we need to focus on the dialogue turns spoken by these speakers and write the summary for them.

Topic is represented by a set of keywords (usually) copied from the dialogue. A dialogue may contain multiple topics, we need to summarize the content that is only related to the given topic.

Normal length: the length of the summary should equal 15%–25% of the related text spans. For example, the dialogue contains 2000 words, and the related text span for the labeled speaker contains 1000 words. Then we need to write 15%–25% x 1000 = 150 – 250 words for the summary. Long summary: the length of the summary should equal 30%–35% of the related text spans. Short summary: the length of the summary should equal 5%–10% of the related text spans. These criteria should be dynamically modified, the target of length control is to differentiate the length of the different outputs. We can adjust the criteria a little bit if the lengths of the three types of summaries are too similar.

Normal extractiveness: the same as a natural summary that humans will write. High extractiveness: copy more sentences/tokens from the source text compared with normal extractiveness. Full extractiveness: copy all the sentences/tokens from the source text. Again, this can be modified if we can better differentiate summaries with different abstractiveness.

Normal specificity: the same as a natural summary that humans will write. High specificity: include more descriptive content in the source text compared with normal specificity.

Table 9 shows five examples of our proposed MAC-Doc dataset. Note that samples 2 and 3, and samples 4 and 5, only differ in Len, Ext, and Spe.