TaxoPro: A Plug-In LoRA-based Cross-Domain Method for Low-Resource Taxonomy Completion Open Access

Taxonomies are knowledge structures that hierarchically organize concepts through hypernym- hyponym (“is-a”) relations. They find extensive applications in fields such as natural language processing (Bai et al., 2022; Hu et al., 2022b), recommendation systems (Cheng et al., 2022), and information retrieval (Karamanolakis et al., 2020).

Most current taxonomies are manually curated by domain experts, which is both time-consuming and labour-intensive. With the constant emergence of new concepts, keeping taxonomies up-to-date for downstream applications has become a critical challenge (Shen et al., 2020; Zhang et al., 2021). To solve this problem, significant effort has been dedicated to the taxonomy expansion task (Shen et al., 2020; Liu et al., 2021; Xue et al., 2024). In this task, the existing taxonomy is expanded by inserting the new concept (query) to the most appropriate hypernym (parent) within the existing taxonomy as a leaf node. However, recent researchers contend that the “leaf-only” assumption is unsuitable (Zhang et al., 2021), leading to significant limitations in real-world scenarios (Wang et al., 2022a). Thus, they turn to the taxonomy completion task (Zhang et al., 2021; Xu et al., 2023; Niu et al., 2024), where the query is inserted between a pair of hypernym and hyponym (child). For example, the query “wearable device” is inserted between the parent “electronic equipment” and the child “AR glass” as shown in Figure 1.

In practical scenarios, the low-resource setting, where only a limited number of concepts exist in the existing taxonomy, is prevalent as most taxonomies typically comprise around a thousand concepts (Takeoka et al., 2021). Under such a setting, the early taxonomy expansion and completion methods (Shen et al., 2020; Zhang et al., 2021; Manzoor et al., 2020) suffer from performance degradation due to insufficient training samples (Takeoka et al., 2021). Several studies (Liu et al., 2021; Takeoka et al., 2021; Xu et al., 2023) have shown that this can be mitigated by incorporating prior knowledge from the pre-trained language model (PLM). However, such knowledge could be generic and irrelevant to the taxonomy tasks (Gururangan et al., 2020; Diao et al., 2023), thus limiting the performance of these PLM-based techniques. Meanwhile, taxonomies across different domains store the same type of knowledge, i.e., hierarchical relations between concepts. Consequently, the high-resource taxonomy can serve as an extra knowledge base for completing the low-resource taxonomy. In this paper, we explore the research question of how to capture knowledge from the high-resource taxonomy for the PLM-based low-resource taxonomy completion enhancement?

Inspired by the recent studies (Diao et al., 2023; Zhang et al., 2023b; Wang et al., 2023c) that utilize the parameter-efficient fine-tuning (PEFT) techniques (Houlsby et al., 2019; Li and Liang, 2021) for knowledge storage and transfer, we utilize LoRA (Hu et al., 2022a), a widely used PEFT technique that leverages low-rank matrices as the knowledge update, to capture domain-shared knowledge from a high-resource source taxonomy and apply it to complete a low-resource target taxonomy. Specifically, we propose a LoRA-based cross-domain method that can be plugged into the PLM-based taxonomy completion techniques. Our method has two main modules: (i) knowledge decomposition and (ii) shareable knowledge flow control. In the first module, we decompose the knowledge of each taxonomy into domain-shared and domain-specific components to prevent negative interference between these two types of knowledge (Du et al., 2023). We store these components using separate low-rank matrices, which are updated through task-specific losses across various domains. In the second module, we employ two auxiliary losses to guide the flow of shareable knowledge. Specifically, we pull shareable knowledge into the domain-shared matrices and push it out from the domain-specific matrices. Our objective is to maximize the extraction of shareable knowledge from the high-resource taxonomy, thereby improving the completion of the low-resource taxonomy.

We plug the proposed method into two representative PLM-based taxonomy completion techniques and conduct extensive experiments on three low-resource taxonomy datasets. Experimental results show that our method improves the PLM-based techniques and achieves state-of-the-art performance in low-resource taxonomy completion.

In summary, our contributions include:

• We propose TaxoPro, a LoRA-based cross-domain framework that can be plugged into the PLM-based taxonomy completion techniques. To the best of our knowledge, it is the first work that captures shareable knowledge from the high-resource taxonomy to enhance low-resource taxonomy completion.

• We leverage the knowledge decomposition to prevent negative interference between domain-specific and domain-shared knowledge and employ two auxiliary losses to regulate the flow of shareable knowledge.

• We conduct extensive experiments to validate the effectiveness of the proposed method. Experimental results demonstrate that TaxoPro enhances PLM-based techniques and achieves state-of-the-art performance in completing low-resource taxonomies.

With regard to automatic taxonomy enrichment, there exist two lines of research: Taxonomy Expansion and Completion. To expand the existing taxonomy, researchers (Shen et al., 2020; Yu et al., 2020; Ma et al., 2021; Wang et al., 2021; Liu et al., 2021; Takeoka et al., 2021; Cheng et al., 2022; Phukon et al., 2022; Xu et al., 2022; Zhai et al., 2023; Jiang et al., 2023; Sun et al., 2024; Zhu et al., 2023; Mishra et al., 2024; Xu et al., 2024; Shen et al., 2024; Zeng et al., 2024; Qingkai et al., 2024; Meng et al., 2024; Moskvoretskii et al., 2024) attempted to insert the emergent concepts to the most appropriate leaf position.

In taxonomy completion, Zhang et al. (2021) extended the candidate insertion position from “leaf-only” to a pair of parent and child nodes. GenTaxo (Zeng et al., 2021) completed the taxonomy in a concept generation manner. TAXBOX (Xue et al., 2024) enhanced taxonomy completion by using specialized geometric scorers in box embedding space. TaxoEnrich (Jiang et al., 2022) and QEN (Wang et al., 2022a) incorporated sibling relations for semantic-rich concept representation. TaxoComplete (Arous et al., 2023) captured fine-grained information from distant nodes. CoSTC (Niu et al., 2024) captured diverse relations and improved representations through intra-view and inter-view contrastive learning. TEMP (Liu et al., 2021) and TacoPrompt (Xu et al., 2023) leveraged pre-trained language models as an implicit knowledge base and achieved remarkable performance. Additionally, researchers explored variant taxonomy completion settings. For instance, ATTEMPT (Xia et al., 2023) suggested initially identifying the parent and then locating all its children within the taxonomy. ICON (Shi et al., 2024) focused on generating new concepts based on the taxonomy’s structure and existing concepts, which are then inserted into the taxonomy. These settings are beyond the scope of this paper.

In this paper, we explore the low-resource taxonomy completion scenario where little in-domain labelled data is available. Unlike Musubu (Takeoka et al., 2021), which solely relies on pre-trained knowledge, our focus lies in capturing shareable knowledge from the high-resource taxonomy for enhancing the completion of the low-resource taxonomy.

One of the recently popular techniques in knowledge transfer involves decomposing input data into domain-specific and domain-shared knowledge (Sarafraz et al., 2024; Wang et al., 2023a, b; Wei et al., 2023; Ben-David et al., 2020). By setting distinct objectives for each, domain-specific information can be separated, enabling the use of domain-shared knowledge for predictions in new domains (Zhang and Gao, 2024). Early work, such as Daumé (2007), proposed expanding the feature space into common, source-specific, and target-specific components. Building on this, Bousmalis et al. (2016) introduced Domain Separation Networks (DSN), which utilize separate encoders to explicitly model domain-shared and domain-specific knowledge, hypothesizing that this separation enhances the extraction of transferable knowledge.

Lately, parameter-efficient fine-tuning (PEFT) methods, which adjust only a subset of model parameters (Li and Liang, 2021; Zhang et al., 2023a, c), have gained traction in NLP for adapting pre-trained language models to downstream tasks (Chen et al., 2022; Dettmers et al., 2023). These methods facilitate knowledge transfer and composition, supporting the integration of diverse knowledge sources (Ding et al., 2023; Wang et al., 2022b; Mao et al., 2022). Researchers have also begun exploring PEFT for knowledge decomposition. For instance, Zhang et al. (2023b) proposed a framework for Event Argument Extraction across datasets using Prompt Tuning and Adapters to manage overlapping and specific knowledge in sequential learning phases. Similarly, Wang et al. (2023c) decomposed knowledge across tasks into shared and task-specific prompt vectors, with shared vectors learned from multiple tasks for efficient adaptation to new tasks.

In our work, we advance this line of research by developing an end-to-end LoRA-based knowledge decomposition approach tailored for taxonomies across domains. Our focus is on (i) disentangling noisy domain-specific knowledge from domain-shared knowledge and (ii) regulating shareable knowledge flow through auxiliary loss functions.

In this section, we provide a formal definition of the taxonomy and the taxonomy completion task.

Building upon the formalism established by Shen et al. (2020), we formalize a taxonomy as a directed acyclic graph $T=N,E$⁠, where nodes $N$ correspond to concepts and edges $E$ encode hypernym-hyponym relations through ordered pairs $〈p,c〉$⁠. This graph structure ensures that each parent concept p maintains maximal specificity while remaining semantically broader than its child concept c. Following Xu et al. (2023), a corpus $D$ is provided, from which the concept descriptions are extracted using established retrieval methods.

In this paper, we focus on the taxonomy completion task in a low-resource setting, where $T0$ comprises only a limited number of training samples. We incorporate an external input, specifically a high-resource taxonomy, to supply additional training samples for completing the low-resource taxonomy. Our goal is to capture cross-domain shareable knowledge from the high-resource taxonomy (source domain) to enhance low-resource taxonomy (target domain) completion.

In this section, we propose a cross-domain method that can be plugged into the PLM-based taxonomy completion techniques (§3.2). The method comprises a knowledge decomposition module (§4.1) and a shareable knowledge flow control module (§4.2), as illustrated in Figure 2.

Taxonomies across domains embody two types of knowledge: (i) domain-shared knowledge, as they all serve as repositories for hierarchical relations between concepts, and (ii) domain-specific knowledge, characterized by their unique semantic distributions and structural granularity. To effectively capture this, we decompose the knowledge from cross-domain taxonomies into domain-shared and domain-specific components. This decomposition achieves two objectives: (i) enhancing the performance on the low-resource domain by leveraging shareable knowledge from the high-resource domain, and (ii) mitigating the negative interference between domain-specific and domain-shared knowledge (Du et al., 2023) by separating noisy domain-specific knowledge.

To facilitate cross-domain learning, we sample B instances from each domain per batch during training. The loss $L$_domain^$M$_KD is then calculated as the average loss over all samples from the corresponding domain in a batch. Importantly, training samples for both source and target domain taxonomies are generated following the original sampling process of the plugged methods.

PLM-based taxonomy completion techniques solely rely on the backbone language model for computing probabilities at candidate positions. The computational complexity of this architecture follows $O(θ×d×l2)$⁠, with θ corresponding to model parameters, d indicating hidden dimension, and l quantifying average text sequence length. Assuming the number of training nodes is $∣Ntrain∣$ and the number of negative samples is N, the training time complexity of PLM-based techniques is $O(∣Ntrain∣×(1+N)×θ×d×l2)$⁠. Due to our method’s utilization of a high-resource taxonomy for extra training samples, it encompasses a larger $∣N∣$ compared to the plugged technique. The auxiliary loss calculation also requires extra encoding by the PLM injected with corresponding LoRA modules. Correspondingly, the computational cost during inference is $O(∣C∣×∣A∣×θ×d×l2)$⁠, where $∣C∣$ corresponds to query count and $∣A∣$ signifies candidate positions. Our approach minimally impacts inference time, as all inference operations are conducted solely on the target dataset. The training and inference time is reported in Appendix A.1 and A.2, respectively.

In this section, we detail our experimental settings. Implementation details are in Appendix A.3.

We leverage low-resource taxonomies from three different domains: Science, Equipment, and Food from SemEval-2015 Task 17 (Bordea et al., 2015) as the target taxonomies to evaluate the proposed TaxoPro. We construct their description corpus using the Wikipedia resource by the script provided by Wang et al. (2022a). Meanwhile, two high-resource taxonomies, MeSH and WordNet-Verb, are leveraged as the source taxonomies. MeSH is a widely used clinical domain taxonomy as a subgraph of the Medical Subject Headings (Lipscomb, 2000). WordNet-Verb is derived from SemEval-2016 Task 14 (Jurgens and Pilehvar, 2016) and it is the hierarchy of verbs from WordNet 3.0. We utilize their description corpus provided by Wang et al. (2022a). For our primary experiments, we use MeSH as the source dataset. The effects of source taxonomy choice will be discussed in Section 6.4.2, where we utilize WordNetVerb to test the impact of changing the source dataset. Following the typical experimental settings of previous taxonomy completion studies (Zhang et al., 2021; Wang et al., 2022a), we split the datasets into non-overlapping train, validation, and test nodes at a ratio of 8:1:1. Detailed dataset statistical information is shown in Table 1.

Following Zhang et al. (2021); Arous et al. (2023), we adopt the all-rank evaluation protocol, where a ranking list of all possible candidate positions is output and evaluated for each query concept. We employ Macro Mean Rank (MR), Mean Reciprocal Rank (MRR), Recall@k, and Hit@k as metrics for taxonomy completion performance evaluation. Notably, we utilize the original instead of the scaled version of the MRR (Shen et al., 2020).

We first reproduce two representative PLM-based taxonomy expansion and completion techniques and plug TaxoPro into them to verify the effectiveness of our method. TEMP (Liu et al., 2021) leverages the PLM to distinguish taxonomy-paths for structure information capture in taxonomy expansion. Following Xu et al. (2023), we adapt this method to the completion task by attaching child node c to the end of the taxonomy-path. To resolve non-unique paths in DAG taxonomies, we follow Xu et al. (2023) by sorting all root-to-parent paths in ascending order of length and selecting the shortest one, thereby ensuring TEMP’s compatibility with general DAG structures. TacoPrompt (Xu et al., 2023) employs the PLM for triplet semantic matching in taxonomy completion. It only provides definitions for the Food dataset, and we follow Wang et al. (2022a) to obtain more accessible Wikipedia descriptions for all target datasets.

Secondly, we leverage the state-of-the-art taxonomy completion methods as baselines, including TMN (Zhang et al., 2021), TaxoEnrich (Jiang et al., 2022), QEN (Wang et al., 2022a), TaxoComplete (Arous et al., 2023), and CoSTC (Niu et al., 2024). Building on Zhang et al. (2021)’s framework, we reconfigure two established taxonomy expansion methods, TaxoExpan (Shen et al., 2020) and Arborist (Manzoor et al., 2020), as completion task methods. Lastly, we adapt the expansion method, Musubu (Takeoka et al., 2021), that utilizes the PLM as the implicit knowledge base to tackle the low-resource taxonomy expansion problem, to the completion task by averaging the scores of expanding query node q to parent node p and child node c to query node q.

Additionally, we train all original baselines with the additional high-resource taxonomy using the loss defined in Equation 6, while retaining their original models without any LoRA module. These models are termed Baseline+Joint.

Table 2 systematically compares taxonomy completion performance between original baselines and their +Joint variants augmented with cross-domain taxonomic knowledge. Through this experiment, we address the core research question:

Yes. Empirical results demonstrate that integrating cross-domain knowledge considerably boosts baselines’ performance on metrics like MRR and Recall@5, even without specialized algorithms. For instance, TacoPrompt+Joint achieves an absolute improvement of 5.5% over TacoPrompt in the Recall@5 metric. These results reinforce the central motivation behind our method: taxonomies across different domains contain shareable knowledge, which can compensate for data scarcity in low-resource settings. This finding can provide insights to future research, encouraging the exploration of cross-domain knowledge transfer in the taxonomy completion task.

We integrate TaxoPro with two representative PLM-based baselines, TEMP and TacoPrompt, forming their +TaxoPro variants. Table 3 compares these variants with Baseline+Joint, enabling us to explore the key question below.

Yes. By analyzing experimental results, we can draw several key observations. First, PLM-based methods, particularly TEMP and TacoPrompt, outperform others in low-resource scenarios, leading in the Recall@5 metric across all three datasets, as shown in Table 2. This indicates that PLMs can serve as an effective implicit knowledge base (Takeoka et al., 2021) for low-resource taxonomy completion. Second, the effectiveness of pre-trained knowledge varies by domain. For instance, TacoPrompt performs worse on the Equipment dataset than on Science or Food, confirming the limitations of relying solely on pre-trained knowledge for taxonomy completion.

Lastly, PLM-based methods’ +TaxoPro variants consistently surpass their original counterparts. Specifically, TEMP+TaxoPro improves TEMP on MRR/Hit@1/Recall@5 by 0.060/9.1%/5.5%, 0.041/2.6%/8.0%, and 0.048/6.9%/3.8% on the Science, Equipment, and Food datasets, respectively. Similarly, TacoPrompt+TaxoPro achieves gains of 0.079/7.6%/10.7%, 0.061/8.1%/10.4%, and 0.033/6.2%/4.3% on these datasets. Additionally, TEMP+TaxoPro and TacoPrompt+TaxoPro outperform their +Joint variants in MRR and Recall@5 while improving Hit@1, which the +Joint variants decrease. Notably, TacoPrompt+TaxoPro surpasses all Baselines+Joint variants on most metrics. These results underscore TaxoPro’s effectiveness in leveraging cross-domain knowledge to enhance PLM-based taxonomy completion in low-resource scenarios.

As indicated in Table 4, we study the performance of TaxoPro under different settings. Specifically, in the settings w/o CD (cross-domain) and w/o CDKD (cross-domain knowledge decomposition), we utilize vanilla LoRA (Hu et al., 2022a), where only a pair of low-rank matrices, namely, B and A, are injected into the PLM to learn knowledge for taxonomy completion, as outlined in Equation 4. In the w/o CD setting, we use training samples solely from the target domain. In contrast, in the w/o CDKD setting, we use samples from both the target and source domains. Notably, Baseline+TaxoPro w/o CD corresponds to the vanilla LoRA-tuned Baseline, while Baseline+TaxoPro w/o CDKD represents the vanilla LoRA-tuned Baseline+Joint.

TacoPrompt is used as the backbone technique for ablation studies and further discussions since it achieves the most competitive performance. In this context, w/o CD refers to the vanilla LoRA-tuned version of TacoPrompt, while w/o CDKD denotes the vanilla LoRA-tuned version of TacoPrompt+Joint. Guided by the ablation results, we discuss the subsequent questions.

Yes, the results reveal a performance degradation when removing the CD module (w/o CD). For instance, Hit@1/Recall@5 drops 11.9%/14.8%, 8.5%/10.6%, and 9.2%/7.3% on Science, Equipment, and Food datasets, respectively. This further demonstrates that shareable knowledge exists between taxonomies varying from domains and scales, and it helps to complete the target low-resource taxonomies, where such knowledge is inadequately learned from the limited training samples.

It can prevent negative interference between domain-specific and domain-shared knowledge. Comparing the results under the settings w/o CD, w/o CDKD, and w/o $L$_pull,$L$_push, we can make the following two observations. First, a notable performance decline is observed when training data from the target domain is used without knowledge decomposition. This drop is particularly pronounced in the Hit@1 metric. Specifically, the method w/o CDKD that learns from the extra target dataset performs even worse than that w/o CD that learns only from the source dataset. This illustrates that domain-specific knowledge will become noise when applied to a different domain.

Second, the method only w/t the CDKD module (w/o $L$_pull,$L$_push) completes low-resource taxonomies better than the method w/o CD. This improvement is due to the CDKD module’s ability to separate noisy domain-specific knowledge from domain-shareable knowledge. Furthermore, it can be observed that the results of w/o CDKD (vanilla LoRA-tuned TacoPrompt+Joint) are worse than TacoPrompt+Joint, indicating that, in the absence of knowledge decomposition, LoRA-tuning is more strongly affected by noisy domain-specific knowledge compared to full fine-tuning. On the other hand, the method only w/t the CDKD module (w/o $L$_pull,$L$_push) outperforms TacoPrompt+Joint, further demonstrating the CDKD module’s ability to isolate and mitigate the impact of noisy domain-specific knowledge.

Additionally, Figure 4 shows that simply adjusting the domain loss balance hyperparameter α in the w/o CDKD setting never outperforms TaxoPro. This underscores the necessity of the CDKD module when utilizing the external high-resource source taxonomy.

Yes. We find that both auxiliary losses, namely $L$_pull and $L$_push, can individually improve the performance of the method w/o any of them. This demonstrates the effectiveness of both auxiliary losses in controlling the flow of shareable knowledge. More importantly, we observe that the method w/t both losses outperforms the method w/t either of them alone. For example, TacoPrompt+TaxoPro outperforms TacoPrompt+TaxoPro w/o $L$_pull by an average of 2.2% on the Recall@10 metric across three datasets. This observation indicates that the two losses control the flow of shareable knowledge from different perspectives, jointly improving performance.

In this section, we discuss in two main ways: the additional perspective of learned knowledge, and the impact of key hyperparameters on TaxoPro.

After training converges, domain-shared and -specific knowledge are stored in their respective low-rank matrices (LoRA), as described in Equation 5. Firstly, we inject domain-shared and -specific LoRA into the PLM using different combinations during inference. As shown in Table 5, TaxoPro injects domain-shared LoRA and target-specific LoRA; Reversed injects domain-shared LoRA and source-specific LoRA; Only Shared injects only domain-shared LoRA; and Only Specific injects only target-specific LoRA. The subsequent research question is examined using evidence derived from the results.

The domain-shared knowledge contains essential information necessary to complete the target low-resource taxonomy. When relying solely on domain-shared knowledge, it achieves comparable Hit@10 and Recall@10 performance to that obtained by combining both domain-shared and -specific knowledge. Meanwhile, domain-specific knowledge captures fine-grained distinctions related to the domain, which significantly impacts Hit@1 and Recall@1 performance. Additionally, the performance drops significantly in the Reversed setting. For instance, the MRR average drops by 0.180 compared to the original setting on three datasets. This observation highlights that domain-specific knowledge corresponds to fine-grained information that exhibits strong relevance to the specific domain.

Furthermore, we adopt two-stage tuning strategies, where the learned domain-shared knowledge is loaded and frozen, allowing us to focus on learning domain-specific knowledge by different techniques. The results are shown in Table 6. Additionally, we study the effectiveness of the learned domain-shared knowledge on transfer learning. The results are displayed in Table 7, from which we explore the following research questions.

During training, we noticed that either tuning specific LoRA or fully fine-tuning can inherit the knowledge acquired during the initial end-to-end stage. Consequently, this leads to performance comparable to the first stage. However, although these methods enhance Hit@1, they also lead to a decrease in Recall@10, indicating a potential issue of overfitting domain-specific knowledge in the second stage. Additionally, we observed that incorporating the Adapter (Houlsby et al., 2019) in the second stage initially yields poor performance and ultimately leads to a significant drop in performance compared to the first stage. In conclusion, the end-to-end training strategy of TaxoPro proved to be more robust than two-stage tuning strategies.

Firstly, the learned domain-shared knowledge can enhance other tuning techniques for the task. As shown in Table 6, we compare the performance of different tuning techniques w/t (“+ Tech”) or w/o (“Only Tech”) the learned domain-shared knowledge. We find that incorporating domain-shared knowledge consistently enhances the tuning techniques. Specifically, the average Hit@1 increases by 9.8% for Fine-Tuning and 4.0% for the Adapter-Tuning. Secondly, the domain-shared knowledge learned from one target domain dataset can be transferred to another. The transfer learning results shown in Table 7 indicate that all transfer settings outperform the single dataset setting (-CD), but not to a greater extent than TaxoPro. This demonstrates the potential for the efficient migration of learned domain-shared knowledge to another target dataset and validates the effectiveness of TaxoPro in augmenting the target dataset.

In this section, we first calibrate the domain loss balance hyperparameter α on the validation set. Drawing from the results shown in Figure 3, we explore the following question.

This hyperparameter modulates the impact of training samples from different domains on the shared matrices. Optimal performance is achieved at α = 1.0, where equal contributions from both domains enhance the shared matrices’ ability to retain shareable knowledge. When α > 1.0, performance declines as the target domain’s influence becomes too dominant, making the result tend to that of using the target dataset only (w/o CD). Conversely, when α < 1.0, performance slightly drops within a certain range, but significantly deteriorates if the value is too small. This indicates that excessive influence from the source domain hampers the effective filtering of interfering knowledge by loss function of the target domain.

We also examine the impact of the hyperparameter α in methods without knowledge decomposition, specifically the +Joint variants. Using vanilla LoRA-tuned TacoPrompt+Joint (TacoPrompt+TaxoPro w/o CDKD) as an example, we address the following question based on the results in Figure 4.

For variants without knowledge decomposition, as α increases beyond 1.0, the decline in Hit@1 and the improvement in Recall@5/10 brought by using additional high-resource taxonomies diminish, eventually converging to the performance of using only the target low-resource taxonomy (w/o CD). Conversely, when α decreases below 1.0, both metrics decline, ultimately converging to the performance of testing on the low-resource dataset after training with only the high-resource dataset. Similarly, we set α = 1.0 for all +Joint variants, achieving the best overall performance.

Furthermore, we present the sensitive analysis of the auxiliary loss function hyperparameters, λ₁ for $L$_push and λ₂ for $L$_pull, as shown in Figure 5, and analyze the related issue.

TaxoPro demonstrates robustness to λ₁ and λ₂ within a certain range. Excessively large values of λ₂ result in diminished performance, as increasing λ₂ makes TaxoPro increasingly resemble the +Joint setting, where only the shared LoRA module is employed.

Then, we leverage datasets varying from domains and scales as the source domain dataset. Based on the results depicted in Figure 6, we investigate the following question.

Our preliminary analysis suggests two potential characteristics that may influence a taxonomy’s suitability as a source domain. First, larger taxonomies may lead to performance improvements, as indicated by the observed gains from the large-scale MeSH and Verb compared to smaller taxonomies. Second, taxonomies with richer semantics could yield better performance. For instance, MeSH shows slightly better results than Verb, despite both being large-scale, which might be attributed to its richer semantic content. Based on our current findings, we hypothesize that a large-scale taxonomy rich in semantic information could be an ideal candidate for the source domain.

We further study the influence of the rank r of low-rank matrices in our framework. In addition, we replace LoRA with Prompt Tuning (Lester et al., 2021) to investigate the effect of the PEFT technique choice. Based on the results depicted in Figure 7, we discuss the questions below.

Generally, a higher rank yields better results, as evidenced by the positive correlation between the Recall@5 metric and rank size. However, an increase in rank beyond a certain threshold results in a decrease in Hit@1. For instance, H@1 decreases when the rank increases from 32 to 256. This may be due to the insufficient training samples in the target low-resource dataset for domain-specific knowledge learning with high-rank matrices. Therefore, it is essential to choose an appropriate rank within a specific range. Our experiments indicate that a rank of 32 provides an optimal balance across all performance metrics.

In line with previous research (He et al., 2022), LoRA outperforms Prompt Tuning in the task of taxonomy completion when only using the training samples from the target dataset. This pattern also applies to the proposed CDKD module, since LoRA surpasses Prompt Tuning as the knowledge decomposition technique. Hence, LoRA is a suitable PEFT choice for TaxoPro.

In this paper, we propose TaxoPro, a LoRA-based plug-in cross-domain method. It leverages shareable knowledge from the high-resource taxonomy to enhance PLM-based techniques in low-resource taxonomy completion. We decompose cross-domain knowledge into domain-shared and domain-specific parts, storing them with the low-rank matrices to avoid negative interference. Two auxiliary losses direct the flow of shareable knowledge. Experiments prove TaxoPro’s effectiveness. We believe our initial exploration of cross-domain taxonomy completion presents an interesting direction for the community.

Our method currently has two main limitations: (i) it relies on a single source taxonomy to enhance low-resource taxonomy completion, and (ii) training with all samples from a single high-resource taxonomy can be computationally expensive. We plan to extend TaxoPro to support multiple source taxonomies and investigate more efficient sampling techniques to alleviate the computational burden. Additionally, we aim to evaluate the effectiveness of TaxoPro on other tasks that require knowledge transfer.

We sincerely thank the anonymous reviewers for their rigorous and conscientious review, as well as their meticulous and insightful suggestions that greatly improved the quality of this work. We are also deeply grateful to the action editors, Hoifung Poon and Tao Ge, for their exacting editorial oversight, and constructive guidance throughout the review process that significantly strengthened the manuscript. We also thank Yuxun Qu and Yuxiao Liu for their helpful discussions during the research. Their questions and ideas during our meetings helped us clarify key points and solve several challenging problems. Additionally, I extend heartfelt appreciation to my close friend Ziheng Xiao for his unwavering support throughout this research endeavor. This research is supported by the National Natural Science Foundation of China (No. 62372252, 72342017), National Engineering Research Center for Digital Construction and Evaluation Technology of Urban Rail Transit, Development of a platform for quantity statistics and budget preparation of urban rail transit projects based on big data analysis (No. 2022A02158007).