Evaluation on ChatGPT for Chinese Language Understanding

Large Language Models (LLMs) are one of the most indispensable techniques in natural language processing research in recent years, and a lot of research work has been generated in academia and industry around LLMs: GPT-3 [1], LaMDA [2], Ernie 3.0 titan [3], PANGU-α [4], WeLM [5] and GLM-130B. According to Kiela et al. [7], LLMs have reached or even exceeded the human average in multiple tasks such as language understanding, dialogue generation, etc. Most studies assess the capabilities of language models by their performance on a range of specific datasets. These studies are still in the initial stage of whether the language model can correctly respond to multiple instructions, meet reasonable requirements, and whether the generated content is aligned with users.

ChatGPT^① has demonstrated impressive language ability in the process of interacting with humans by tracking the progress of dialogue and answering questions, admitting mistakes, and correcting and rejecting unreasonable requests. There have been some studies on the impact of ChatGPT on academia and education industries, and it is generally believed that ChatGPT can improve work efficiency in a variety of fields, but it needs to be used wisely [8]. On the impact of LLMs on scientific research, van Dis et al believe that AI has some benefits, but that academics need to establish more explicit accountability for using AI [9]. In the scientific research process, researchers need to be more open and transparent about the usage of AI, and Thorp [10] takes a negative view on questions such as whether LLMs can be authors of papers or whether artificial intelligence can be copyrighted.

The previous research work on performance testing of ChatGPT and other GPT-3.5 series models has focused on zero-shot and few-shot using public datasets, and comparing the evaluation results with supervised fine-tuning models, and previous studies have shown that ChatGPT performs extremely well on many English datasets, and ChatGPT is even comparable to supervised fine-tuning models on some evaluation metrics [11]. In the fields of machine translation, sentiment analysis, and fake information detection, ChatGPT surpassed SOTA performance at zero-shot but performed poorly on open-domain knowledge question answering and task-based dialogue tasks [12], machine translation performance in low-resource languages is also weaker than existing translation products [14]. However, there are currently few comprehensive evaluation of ChatGPT on Chinese language understanding tasks.

In this paper, we first give a brief introduction to the GPT family models and related research with large language models. Then we use several public datasets to evaluate the Chinese language performance of ChatGPT, compare it with the models developed in China, and study its specific performance in Chinese. We found that despite ChatGPT has impressive performance on classic NLP tasks, it still faces the challenge of fabricating facts due to hallucination. In addition, on two more difficult Chinese understanding tasks, we show that the performance of ChatGPT can be improved by chain-of-thought prompts^②.

We analyze its impact and potential risks based on the performance evaluation results. Finally, we introduce our ChatBIT, a knowledge-augmented dialogue system that uses citation to enhance the credibility of generated content and mitigate the impact of hallucination.

In summary, our main contributions include:

• We evaluate ChatGPT with zero-shot prompting on sentiment analysis, summarization, reading comprehension, closed-book question answering, cants understanding, and idiom filling in the blank. We also improved ChatGPT's answer accuracy on the last two tasks by using simple chain of thought prompts.

• We analyze the potential risks of ChatGPT from three aspects: hallucination, ethics, and privacy security.

• We briefly present research progress on ChatBIT, our more trusted chat system.

Language model is one of the research topics in natural language processing, BERT [15]. GPT [1] and other models are widely influential innovative research work in LLMs. ChatGPT is a widely influential and groundbreaking product based on GPT-3.5. This section will introduce the main technological innovations of ChatGPT.

Generative pre-trained models. Generative Pre-trained Transformer (GPT) is an autoregressive pre-trained model that inputs a sequence of tokens of the current text and predicts the next token. Generative models such as RNN [18], LSTM [19] and Transformer [20] are widely used in deep learning. The basis of the GPT series of models is the decoder of Transformer. In each layer of the decoder, the multi-head self-attention can focus on the different aspects of the sentence, and with the increase of network layers, more advanced and abstract semantic information can be extracted, and the model becomes more capable of understanding textual features. In addition to solid semantic extraction capabilities, multiple language models based on Transformer can also learn from unlabeled data and have better transfer learning capabilities. GPT-1 [16] model follows the decoder structure of Transformer. In GPT-2 [17], layer normalization is placed before each submodule, and layer normalization is added after the last module.

In-context learning. When LLMs are self-supervised on the large-scale corpus, surprising pattern recognition and reasoning capabilities emerge, and in-context learning is one of the capabilities found in GPT-3 [1]. With In-context learning, there is no need for gradient backpropagation but a small number of labeled samples and prompts, and then input into the model to guide the inference prediction of GPT-3, which means that LLMs can automatically identify the type of task during decoding and make inferences based on tasks and samples. Noting that GPT-3 has weak code generation capabilities, OpenAI further designed Codex [21], Which is a fine-tuned GPT model on the GitHub public available code. The results of Codex indicate that GPT models can generate computer program code.

Instruction tuning. To make the large language model have better task recognition ability in the case of zero-shot, instruction tuning describes a variety of tasks in natural language and then inputs the task description and training samples into the model for fine-tuning [22]. With instruction tuning, language models can generalize their natural language understanding ability to the recognition and understanding of tasks so that they can correctly recognize new tasks in zero-shot inference.

Chain-of-Thought. Chain-of-Thought (CoT) refers to a series of intermediate steps including the reasoning process. In large-scale language models, CoT prompts can be introduced into the context to improve LLM's performance in arithmetic, common sense, and symbolic reasoning [37]. CoT prompts can also be automatically generated by LLMs, such as generating code to solve problems [39] based on majority voting [38], etc. A large amount of high-quality labeled data can be obtained through CoT prompts, and the performance of the language model can be further improved [40].

Reinforcement learning of human feedback. The results of DialoGPT [23], an open-domain dialogue model based on GPT-2, show that it is necessary to use reinforcement learning to prevent the model from producing abnormal responses. To obtain higher quality output, WebGPT [24] uses human feedback to optimize the quality of answers generated by the model and obtain results that are more aligned with human preferences through imitation learning and reinforcement learning. Generative language models need to control the generated content to avoid generating controversial or incorrect text. InstructGPT [25] applies reinforcement learning from human feedback (RLHF) to control the generated content of GPT. Firstly supervised fine-tuning is performed on GPT-3, and then the reward model is trained by ranking the output quality of the model by human annotators [26], and Finally, the supervised fine-tuning model is trained using the PPO [27] algorithm to maximize reward. Through this training method, the model can output text more aligned with humans, avoid sending toxic speech and reduce the likelihood of responding to unreasonable requests.

GPT-3.5 is trained for computer language code generation and instruction fine-tuning, Chat GPT is a language model obtained after supervised instruction fine-tuning and RLHF on GPT-3.5. This makes the text generated by ChatGPT more in line with human language habits.

Existing evaluations show that ChatGPT has achieved good results in the fields of machine translation [14], natural language understanding, and sentiment analysis [12]. To quantify the performance of ChatGPT in Chinese and compare it with large-scale language models developed in China, this section uses publicly available Chinese datasets to test and evaluate ChatGPT. We mainly test ChatGPT on zero-shot learning in sentiment analysis, automatic summary, reading comprehension, and closed-book question and answer, and in the idiom filling in the blank and cants understanding task, we conducted additional chain of thought reasoning tests. Note that in the evaluation results, except for ChatGPT, for other models we directly use the published results, and although the prompts used by these models are different, we assume that these publicly available results reflect the best performance of these baseline models. In addition, the system message and hyperparameters used in our experiments apply the default settings in OpenAI API.

Sentiment analysis is a classic NLP task, the model needs to determine the categories of sentiment in the input context, this paper uses the test set of ChnSentiCorp^③ and EPRSTMT [28], Each of these samples is labeled as two emotional polarities: positive or negative, and the results are shown in Table 1.

ChatGPT demonstrated competitive performance in sentiment analysis tasks. We observed a bias in ChatGPT's understanding of context, for example, we used a prompt to indicate that the input text contains only positive and negative sentiment, but “neutral” still appears in the output text of ChatGPT. In Addition, there are some special samples in ChnSentiCorp in which each sentence has a different sentiment tendency, in this case, the advantage we find in a generative model like ChatGPT over a discriminant model is that ChatGPT can automatically analyze the sentiment sentence by sentence and return the result, as shown in Table 2.

The text automatic summary task requires the language model to perform inference on the input long text to output concise and accurate summary text. The text summarization task is divided into extractive abstracts and generative abstracts, and the experiments in this paper mainly focus on generative abstracts and use the test set of LCSTS [29] and TTNews [30]. Considering that some samples in TTNews are too long to cause ChatGPT to return results, we randomly sampled 1k samples with less than 1k Chinese characters. the evaluation index is ROUGE-1 [35].

For a summary and headline generation of news texts, ChatGPT achieves excellent performance, but we noticed that in the LCSTS dataset if the prompt is changed to “生成30字以内的摘要:” (“generate summarization under 30 words:”), the average length of the output summary reaches 70 words, which is far beyond the limit of “less than 30 words” in the prompt. So, we changed the prompt to “Generate headlines for the following news:”, and the text generated by ChatGPT is closer to the number of words of ground-truth. The results are shown in Table 3, and Table 4 includes examples of input and output samples.

Machine Reading Comprehension (MRC) is one of the mainstream tasks for assessing the comprehension and reasoning ability of LLMs. Extractive MRC, that is, for a given context and a question, language model needs to identify a subsequence that can answer the question from the context as output and evaluates the performance of the model by comparing the matching score of the output sequence with ground-truth. Considering that ChatGPT is a generative language model, this paper converts the extraction task into a generation task and tests the performance of ChatGPT in zero-shot learning by constructing the same prompt template and hyperparameter settings as in PANGU-α [4]. The experiment was performed using the validation set of CMRC2018, which is in Chinese Simplified and DRCD [32] of Chinese Traditional. The evaluation metrics are EM (Exact Match) and F1.

Common sense MRC means that the answer to the question is not necessarily given explicitly in the context, the model needs to conduct common sense reasoning through the context and select the most suitable one as the output among the many candidate answers provided. In this paper, the validation set of C³ [33] is used for evaluation, and the metric is accuracy. Test Results are in Table 5.

The evaluated results of reading comprehension show that the Ernie 3.0 titan with a larger parameter scale of 260B has a greater advantage than PANGU-α with a 13B parameter and WeLM with a 10B parameter, confirming that larger general models generally perform better for generative tasks. Locally, ChatGPT is comparable to Ernie 3.0 Titan in terms of reading comprehension in both simplified and traditional Chinese, and overall Ernie 3.0 Titan has more advantages. The reason why ChatGPT's EM score is low and F1 is higher is presumably due to its training method. EM requires two texts to match exactly at the character level to score. ChatGPT is trained by RLHF to generate question-and-answer text with more consideration of human users’ feelings, that is, to generate sentences that make the semantics more coherent rather than giving answers directly. Table 6 is an intuitive example. Ground-truth is a segment extracted directly from the context and may not form complete sentences grammatically and semantically. For long input context, the table shows only the beginning and omits the rest of it.

Closed-book question-answering tests do not give context for the answer in the input, and the language model needs to answer based on the knowledge and common sense learned in the pre-training data. The experiments in this paper selected the test dataset of WebQA [34] and CKBQA^④. The evaluated metrics are EM/F1 or accuracy, and the results are shown in Table 7. Closed-book question answering requires the model to have a certain reserve of Chinese knowledge and common sense, and ChatGPT is more prone to factual errors in this task.

We further analyzed the output text of ChatGPT on the WebQA test set and found that only 56.96% of the correct answers labeled were contained in its output text, which means that in about half of the cases, ChatGPT will make mistakes, tables 8 is an example of a closed-book question answer. We speculate that the possible reasons may be that the training corpus of the Chinese used is small in scale, or that the Chinese training corpus is of poor quality.

To understand cants on the Chinese Internet, one usually requires to have a certain understanding of Chinese homonyms, font shapes, and common sense. The DogWhistle dataset [41] comes from the adapted online puzzle game Decrypto. In this game, four players are divided into two groups. Each group of players needs to try to understand the code words sent by teammates and crack the code words of the enemy. We tested ChatGPT's ability to associate and interpret code words using a validation set of its Insider subset.

The task of idiom comprehension and blank filling requires a language model to fill in the appropriate idioms into the blanks of a given sentence based on the meaning of a set of candidate idioms. This task requires the model to have a fuller understanding of the meaning of Chinese idioms. Experiments are evaluated using the validation set of the Chinese idiom reading comprehension fill-in-the-blank dataset ChID [28].

Although CoT prompts are usually used in natural language tasks such as arithmetic and reasoning [37], our experiments show that CoT prompts can be used to improve the ability of ChatGPT in cants and idioms understanding. Specifically, we first ask ChatGPT to explain the meaning of each cant or idiom through prompts, and then let it choose or fill in the blanks according to the previous explanation.

Our experimental results are shown in Table 9, and the evaluation metric is accuracy.

The experimental results show that ChatGPT will get higher accuracy using CoT prompts on these two difficult tasks, but it is still not as good as the supervised fine-tuning model. The following are input examples for evaluation, the first round of dialogue for each example is used to generate the inference process of CoT, while the zero-shot group without CoT directly takes the second round of dialogue as input.

In cants understanding, we first prompt ChatGPT to explain the meaning of each cant in a candidate, and then prompt to guess the meaning of the code word based on the context and the code word itself. In the task of filling in the idioms, we first prompt ChatGPT to explain the meaning of each idiom, and then fill in the blanks, and simulate the thinking process of ordinary people through the CoT prompts of these two rounds of dialogue. In both tasks, CoT prompts allow ChatGPT to be more accurate than directly giving the answers.

For idiom filling in the blanks, explaining the meaning of idioms is more important for the model to understand the task of filling in the blanks, because the meaning of idioms is highly semantically related to the following sentences, and the range of associations required for cants understanding is vaguer, since in the first round of dialogue, ChatGPT is prompted associating the meaning of the relevant phrase first is not necessarily related to the code word.

In the experiment, we noticed that the choice of CoT prompts is very important for ChatGPT to generate high-quality replies. For example, in the idiom fill-in-the-blank task, if we replace “explain the meaning of each idiom separately” with “make a sentence for each idiom separately”, then it will cause the accuracy rate to drop to 45.0%. The following is an example of using different CoT prompts under the same sample as Table 11.

The hallucination of generative models can be observed in this example, where the model generates fluent but meaningless text [42]. During training, the model memorizes some frequently occurring segments in the training data, so that similar text may be sampled with higher probability during inference. However, the generative language models built based on neural networks are hard to avoid hallucinations. A similar example of generating hallucinations dominates the entire data. ChatGPT finally uses “深度估计“ to fill “#idiom#” in the input, but this phrase does not appear in the idiom candidate set, and this phrase is not even an idiom.

In the performance test, it was observed that ChatGPT frequently commits errors on Chinese knowledge and common-sense Q&A, and it uses fluent sentences to illustrate fabricated false facts, for groups with limited sources of information have a certain degree of confusion. This section aims to analyze the potential usage risks of ChatGPT based on the previous performance tests.

Risk of Hallucination. The issue of hallucination in generative language models is currently a critical concern, as it undermines the credibility of the content produced by such models. Unfaithful or nonsensical hallucination text may mislead users, and lead to community concerns about the usage of language models. ChatGPT, as a generative language model, may not be able to correctly answer academic questions and some common-sense questions, which means that the general language model cannot replace the dedicated model in the professional fields for the time being. In addition, because the content generated by LLMs such as ChatGPT may contain factual errors, thus, improper use in the scientific process may distort scientific facts and spread misinformation.

Ethical Risks. The development and application of LLMs will be inevitably accompanied by some ethical issues. On the one hand, the content generated by the LLMs may lead to the creator inadvertently plagiarizing the work of others, resulting in the risk of infringement of intellectual property rights, on the other hand, the reasoning ability and creative ability of LLMs may be used to help illegal acts such as cheating in the exams. It is necessary to restrict the use of LLMs more strictly in education, this is due to the information contained in LLMs is difficult to be quantitatively analyzed, and the impact of the generated content on students’ values is uncontrollable. Additionally, because students can easily get answers from LLMs, they may be inclined to use LLMs to complete homework and cheat on exams, which will affect students’ ability to think independently.

Risks of Privacy Security. Users need to upload the content of the conversation to the server to obtain the content generated by LLMs. These conversations may contain information such as the user's personality, preferences, and even identity. Users may inadvertently disclose this information to service providers while working with LLMs. Therefore, how to balance the use of LLMs and the protection of personal information may be a problem that AGI will face in the future.

To address the challenges of LLMs and ensure trusted and controllable generated content, we are developing ChatBIT with a focus on knowledge extraction, reasoning, and reliable citation to the content. The objective is to achieve believable, instant, and transparent LLMs. The model supports domain knowledge question answering, multi-round dialogue, information retrieval, and other functions. At the same time, multiple languages such as Chinese, English, and Japanese are supported, and different languages can be supported according to requirements. Currently, the system has provided web and API services and supports multiple users.

The entire development process of ChatBIT, including data preparation, training, and reasoning is secure and dependable, In addition, we use mixed precision training [43], model quantification and pruning to achieve computational acceleration and reduce the time complexity of training and reasoning. The domain data and instructions we utilize for fine-tuning are of high quality, including the relevant information of the Beijing Institute of Technology and the Chinese knowledge encyclopedia. Fully autonomous and controllable code is utilized for model construction and training, while a controllable scheme is employed for the external service. Table 13 is an example of a trusted citation of ChatBIT.

This paper evaluates ChatGPT on 11 Chinese datasets. Our findings indicate that ChatGPT has notable performance on the classical natural language processing tasks of sentiment analysis, automatic summarization, and reading comprehension and the accuracy of its answer can be enhanced by CoT style prompts, but it is prone to factual errors in closed-book knowledge question answering. Based on the evaluation results, this paper further analyzes ChatGPT on the possible risks and challenges. Finally, this paper present ChatBIT, our dependable dialogue language model. We hope this evaluation experiment provides some inspiration for designing trustworthy and controllable language models.

The data sets generated during and/or analyzed during the current study are available in the CLUE, FewCLUE, CKBQA, dogwhistle, ChnSentiCorp, et al repository,

https://github.com/pengming617/bert_classification

https://github.com/CLUEbenchmark/FewCLUE

http://tcci.ccf.org.cn/conference/2017/taskdata.php

https://github.com/CLUEbenchmark/CLUE

https://github.com/pkumod/CKBQA,

https://github.com/JetRunner/dogwhistle.

Li Linhan: designed the research framework, designed the research framework and wrote the manuscript. Zhang Huaping: proposed the research problems. Li Chunjin: wrote and revised the manuscript. You Haowen: revised the manuscript. Cui Wenyao: collected and analyzed the data and revised the manuscript.