Skip to Main Content
Table 1: 

Overview of existing work on EBHD of NLP models. We use abbreviations as follows: Task: TC = Text Classification (single input), VQA = Visual Question Answering, TQA = Table Question Answering, NLI = Natural Language Inference / Model: NB = Naive Bayes, SVM = Support Vector Machines, LR = Logistic Regression, TellQA = Telling QA, NeOp = Neural Operator, CNN = Convolutional Neural Networks, BERT* = BERT and RoBERTa / Bug sources: AR = Natural artifacts, SS = Small training subset, WL = Wrong label injection, OD = Out-of-distribution tests / Exp. scope: G = Global explanations, L = Local explanations / Exp. method: SE = Self-explaining, PH = Post-hoc method / Feedback (form): LB = Label, WO = Word(s), WS = Word(s) Score, ES = Example Score, FE = Feature, RU = Rule, AT = Attention, RE = Reasoning / Update: M = Adjust the model parameters, D = Improve the training data, T = Influence the training process / Setting: SP = Selected participants, CS = Crowdsourced participants, SM = Simulation, NR = Not reported.

PaperContextWorkflowSetting
TaskModelBug sourcesExp. scopeExp. methodFeedbackUpdate
Kulesza et al. (2009) TC NB AR G,L SE LB,WS M,D SP 
Stumpf et al. (2009) TC NB SS SE WO SP 
Kulesza et al. (2010) TC NB SS G,L SE WO,LB M,D SP 
Kulesza et al. (2015) TC NB AR G,L SE WO,WS SP 
Ribeiro et al. (2016) TC SVM AR PH WO CS 
Koh and Liang (2017) TC LR WL PH LB SM 
Ribeiro et al. (2018b) VQA TellQA AR PH RU SP 
TC fastText AR,OD 
Teso and Kersting (2019) TC LR AR PH WO SM 
Cho et al. (2019) TQA NeOp AR SE AT NR 
Khanna et al. (2019) TC LR WL PH LB SM 
Lertvittayakumjorn et al. (2020) TC CNN AR,SS,OD PH FE CS 
Smith-Renner et al. (2020) TC NB AR,SS SE LB,WO M,D CS 
Han and Ghosh (2020) TC LR WL PH LB SM 
Yao et al. (2021) TC BERT* AR,OD PH RE D,T SP 
Zylberajch et al. (2021) NLI BERT AR PH ES SP 
PaperContextWorkflowSetting
TaskModelBug sourcesExp. scopeExp. methodFeedbackUpdate
Kulesza et al. (2009) TC NB AR G,L SE LB,WS M,D SP 
Stumpf et al. (2009) TC NB SS SE WO SP 
Kulesza et al. (2010) TC NB SS G,L SE WO,LB M,D SP 
Kulesza et al. (2015) TC NB AR G,L SE WO,WS SP 
Ribeiro et al. (2016) TC SVM AR PH WO CS 
Koh and Liang (2017) TC LR WL PH LB SM 
Ribeiro et al. (2018b) VQA TellQA AR PH RU SP 
TC fastText AR,OD 
Teso and Kersting (2019) TC LR AR PH WO SM 
Cho et al. (2019) TQA NeOp AR SE AT NR 
Khanna et al. (2019) TC LR WL PH LB SM 
Lertvittayakumjorn et al. (2020) TC CNN AR,SS,OD PH FE CS 
Smith-Renner et al. (2020) TC NB AR,SS SE LB,WO M,D CS 
Han and Ghosh (2020) TC LR WL PH LB SM 
Yao et al. (2021) TC BERT* AR,OD PH RE D,T SP 
Zylberajch et al. (2021) NLI BERT AR PH ES SP 
Close Modal

or Create an Account

Close Modal
Close Modal