Justification is an explanation that supports the veracity assigned to a claim in fact-checking. However, the task of justification generation has been previously oversimplified as summarization of a fact-check article authored by fact-checkers. Therefore, we propose a realistic approach to generate justification based on retrieved evidence. We present a new benchmark dataset called ExClaim (for Ex plainable fact-checking of real-world Claim s), and introduce JustiLM, a novel few-shot Justi fication generation based on retrieval-augmented L anguage M odel by using fact-check articles as an auxiliary resource during training only. Experiments show that JustiLM achieves promising performance in justification generation compared to strong baselines, and can also enhance veracity classification with a straightforward extension. 1 Code and dataset are released at https://github.com/znhy1024/JustiLM .