This letter focuses on the problem of lifelong classification in the open world, the goal of which is to achieve an endless process of learning. However, incremental data sets (like the streaming data) in the open world, where the new classes may be emerging, are unsuited for classical classification methods. For addressing this problem, existing methods usually retrain the whole observed data sets with the complex computation and the expensive storage cost. This letter attempts to improve the performance of classification in the open world and decomposes the problem into three subproblems: (1) to reject unknown instances, (2) to classify accepted instances, and (3) to cut the cost of learning. Rejecting unknown instances refers to recognize those instances whose classes are unknown according to the learner, which could reduce the computation of the retraining process and eliminate the storage of historical data sets. We employ outlier detection for rejecting instances and a variant artificial neural network for classifying with fewer weights. Results on several experiments show that the work is effective. Source code can be found at https://github.com/wangbi1988/Lifelong-learning-in-Open-World-Classification.