Accurate causal inference among time series helps to better understand the interactive scheme behind the temporal variables. For time series analysis, an unavoidable issue is the existence of time lag among different temporal variables. That is, past evidence would take some time to cause a future effect instead of an immediate response. To model this process, existing approaches commonly adopt a prefixed time window to define the lag. However, in many real-world applications, this parameter may vary among different time series, and it is hard to be predefined with a fixed value. In this letter, we propose to learn the causal relations as well as the lag among different time series simultaneously from data. Specifically, we develop a probabilistic decomposed slab-and-spike (DSS) model to perform the inference by applying a pair of decomposed spike-and-slab variables for the model coefficients, where the first variable is used to estimate the causal relationship and the second one captures the lag information among different temporal variables. For parameter inference, we propose an efficient expectation propagation (EP) algorithm to solve the DSS model. Experimental results conducted on both synthetic and real-world problems demonstrate the effectiveness of the proposed method. The revealed time lag can be well validated by the domain knowledge within the real-world applications.