Design of the high-order sequence prediction task. (A). Structure of high-order sequences with shared sub-sequences. (B). High-order sequences with multiple possible endings. (C). Stream of sequences with noise between sequences. Both learning and testing occur continuously. After the model learned one set of sequences, we switched to a new set of sequences with contradictory endings to test the adaptation to changes in the data stream.