For any discrete-state sequence prediction algorithm A , it is always possible, using an algorithm B no more complicated than A , to generate a sequence for which A 's prediction is always wrong. For any prediction algorithm A and sequence x , there exists a sequence y no more complicated than x , such that if A performs better than random on x , then it will perform worse than random on y by the same margin. An example of a simple neural network predicting a bit sequence is used to illustrate this very general but not widely recognized phenomenon. This implies that any predictor with good performance must rely on some (usually implicitly) assumed prior distributions of the problem.

It is known theoretically that an algorithm cannot be good for an arbitrary prior. We show that in practical terms this also applies to the technique of “cross-validation,” which has been widely regarded as defying this general rule. Numerical examples are analyzed in detail. Their implications to researches on learning algorithms are discussed.