Before neural networks, language models were tables of observed continuations like this one. The generation loop is unchanged in modern systems; what changed is the model in the middle, since a neural network can share information between similar contexts and generalize to contexts it has never seen.

The dead end restarts statistic counts contexts that appeared too rarely in the training text to continue from, which becomes more common at higher context lengths and illustrates why larger context windows require more training data.