What does a language model predict?

A language model primarily predicts the likelihood of a sequence of words, which is fundamental to its functioning. It analyzes patterns in the training data to determine the probability that a given word will follow a sequence of preceding words. This probabilistic approach allows the model to generate coherent text and to understand context, thereby enabling tasks such as text completion, translation, and summarization.

In the context of natural language processing, understanding and predicting sequences of words helps in various applications, such as speech recognition and chatbots, where the model needs to generate or recognize sentences that make sense logically and contextually. This predictive capability is what distinguishes language models, emphasizing their role in managing language through probability rather than defining word meanings, checking grammatical correctness, or analyzing sentiment, which belong to different tasks in the field of NLP.