What is a large language model like ChatGPT?

Large language models are built on a neural network design called the transformer, introduced by Google researchers in a 2017 paper titled Attention Is All You Need. During training, the model reads enormous collections of text and repeatedly learns to predict the next word, adjusting billions of internal numbers called parameters. After this pre-training, models are fine-tuned, often with human feedback, to follow instructions and give helpful answers. OpenAI released ChatGPT in November 2022, and it reached an estimated 100 million users in about two months. These models can produce fluent text but can also state false information confidently, an issue often called hallucination.