How does ChatGPT work?

ChatGPT is built on a type of neural network called a transformer, introduced by Google researchers in the 2017 paper Attention Is All You Need. During training, the model reads enormous amounts of text and learns to predict the next token, a chunk of text about three-quarters of a word long, adjusting billions of internal settings called parameters. It is then fine-tuned using a method called reinforcement learning from human feedback, where people rate its answers so it learns to be more helpful and safe. When you type a question, it generates a reply one token at a time, choosing likely words based on patterns rather than looking facts up in a database. That is why it can write fluently but sometimes produces confident mistakes, known as hallucinations.