Generative Pre-trained Transformer (GPT)

Generative Pre-trained Transformer (GPT) is a type of neural network architecture that uses unsupervised learning to generate human-like language. The GPT model was introduced in a 2018 paper by OpenAI titled Improving Language Understanding by Generative Pre-Training.

Models that use GPT architecture are pre-trained on a enormous amounts of text data, such as Wikipedia and countless other websites. During training, the model learns to predict missing words in a sentence based on the context of the surrounding words. Once a model is trained, it can be fine-tuned on specific tasks such as language translation, question answering, and text classification. This allows the model to adapt its language generation capabilities to the specific requirements of various downstream tasks.

Deeper Knowledge on Generative Pre-trained Transformer (GPT)

ChatGPT: Introduction and Examples

An overview of ChatGPT and common use cases

Broader Topics Related to Generative Pre-trained Transformer (GPT)

Unsupervised Machine Learning

Machine learning that uses unlabeled input as training data