A generative model learns the probability distribution of training data, enabling it to create new samples that resemble the training examples without copying them directly. This contrasts with discriminative models that learn boundaries between classes without modeling the data distribution itself. Given training images of cats, a generative model learns the statistical structure of cat images, textures, shapes, colors, spatial relationships, and can sample new cat images from this learned distribution. The quality of generated samples reflects how well the model captured the true data distribution. Generative models come in several families: autoregressive models like GPT generate sequences one element at a time, each conditioned on previous elements; variational autoencoders learn probabilistic latent spaces and generate by sampling and decoding; GANs use adversarial training between generator and discriminator; diffusion models learn to reverse a noising process; and energy-based models define an energy function over data. Each approach has different tradeoffs in sample quality, diversity, training stability, and generation speed. Generative AI has transformed creative industries: text generation powers chatbots and writing assistants, image generation creates art and product mockups, audio generation produces music and voice synthesis. The ethical questions are real as generated content becomes indistinguishable from human-created content.
Back to Glossary