Hands-On Generative AI with Transformers and Diffusion Models

Book excerpt: Learn how to use generative media techniques with AI to create novel images or music in this practical, hands-on guide. Data scientists and software engineers will understand how state-of-the-art generative models work, how to fine-tune and adapt them to your needs, and how to combine existing building blocks to create new models and creative applications in different domains. This book introduces theoretical concepts in an intuitive way, with extensive code samples and illustrations that you can run on services such as Google Colaboratory, Kaggle, or Hugging Face Spaces with minimal setup. You'll learn how to use open source libraries such as Transformers and Diffusers, conduct code exploration, and study several existing projects to help guide your work. Learn the fundamentals of classic and modern generative AI techniques Build and customize models that can generate text, images, and sound Explore trade-offs between training from scratch and using large, pretrained models Create models that can modify images by transferring the style of other images Tweak and bend transformers and diffusion models for creative purposes Train a model that can write text based on your style Deploy models as interactive demos or services