Build an Orchestrator in Go (From Scratch)

Book excerpt: Develop a deep understanding of Kubernetes and other orchestration systems by building your own with Go and the Docker API. Orchestration systems like Kubernetes can seem like a black box: you deploy to the cloud and it magically handles everything you need. That might seem perfect—until something goes wrong and you don’t know how to find and fix your problems. Build an Orchestrator in Go (From Scratch) reveals the inner workings of orchestration frameworks by guiding you through creating your own. In Build an Orchestrator in Go (From Scratch) you will learn how to: Identify the components that make up any orchestration system Schedule containers on to worker nodes Start and stop containers using the Docker API Manage a cluster of worker nodes using a simple API Work with algorithms pioneered by Google’s Borg Demystify orchestration systems like Kubernetes and Nomad Build an Orchestrator in Go (From Scratch) explains each stage of creating an orchestrator with diagrams, step-by-step instructions, and detailed Go code samples. Don’t worry if you’re not a Go expert. The book’s code is optimized for simplicity and readability, and its key concepts are easy to implement in any language. You’ll learn the foundational principles of these frameworks, and even how to manage your orchestrator with a command line interface. About the technology Orchestration frameworks like Kubernetes and Nomad radically simplify managing containerized applications. Building an orchestrator from the ground up gives you deep insight into deploying and scaling containers, clusters, pods, and other components of modern distributed systems. This book guides you step by step as you create your own orchestrator—from scratch. About the book Build an Orchestrator in Go (From Scratch) gives you an inside-out perspective on orchestration frameworks and the low-level operation of distributed containerized applications. It takes you on a fascinating journey building a simple-but-useful orchestrator using the Docker API and Go SDK. As you go, you’ll get a guru-level understanding of Kubernetes, along with a pattern you can follow when you need to create your own custom orchestration solutions. What's inside Schedule containers on worker nodes Start and stop containers using the Docker API Manage a cluster of worker nodes using a simple API Work with algorithms pioneered by Google’s Borg About the reader For software engineers, operations professionals, and SREs. This book’s simple Go code is accessible to all programmers. About the author Tim Boring has 20+ years of experience in software engineering. For most of that time he has worked with orchestration systems, including Borg, Kubernetes, and Nomad. Table of Contents PART 1 INTRODUCTION 1 What is an orchestrator? 2 From mental model to skeleton code 3 Hanging some flesh on the task skeleton PART 2 WORKER 4 Workers of the Cube, unite! 5 An API for the worker 6 Metrics PART 3 MANAGER 7 The manager enters the room 8 An API for the manager 9 What could possibly go wrong? PART 4 REFACTORINGS 10 Implementing a more sophisticated scheduler 11 Implementing persistent storage for tasks PART 5 CLI 12 Building a command-line interface 13 Now what?