In the fast-paced world of software development, automation is not just a luxury—it’s a necessity. Jenkins, one of the most widely used open-source automation servers, empowers teams to automate the building, testing, and deployment of their applications.
As your CI/CD pipelines grow in complexity and your build workloads increase, running all your jobs on a single machine becomes inefficient. That’s where the Jenkins Master and Slave (now referred to as "controller and agent") setup comes into play.
This blog takes you through an extensive journey into Jenkins' master-slave architecture—explaining what it is, why you need it, how it works, and step-by-step instructions to implement it effectively.
What is Jenkins?
Jenkins is a free and open-source automation server that helps developers build, test, and deploy software reliably. It supports continuous integration (CI) and continuous delivery (CD), making it a critical component in modern DevOps pipelines.
- License: MIT (Massachusetts Institute of Technology,United States)
- Language: Java
- Web Interface: Built-in with 1000+ plugins
- Extensible: Plugin architecture for integration with tools like Git, Docker, Maven, Kubernetes, and more.
What is Jenkins Master-Slave Architecture?
In Jenkins, the master-slave (or controller-agent) architecture is a way to distribute the workload of building and testing projects across multiple machines.
- Master (Controller): Orchestrates build jobs and assigns tasks.
- Slave (Agent): Executes the assigned tasks and sends results back to the master.
This image illustrates the Jenkins Master-Slave Architecture with a Git Repository, as used for Continuous Integration (CI) in software development.
Here’s the concept step-by-step:
1. Code Commit & Continuous Integration Trigger
- Developers push their code changes to a Git repository (shown on the left with a database-like icon).
- Once code is committed, a webhook or polling mechanism in Jenkins detects the change.
- This triggers the Continuous Integration process in Jenkins.
2. Jenkins Master Server
- The Jenkins Server (Master) is responsible for:
- Receiving build triggers from the Git repository.
- Managing the overall pipeline configuration.
- Distributing tasks to Jenkins Slaves for execution.
- Collecting results and generating reports.
- It communicates with Jenkins Slaves via TCP/IP.
3. Jenkins Slave Nodes
- The Slaves are worker nodes that execute the actual build, test, and deployment steps.
- Each slave can run on a different operating system for cross-platform testing:
- Windows
- Linux
- MacOS
- Jenkins assigns jobs to slaves based on labels, availability, and job requirements.
4. Test Execution & Reporting
- Slaves run the builds/tests and send the test reports back to the Jenkins Master.
- The Jenkins Master aggregates these results and displays them in the Jenkins dashboard.
- This ensures that the build status and test outcomes are visible to the development team.
Key Advantages of This Architecture
- Scalability: Multiple slaves handle multiple builds in parallel.
- Cross-Platform Testing: Ensures code works across different OS environments.
- Load Distribution: Master delegates heavy workloads to slaves.
- Centralized Management: One master node manages pipelines and resources.
Element and Roles :
Why Use Master-Slave Setup?
Jenkins Master Responsibilities
- UI rendering and configuration
- Scheduling jobs
- Assigning jobs to slaves
- Managing job configurations
- Handling plugin updates
- Storing build logs and artifacts
Jenkins Slave Responsibilities
- Receive and execute build tasks from the master
- Run jobs in specific environments
- Report job statuses back to the master
- Can be configured to run specific jobs (label-based assignment)
Communication Between Master and Slave
There are two main modes of communication:
- SSH : Master connects via SSH to a Unix-based slave
- JNLP : Slave initiates connection to the master, useful for firewall-protected environments
Types of Jenkins Agents
- Static Agents: Always available for builds.
- Dynamic Agents: Created and destroyed as needed (e.g., Kubernetes pods).
- Cloud-based Agents: Hosted on AWS, Azure, GCP, or Docker containers.
- Permanent Agents: Manually configured and maintained.
- Ephemeral Agents: Spawned per build job.
Prerequisites for Setup
- Java installed on both master and slave nodes
- SSH access or firewall rules for JNLP
- Jenkins installed on the master node
- Proper hostname resolution
- Jenkins user created with appropriate permissions
Jenkins Master and Slave Task Working Flow
The image illustrates a distributed Jenkins environment, showing how Jenkins Master delegates build jobs to connected Agent (Slave) Nodes, each with multiple Executors running inside a JVM (Java Virtual Machine).
Components and Roles
- Jenkins Master Node : Job scheduling, configuration, task delegation
- Agent Slave Node : Remote machine to execute build jobs
- Executor : Execution thread for individual jobs
- JVM : Runtime environment for both Master and Agent
Key Takeaways from the Image:
Jenkins Master Node (Left Side - bright cyan Box)
- This is the central control unit in the Jenkins ecosystem.
- Contains multiple Build Jobs such as:
- Build Job1
- Build Job2
- Build Job3
- Each job contains:
- Task Execution Logic
- Build instructions (e.g., compile, test, deploy)
Role of Master:
- Schedules and assigns jobs to agents.
- Monitors job execution.
- Stores job configuration and build history.
- Delegates the actual job execution to agents (if configured).
Agent Slave Nodes (Right Side - Dark Teal JVM Boxes)
There are two Agent Slave Nodes, each running inside its own JVM:
- Each node contains three Executors.
- Executors are the actual execution units that carry out assigned build tasks.
- They allow parallel execution of multiple builds on a single agent.
Function of Agent Nodes:
- Receive tasks assigned by the Master.
- Execute the job via one of their available Executors.
- Report the status and results back to the Master.
JVM (Java Virtual Machine):
- Both Master and Agent nodes run on top of JVM, enabling Jenkins to be platform-independent.
Communication Flow (White Arrows):
The arrows in the image show how:
- The Master delegates each build job (Job1, Job2, Job3) to different Executors on the Agent Nodes.
- These Executors independently run the build and send results back.
Conclusion
The Jenkins Master-Slave architecture is an incredibly powerful way to scale your CI/CD pipelines. Whether you're managing a few builds a day or deploying code across thousands of services, a distributed Jenkins setup helps you stay efficient, organized, and performant.
Investing time in understanding and implementing this architecture can future-proof your DevOps infrastructure and boost developer productivity significantly.
