Jenkins Master (Server) 🧙‍♂️

🏰 Picture the Jenkins Master as the wise wizard in our DevOps kingdom. It's the brains behind the operation, responsible for managing the entire Jenkins environment. 🧠 It's where you configure jobs, manage plugins, and create pipelines. 🛠️ The Master also stores your precious build and configuration data.

Jenkins Agent 🏃‍♂️

Now, meet our trusty sidekick, the Jenkins Agent! 🦸‍♂️ The agent is like the muscle in our DevOps team, doing the heavy lifting. 🏋️ When you trigger a Jenkins job from the Master, the real action happens on the Agent node configured in the job. 🚀 It's like dispatching your heroes to complete a quest!

When the Magic Happens 🪄

So, here's the cool part! When you kick off a Jenkins job, the Master uses its mystical powers to communicate with the Agent. 📡✨ The Agent takes your job's instructions and executes them on its own machine. This magic enables Jenkins to handle multiple jobs simultaneously by distributing the workload among different Agents. 🌐🔁

Pre-requisites 📝

Before diving into Jenkins mastery, make sure you've got your prerequisites in order. 📋🔍 When creating an Agent, it's crucial to separate rights, permissions, and ownership for Jenkins users. This ensures a secure and organized DevOps environment. 💼🔒

In a nutshell, the Jenkins Master-Server is your commander-in-chief, orchestrating the grand plan. Meanwhile, the Jenkins Agent is your trusted field agent, executing missions on various machines. Together, they make DevOp's dreams come true! 🌟🚀

How to Set Up Jenkins Master and Agent on AWS EC2 Machines?

Step 1: Create Two EC2 Instances

Log in to your AWS Management Console.
Navigate to the EC2 console.
Click Launch Instance.
Select an Amazon Machine Image (AMI) for your Jenkins master. I recommend using the Jenkins AMI for Amazon Linux.
Configure your instance type and other settings.
Create a new security group or select an existing security group that allows inbound traffic on port 8080.
Launch your instance.
Repeat steps 4-7 to create a Jenkins agent instance.

Step 2: Connect Master with Agent using SSH Keys

Generate an SSH key pair on your Jenkins master instance.

using the steps:
- Go to the ssh folder of the master using the command - cd .ssh/
- Generating public/private RSA key pair - ssh-keygen
- Enter the file in which to save the key (/home/ubuntu/.ssh/id_rsa): keep entering and your file will created.
- Now check the file name with the command - ls
- Using cat command check the public key - cat id_rsa.pub
  
  Now copy this public key and paste in the agent's authorized key folder like this:
  
  Now to connect the master with agent kinldy go in master and write command - ssh ubuntu@ip address of agent instances.
Copy the public SSH key to your Jenkins agent instance.
Add the public SSH key to the authorized_keys file on your Jenkins agent instance.

Step 3: Install Jenkins on Master and Agent

Steps to install the Jenkins on AWS EC2 instance:

This is the Debian package repository of Jenkins to automate installation and upgrade. To use this repository, first add the key to your system.

Copy this code and paste it on the terminal
```
   curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
     /usr/share/keyrings/jenkins-keyring.asc > /dev/null
```

Then add a Jenkins apt repository entry:

 echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
     https://pkg.jenkins.io/debian binary/ | sudo tee \
     /etc/apt/sources.list.d/jenkins.list > /dev/null

Update your local package index, then finally install Jenkins:

copy these commands one by one in your terminal.
```
  sudo apt-get update
  sudo apt-get install fontconfig openjdk-17-jre
  sudo apt-get install jenkins
```

Step 4: Configure Jenkins on Master and Agent

On your Jenkins master instance, open a web browser and navigate to http://<your-master-instance-public-ip>:8080.
You will be prompted to unlock Jenkins using an initial admin password. Retrieve this from your master instance by running the following command:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Follow the on-screen instructions to complete the setup.
Install the suggested plugins and create your first admin user.
To configure the Jenkins agent, open a terminal window and navigate to the Jenkins agent directory:

cd /opt/jenkins/agent

Run the following command to start the Jenkins agent:

./agent.jar -url http://<your-master-instance-public-ip>:8080

Create an Agent on Master and Configure its Launch Method and Credentials

To create an agent on the master and configure its launch method and credentials, follow these steps:

Go to Manage Jenkins > Manage Nodes and Clouds > New Node
Enter a name for the agent (e.g. agent-1) and choose Permanent Agent
Click OK
Enter the following configuration for the agent:
- Remote root directory: /home/ubuntu (or any other directory that you want to use for Jenkins workspace)
- Labels: agent (or any other label that you want to use to identify the agent)
- Usage: Only build jobs with label expressions matching this node (or any other option that suits your needs)
- Launch method: Launch agents via SSH
- Host: agent-ip (the public IP address of your agent instance)
- Credentials: Add > Jenkins > SSH Username with private key > Enter username (ubuntu) and private key (paste the content of your key pair file .pem) > Add > Choose the credential that you just added
- Host Key Verification Strategy: Non-verifying Verification Strategy (or any other option that suits your security needs)
Click Save

Connect the Agent to the Master and Verify its Status

To connect the agent to the master and verify its status, follow these steps:

Go to Manage Jenkins > Manage Nodes and Clouds > agent-1 > Launch Agent
Wait for a few seconds until you see the message “Agent successfully connected and online”
Go back to Manage Jenkins > Manage Nodes and Clouds and check that the agent is online and idle

How to Run a Pipeline Job for a Node.js todo App on a Jenkins Agent?

Step 1: Create a Pipeline Job on the Master

Log in to your Jenkins dashboard.
Click New Item.
Enter a name for your pipeline job. I recommend using the name of your Node.js todo app.
Select Pipeline.
Click OK.

Step 2: Write a Pipeline Script in Jenkinsfile

To write a pipeline script in Jenkinsfile, follow these steps:

Open your GitHub repository of the Node.js todo app in your code editor or IDE
Create a file named Jenkinsfile in the root directory of your repository
Write the following code in Jenkinsfile:

pipeline {
    agent none // This means that the pipeline will not run on any agent by default
    stages {
        stage('Clone') {
            agent { label 'agent' } // This means that this stage will run on an agent with the label 'agent'
            steps {
                echo "cloning the repository"
                git 'https://github.com/avanishnit08/node-todo-cicd.git' // This will clone the GitHub repository to the agent's workspace
            }
        }
        stage('Build') {
            echo "Building the image"
            steps {
                sh 'docker build -t todo-app .'
            }
        }
        stage('Deploy') {
            steps {
                echo "Deploying todo app"
                sh 'docker-compose down'
                sh 'docker-compose up -d'
            }
        }
    }
}

Step 3: Run the Pipeline Job on Master and Verify the Results

Click Run to start the pipeline job.
Monitor the pipeline job progress in the Jenkins dashboard.
Once the pipeline job completes successfully, you can verify the results by accessing your Node.js todo app.

Conclusion

In this blog, I have explained what is Jenkins master-slave architecture and how to set up Jenkins agents on AWS EC2 machines. I have also demonstrated how to run a pipeline job for a Node.js todo app on a Jenkins agent. I hope you found this blog useful and learned something new.

If you have any questions or feedback, please feel free to leave a comment below or contact me on GitHub or LinkedIn.

Thank you for reading and happy learning!

Jenkins Master-Slave Architecture and Agents | Day 28 | 90 Days of DevOps

Table of contents