Configuring Your Own Linux Build Agent for Azure CI/CD

Azure Pipelines is a powerful CI/CD service within Microsoft Azure DevOps that enables teams to automate building, testing, and deploying applications across multiple platforms and clouds. It supports a wide range of languages, frameworks, and deployment targets, including Windows, Linux, macOS, Docker, and Kubernetes, and integrates seamlessly with GitHub, Bitbucket, and other source control systems.

One of the key decision points when configuring Azure Pipelines is choosing between Microsoft-hosted agents and self-hosted agents. While Microsoft-hosted agents offer convenience and scalability, self-hosted build agents provide distinct advantages for enterprise environments with specific operational, security, or performance needs.

Advantages of Using Self-Hosted Build Agents with Azure Pipelines:

1. Enhanced Security & Compliance
   Self-hosted agents run on infrastructure you control, making them ideal for organizations subject to strict compliance standards (e.g., PCI-DSS, HIPAA). Sensitive code, secrets, and artifacts never leave your internal network, reducing exposure to external threats and simplifying audit trails.

2. Custom Environments & Tooling
   Unlike Microsoft-hosted agents, which come with pre-installed software, self-hosted agents allow full control over the OS, runtime versions, and third-party tools. This is critical for legacy applications, proprietary software, or specialized build dependencies (e.g., custom compilers, licensed tools, or internal libraries).

3. Improved Performance & Reduced Latency
   Hosting agents closer to your source code repositories, artifact storage, or deployment targets minimizes network overhead. For large monorepos or resource-intensive builds, this can significantly reduce pipeline execution time and improve developer productivity.

4. Cost Efficiency at Scale
   While Microsoft-hosted agents incur per-minute usage fees, self-hosted agents—especially when running on existing on-premises or cloud VMs—can be more cost-effective for teams with high-volume or continuous build requirements.

5. Predictable Resource Allocation
   Self-hosted agents let you allocate dedicated CPU, memory, and storage resources, avoiding the “noisy neighbor” effect common in shared hosted environments. This ensures consistent build performance, which is vital for mission-critical deployments.

6. Integration with Internal Systems
   Self-hosted agents can access internal APIs, databases, or network resources (e.g., private NuGet feeds, on-premises artifact repositories, or internal authentication systems) that are unreachable from public cloud agents.

---

Configure a Sample index.html file on Azure Pipelines with a Self-Hosted Linux Build Agent

This simple example shows how to deploy a static HTML page to a web server by pushing a change to Azure Repos and letting the Pipeline do the rest. This pipeline will deploy the static page to an Azure VM using Password-based SSH authentication.

Prerequisites:

Azure account

Azure DevOps account

Step 1: Create an Azure VM

Create an Azure VM running Linux OS with password access, ensure to open port 80 (for HTTP access) and 22 (for SSH access)

Enable Public IP for the VM

Once created, SSH into the VM and install NGINX. Ensure its running on the VM. Confirm on your browser:

http://<vm_public_ip>

Step 2: Create Self hosted Build Agent

In your Azure DevOps Organization home page, at the bottom left, click on Organization Settings

Under Pipelines —> Agent Pools, then click on New Pool

Ensure to select Self-hosted. Use self-hosted-agent-pool as the name. Then click Create

Click on the newly created agent. Go to Agents and click on New Agent

Ensure to click on Linux. Under Download the agent, click on the copy icon to copy the url for the build agent.

Go to your Azure VM, and use the following command to download the build agent to the VM

wget <link_you_copied> - downloads the build agent

mkdir myagent - creates a new directory called myagent

mv vsts-agent-linux-x64-4.264.2.tar.gz myagent - moves the agent to the new directory

cd myagent - changes directory to myagent

tar zxvf vsts-agent-linux-x64-4.264.2.tar.gz myagent.tar.gz - unzips the agent

./config.sh - configures the agent on your VM

Follow the prompt to configure the agent:

Enter (Y/N) Accept the Team Explorer Everywhere license agreement now? (press enter for N) > Y

Where you see “Enter Server URL”, ensure to use your organization’s url eg https://dev.azure.com/estheromiete01

Enter authentication type (press enter for PAT) > (CLICK ENTER)

Go to your Azure DevOps page, click on the red highlighted icon below:

And click on Personal Access Tokens, top right hand-side click on New Tokens. Put a name, the click on Full Access, and then Create. Copy the token displayed and paste in your VM where “Enter personal access token” is requested.

Complete the following:

Once completed, run command: ./run.sh

Now the agent will begin listening for jobs

Step 3: Create a New Project and Set Up Repo in Azure DevOps

In your Azure DevOps Organization home page, create a new project and name it SampleProject

Go into your Project and go to Repos → Files → New Repository

Name it html-deploy-lab. Create a new file called index.html. Copy and paste the following in the html file:

<!DOCTYPE html>
<html>
<head><title>Azure DevOps Deployed Page</title></head>
<body>
  <h1>This page was deployed via Azure Pipeline to a real VM!</h1>
</body>
</html>

Commit the file to the main branch

This is the file that will be delivered to the webserver.

Step 4: Create an SSH Service Connection (Password-Based)

To allow Azure DevOps to copy files to your VM, you need a service connection.

1. Go to Project Settings → Service Connections

2. Click New service connection → SSH

3. Fill in the following:

   • Host name: Your VM’s public IP

   • Port: 22

   • Username: e.g., azureuser

   • Password: Your VM's SSH password

   • Service connection name: ssh-to-nginx-vm

4. Save the service connection.

You now have a secure way for Azure DevOps to access your VM.

Step 5: Define the Azure Pipeline

Now let’s automate the deployment using YAML.

1. Go to Pipelines → Create Pipeline

2. Select your repo

3. Choose YAML, then select “Starter Pipeline”

4. Name your pipeline azure-pipelines.yml and paste the following:

trigger:
  - main

pool:
  name: "self-hosted-agent-pool"

variables:
  sshService: 'ssh-to-nginx-vm'
  webRoot: '/var/www/html'

stages:
  - stage: DeployHTML
    displayName: 'Deploy HTML Page to Azure VM'
    jobs:
      - job: Deploy
        displayName: 'Copy index.html to NGINX web root'
        steps:
          - checkout: self

          - task: SSH@0
            inputs:
              sshEndpoint: '$(sshService)'
              runOptions: 'inline'
              inline: |
                sudo usermod -aG www-data $(whoami)
                sudo chown -R www-data:www-data /var/www/html
                sudo chmod -R 775 /var/www/html
            displayName: 'Set secure permissions for web root'

          - task: CopyFilesOverSSH@0
            inputs:
              sshEndpoint: '$(sshService)'
              sourceFolder: '$(Build.SourcesDirectory)'
              contents: 'index.html'
              targetFolder: '$(webRoot)'
            displayName: 'Copy index.html to VM'

          - task: SSH@0
            inputs:
              sshEndpoint: '$(sshService)'
              runOptions: 'inline'
              inline: |
                echo "Restarting NGINX..."
                sudo systemctl restart nginx
            displayName: 'Restart NGINX on VM'

This pipeline does exactly what we want:

• Watches the main branch

• Copies index.html to the VM’s NGINX root directory

• Restarts NGINX so the new page is served

Step 6: Run the Pipeline

Let’s test it.

1. Commit the YAML file and push to main

2. Azure DevOps will trigger the pipeline

3. Watch the output under Pipelines → Runs

When it finishes, your index.html is live.

Check your VM for running jobs

Check Azure DevOps, Pipelines —> Runs for ongoing job runs:

Confirm the deployment on your browser:

http://<vm_public_ip>

You should see the page you created in index.html. That’s CI/CD in action.

---

NB: Ensure to delete any running resources when done to avoid charges to your account.

---

Conclusion

For organizations with established DevOps practices—particularly those leveraging Infrastructure-as-Code (Terraform), containerization (Docker/Kubernetes), and secure cloud architectures—self-hosted agents in Azure Pipelines offer a flexible, secure, and high-performance foundation for enterprise-grade automation. When combined with robust monitoring, scaling strategies (e.g., agent pools with auto-scaling VMs), and secure configuration practices, they become a cornerstone of reliable, repeatable delivery pipelines.