CI/CD Pipeline Implementation: Jenkins and GitLab Automation Guide

Build robust CI/CD pipelines using Jenkins and GitLab to automate testing, building, and deployment processes. Learn industry best practices for continuous integration and deployment automation.

Introduction to CI/CD Pipelines

Continuous Integration and Continuous Deployment (CI/CD) pipelines are essential for modern software development, enabling teams to deliver code changes rapidly and reliably.

1. Jenkins Installation and Configuration

Set up Jenkins as your CI/CD automation server:

Jenkins Installation on Ubuntu

# Add Jenkins repository
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo tee 
    /usr/share/keyrings/jenkins-keyring.asc > /dev/null

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] 
    https://pkg.jenkins.io/debian-stable binary/ | sudo tee 
    /etc/apt/sources.list.d/jenkins.list > /dev/null

# Update package index and install Jenkins
sudo apt update
sudo apt install fontconfig openjdk-11-jre jenkins -y

# Start and enable Jenkins
sudo systemctl start jenkins
sudo systemctl enable jenkins

# Configure firewall
sudo ufw allow 8080

# Get initial admin password
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

2. GitLab CI/CD Pipeline Configuration

Create comprehensive GitLab CI/CD pipelines for automated testing and deployment:

Advanced GitLab CI Configuration

# .gitlab-ci.yml for a comprehensive pipeline
stages:
  - validate
  - test
  - security
  - build
  - deploy-staging
  - integration-tests
  - deploy-production

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"

# Validation stage
code-quality:
  stage: validate
  image: sonarsource/sonar-scanner-cli:latest
  script:
    - sonar-scanner
      -Dsonar.projectKey=$CI_PROJECT_NAME
      -Dsonar.sources=.
      -Dsonar.host.url=$SONAR_HOST_URL
      -Dsonar.login=$SONAR_TOKEN
  only:
    - merge_requests
    - main
    - develop

# Testing stage
unit-tests:
  stage: test
  image: php:8.1-cli
  services:
    - postgres:13
    - redis:alpine
  script:
    - apt-get update -qq && apt-get install -y -qq git curl libpq-dev
    - docker-php-ext-install pdo pdo_pgsql
    - curl -sS https://getcomposer.org/installer | php
    - php composer.phar install --no-dev --optimize-autoloader
    - php artisan migrate --force
    - php artisan test --coverage --min=80
  artifacts:
    reports:
      junit: tests/reports/junit.xml
      coverage_report:
        coverage_format: cobertura
        path: tests/reports/coverage.xml

# Security scanning
security-scan:
  stage: security
  image: owasp/zap2docker-stable
  script:
    - mkdir -p /zap/wrk/reports
    - /zap/zap-baseline.py -t http://staging.example.com -r baseline-report.html
  artifacts:
    paths:
      - baseline-report.html
  allow_failure: true

# Build stage
build-application:
  stage: build
  image: docker:20.10.16
  services:
    - docker:20.10.16-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:latest .
    - docker tag $CI_REGISTRY_IMAGE:latest $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  only:
    - main
    - develop

# Staging deployment
deploy-staging:
  stage: deploy-staging
  image: alpine:latest
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - apk add --no-cache openssh-client curl
    - ssh deploy@staging-server "cd /var/www/app && ./deploy.sh staging $CI_COMMIT_SHA"
    - sleep 30
    - curl -f https://staging.example.com/health || exit 1
  only:
    - develop
  when: manual

# Production deployment
deploy-production:
  stage: deploy-production
  image: alpine:latest
  environment:
    name: production
    url: https://example.com
  script:
    - ssh deploy@prod-server "cd /var/www/app && ./deploy.sh production $CI_COMMIT_SHA"
    - sleep 60
    - for i in {1..5}; do curl -f https://example.com/health && break || sleep 10; done
  only:
    - main
  when: manual

3. Jenkins Pipeline Implementation

Create sophisticated Jenkins pipelines using Jenkinsfile:

Declarative Jenkins Pipeline

// Jenkinsfile for comprehensive CI/CD pipeline
pipeline {
    agent any

    environment {
        REGISTRY = "docker.io/mycompany"
        IMAGE_NAME = "webapp"
        SLACK_CHANNEL = "#deployments"
    }

    options {
        buildDiscarder(logRotator(numToKeepStr: "10"))
        timeout(time: 1, unit: "HOURS")
        retry(2)
    }

    stages {
        stage("Checkout & Validate") {
            parallel {
                stage("Code Quality") {
                    steps {
                        script {
                            def scannerHome = tool "SonarQube Scanner"
                            withSonarQubeEnv("SonarQube") {
                                sh "${scannerHome}/bin/sonar-scanner"
                            }
                        }

                        timeout(time: 10, unit: "MINUTES") {
                            waitForQualityGate abortPipeline: true
                        }
                    }
                }

                stage("Dependency Check") {
                    steps {
                        sh "npm audit --audit-level moderate"
                        sh "composer audit"
                    }
                }
            }
        }

        stage("Test") {
            parallel {
                stage("Unit Tests") {
                    steps {
                        sh "composer install --no-dev --optimize-autoloader"
                        sh "php artisan test --coverage --min=80"

                        publishTestResults testResultsPattern: "tests/reports/junit.xml"
                    }
                }

                stage("Integration Tests") {
                    steps {
                        sh "npm ci"
                        sh "npm run build"
                        sh "cypress run"
                    }
                }
            }
        }

        stage("Build & Push") {
            when {
                anyOf {
                    branch "main"
                    branch "develop"
                }
            }
            steps {
                script {
                    def image = docker.build("${REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}")

                    docker.withRegistry("https://index.docker.io/v1/", "dockerhub") {
                        image.push()
                        image.push("latest")
                    }
                }
            }
        }

        stage("Deploy to Staging") {
            when {
                branch "develop"
            }
            steps {
                sh "kubectl apply -f k8s/staging/ --namespace=staging"
                sh "kubectl rollout status deployment/webapp --namespace=staging"
            }
        }

        stage("Deploy to Production") {
            when {
                branch "main"
            }
            steps {
                input "Deploy to Production?"
                sh "kubectl apply -f k8s/production/ --namespace=production"
                sh "kubectl rollout status deployment/webapp --namespace=production"
            }
        }
    }

    post {
        success {
            slackSend(
                channel: env.SLACK_CHANNEL,
                color: "good",
                message: "Pipeline succeeded for ${env.JOB_NAME}"
            )
        }

        failure {
            slackSend(
                channel: env.SLACK_CHANNEL,
                color: "danger",
                message: "Pipeline failed for ${env.JOB_NAME}"
            )
        }
    }
}

4. Deployment Automation Scripts

Create automated deployment scripts for reliable deployments:

Blue-Green Deployment Script

#!/bin/bash
# deploy.sh - Blue-green deployment script

ENVIRONMENT=$1
IMAGE_TAG=$2
STRATEGY=${3:-rolling}

log() {
    echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*"
}

blue_green_deploy() {
    log "Starting blue-green deployment..."

    # Deploy to green environment
    kubectl apply -f k8s/$ENVIRONMENT/green/
    kubectl rollout status deployment/webapp-green --namespace=$ENVIRONMENT

    # Health checks
    sleep 30
    kubectl exec -n $ENVIRONMENT deployment/webapp-green -- curl -f http://localhost/health

    if [ $? -eq 0 ]; then
        log "Green environment healthy, switching traffic..."
        kubectl patch service webapp -n $ENVIRONMENT -p '{"spec":{"selector":{"version":"green"}}}'

        # Clean up blue environment
        sleep 60
        kubectl delete deployment webapp-blue -n $ENVIRONMENT || true

        log "Blue-green deployment completed successfully"
    else
        log "Health check failed, aborting deployment"
        kubectl delete deployment webapp-green -n $ENVIRONMENT
        exit 1
    fi
}

rolling_deploy() {
    log "Starting rolling deployment..."

    kubectl set image deployment/webapp webapp=$REGISTRY/$IMAGE_NAME:$IMAGE_TAG --namespace=$ENVIRONMENT
    kubectl rollout status deployment/webapp --namespace=$ENVIRONMENT --timeout=600s

    log "Rolling deployment completed"
}

case $STRATEGY in
    blue-green)
        blue_green_deploy
        ;;
    rolling)
        rolling_deploy
        ;;
    *)
        log "Unknown deployment strategy: $STRATEGY"
        exit 1
        ;;
esac

Conclusion

Implementing robust CI/CD pipelines with Jenkins and GitLab enables teams to deliver software faster and more reliably. These automated workflows reduce manual errors, improve code quality, and provide rapid feedback to developers.