Building a declarative Jenkinsfile for Continuous Delivery

Continuous Integration(CI) methodologies help developers automate code development so that code can be pushed more frequently which leads to better collaboration and faster software delivery,
Continuous Delivery (CD) methodologies automate the delivery of application on infrastructures such as servers, this reduces the load on developers as they only have to worry about developing the code and pushing it onto a repository while the deployment of said code is all automated.
At the heart of continuous integration and delivery lies a powerhouse called Jenkins, Jenkins is an automation tool that lets you build and test deployments of your developed code while maintaining a build log to trace all changes made.
Jenkins can integrate with a lot of plugins such as Git to pull all changes from a repository or maven to build the code automatically.

Jenkins can also run your tests and provide code coverage and code reports by integrating with plugins like Sonarqube so that all commits made to a repository are covered and tested and all decreases/ increases in code coverage can be traced to each commit made to the repository.

So how do I integrate with such plugins in my Jenkins deployments?

In comes the Jenkinsfile, Jenkinsfile is a text file that is written in Groovy syntax that lets you identify the plugins used and the commands that should run during your deployment.

I will be showing you a run-through of a Jenkinsfile we use in our manual deployments while showcasing the basics of a Jenkins file.

1) The first thing we do is to identify an agent which instructs Jenkins to allocate an executor which will run our deployment commands.

 pipeline {
        agent any

2) The next step is to identify all the tools we will be using in our deployment, for this example, we only need to define maven as we can then use maven to call other tools such as fabric8 without needing to implicitly identify which tools we will be using which makes our Jenkinsfile very versatile.

 tools {
        maven 'maven'
            }

3) We can also list a few parameters in the case of manual deployment so that the user can choose the host/namespace they will be deploying on and choose whether they would like to run unit and integration tests and send them over to sonarqube to report. To do so we can either keep them as text inputs with default values such as the HOST option or we can give the user options to choose from such as the NAMESPACE option or just a checkbox like the SKIP_TESTS option.
We can then use these parameters during our builds for dynamic deployment.

  parameters {
      string(name: 'HOST', description: 'Ingress host', defaultValue: 'api.sumerge.gov.local')
      choice(name: 'NAMESPACE', choices: [sumerge-inspections'], description: 'K8s Namespace')
      choice(name: 'DOCKER_REGISTRY', choices: ['registry.sumerge.local'], description: 'Docker registry')
      booleanParam(name: 'SKIP_TESTS', defaultValue: false, description: 'Do you want to skip tests and sonarqube?')
      choice(name: 'SONAR', choices: ['http://sonarqube.sumerge.local'], description: 'SonarQube host url')
  }

4) The most important part of a Jenkins file is the stages section where the deployment can be split into various checkpoints all responsible for performing a command, this helps us visualize and easily identify where a deployment fails. It can also visualize which steps of the deployment take the longest so that we can identify where we would need to optimize our deployment if needed.
The first we do in our deployment is run mvn clean to delete any old build directories.

stages {
stage('Clean') {
steps {
sh 'mvn clean post-clean -Dbuild.number=${BUILD_NUMBER}'
}
}

5) Then we can run our unit and integration tests using maven, keep in mind that these steps can now be skipped as we are using the parameters we created in step 2 which allowed the user to pick the skip tests option.

stage('Unit tests') {
    steps {
        sh 'mvn test -Dmaven.test.skip=${SKIP_TESTS} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
           }
                    }

stage('Integration tests') {
     steps {
        sh 'mvn integration-test verify -Dskip.surefire.tests -Dmaven.test.skip=${SKIP_TESTS} -Dbuild.number=${BUILD_NUMBER}
        -Dspring.profiles.active=kubernetes'
            }
                           }

6)  We can now take the output of these tests that we ran and integrate them with sonarqube to generate our reports. This is all dynamic using our parameters as the user can direct these test results to their sonarqube host that they input in the parameters or they can completely skip the tests thus skipping the sonarqube stage.

    stage ('SonarQube') {
      steps {
        script {
            if (params.SKIP_TESTS) {
                echo "Test skip $SKIP_TESTS, so no SonarQube"
            } else {
                sh 'mvn sonar:sonar -Dsonar.host.url=${SONAR} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
            }
        }
      }
    }

7) After we are done with building our code and running our tests we can create a docker image which we can then deploy onto Kubernetes dynamically using the docker registry and Kubernetes namespaces declared in the parameters.

stage ('Docker') {
      steps {
        sh 'mvn spring-boot:build-image k8s:push -Ddocker.registry=${DOCKER_REGISTRY} -Dk8s.namespace=${NAMESPACE}
        -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
      }
    }

8)  And the final step is to deploy our Docker image on Kubernetes by listing out our commands that will run on our image once it is deployed.

    stage ('K8s deploy') {
      steps {
        sh """
        sed -i "s|@DOCKER_REGISTRY@|${DOCKER_REGISTRY}|g" $WORKSPACE/kubernetes-db.yml
        sed -i "s|@NAMESPACE@|${NAMESPACE}|g" $WORKSPACE/kubernetes-db.yml
        mvn k8s:resource -Ddocker.registry=${DOCKER_REGISTRY} -Dhost=${HOST} -Dk8s.namespace=${NAMESPACE} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes
        kubectl apply -f $WORKSPACE/target/classes/META-INF/jkube/kubernetes.yml -n ${NAMESPACE} --kubeconfig /var/jenkins_home/k8s/${NAMESPACE}
        kubectl apply -f $WORKSPACE/kubernetes-db.yml -n ${NAMESPACE} --kubeconfig /var/jenkins_home/k8s/${NAMESPACE}
        """
      }
    }
  }

Below is a copy of the Jenkinsfile referenced in our steps.

pipeline {
  agent any

  tools {
    maven 'maven'
  }

  parameters {
      string(name: 'HOST', description: 'Ingress host', defaultValue: 'api.sumerge.gov.local')
      choice(name: 'NAMESPACE', choices: [sumerge-inspections'], description: 'K8s Namespace')
      choice(name: 'DOCKER_REGISTRY', choices: ['registry.sumerge.local'], description: 'Docker registry')
      booleanParam(name: 'SKIP_TESTS', defaultValue: false, description: 'Do you want to skip tests and sonarqube?')
      choice(name: 'SONAR', choices: ['http://sonarqube.sumerge.local'], description: 'SonarQube host url')
   }

  options {
    skipStagesAfterUnstable()
    disableConcurrentBuilds()
  }

  stages {
   stage('Clean') {
    steps {
    sh 'mvn clean post-clean -Dbuild.number=${BUILD_NUMBER}'
   }
 }

   stage('Unit tests') {
    steps {
    sh 'mvn test -Dmaven.test.skip=${SKIP_TESTS} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
   }
 }

   stage('Integration tests') {
    steps {
    sh 'mvn integration-test verify -Dskip.surefire.tests -Dmaven.test.skip=${SKIP_TESTS} -Dbuild.number=${BUILD_NUMBER}
    -Dspring.profiles.active=kubernetes'
   }
 }

  stage ('SonarQube') {
   steps {
   script {
       if (params.SKIP_TESTS) {
          echo "Test skip $SKIP_TESTS, so no SonarQube"
       } else {
          sh 'mvn sonar:sonar -Dsonar.host.url=${SONAR} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
       }
    }
  }
 }

  stage ('Docker') {
    steps {
       sh 'mvn spring-boot:build-image k8s:push -Ddocker.registry=${DOCKER_REGISTRY} -Dk8s.namespace=${NAMESPACE}
       -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes'
     }
   }

  stage ('K8s deploy') {
   steps {
     sh """
     sed -i "s|@DOCKER_REGISTRY@|${DOCKER_REGISTRY}|g" $WORKSPACE/kubernetes-db.yml
     sed -i "s|@NAMESPACE@|${NAMESPACE}|g" $WORKSPACE/kubernetes-db.yml
     mvn k8s:resource -Ddocker.registry=${DOCKER_REGISTRY} -Dhost=${HOST} -Dk8s.namespace=${NAMESPACE} -Dbuild.number=${BUILD_NUMBER} -Dspring.profiles.active=kubernetes
     kubectl apply -f $WORKSPACE/target/classes/META-INF/jkube/kubernetes.yml -n ${NAMESPACE} --kubeconfig /var/jenkins_home/k8s/${NAMESPACE}
     kubectl apply -f $WORKSPACE/kubernetes-db.yml -n ${NAMESPACE} --kubeconfig /var/jenkins_home/k8s/${NAMESPACE}
     """
     }
   }
  }
 }

Conclusion

In our Jenkins file, we can instruct our agent to use tools such as maven which can build our module and run its tests for we can then use with other CI/CD tools such as sonarqube, and we can also instruct it to dockerize that build and deploy it on Kubernetes for easy automatic deployment.

What we have just demonstrated is only a glimpse of the possibilities that can be done using a Jenkins file for automatic deployment, as Jenkins is free and can integrate with over 1000 plugins I urge you to experiment with your own Jenkinsfile and fully utilize its array of tools to easily optimize your project pipelines.