Securing your Jenkins CI/CD Container Pipeline with Anchore (in under 10 minutes)

Jenkins is by far the most popular Continuous Integration/Continuous Delivery platform in use today with over a million users relying on Jenkins every day to improve developer productivity and streamline their development and testing process. In Anchore’s recent container usage survey over 65% of Docker users surveyed were building and testing their containers using Jenkins.

Key among the fundamental tenets of agile development is the notion of “fail fast, fail often” which is where CI/CD comes in: A developer commits code into the source code repository, such as github, which automatically triggers Jenkins to perform a build of the application which is then run through automated tests. If these test fails the developer is notified immediately and can quickly correct the code. This level of automation increases the overall quality of code and speeds development.

While some may feel that “fail fast” sounds rather negative, you could better describe this process as “learn fast” as mistakes are found earlier in the development cycle and can be easily corrected.

The increased use of CI/CD platforms such as Jenkins has helped to improve the efficiency of development teams and streamlined the testing process and we can leverage the same CI/CD infrastructure to improve the security of our container deployments.

For many organizations the last step before deploying an application is for the security team to perform an audit, this may entail scanning the image for vulnerable software components, for example outdated operating system packages that contain known security vulnerabilities (CVEs), to verify that the applications and operating systems are correctly configured and that the organization’s best practices and compliance policies have been correctly implemented.

Today we want to show you how you can add security and compliance checking into the CI/CD process so you can “learn fast” and correct any security or compliance issues early in the development cycle.

This document will outline the steps to deploy Anchore’s open source project with Jenkins to add analytics, compliance and governance to your CI/CD pipeline.

Anchore has been designed to plug seamlessly into the CI/CD workflow.

A developer commits code into the source code management system. This change triggers Jenkins to start a build which creates a container image.

In the typical workflow this container image is then run through automated testing. If an image does not meet your organization’s requirements for security or compliance then it makes little sense to invest the time required to perform automated tests on the image, it would be better to “learn fast” by failing the build and returning the appropriate reports back to the developer to allow the issue to be addressed.

Anchore has published a plugin for Jenkins which, along with Anchore’s open source engine or Enterprise offering, allows container analysis and governance to be added quickly into the CI/CD process.

The following guide will allow you to add image scanning and analysis into your CI/CD process in less time than it has already taken to read this blog post!

Requirements:

This guide presumes the following prerequisites have been met:

  • Jenkins 2.x running on a virtual machine or physical server
  • Each Jenkins node should have Docker 1.10 or higher installed.

Anchore’s Jenkins plugin can work with single node installations or installations with multiple worker nodes.

Notes:

  • Docker should be configured to allow the jenkins user to run Docker commands either directly or through the use of sudo.
  • For most platforms you can simply add the jenkins user to the docker group in /etc/group.
  • For Red Hat based systems using Red Hat’s Docker distribution rather than Docker Inc. then typically the use of sudo is required.
  • To use sudo ensure that the jenkins user is part of the wheel group in /etc/group and ensure that requiretty is not set in /etc/sudoers.

Step 1:

Pull the Anchore’s Docker container image onto the Jenkins host.

If you are running multiple Jenkins workers then this step should be run on each Jenkins node.

# docker pull anchore/jenkins:latest

Step 2:

Create a local data directory

Create a directory on the Jenkins host to hold Anchore data such as CVE feeds and analysis data. This step is optional. If this directory is not created then the Anchore database and feed data will be reconstructed on each use.

If you are running multiple Jenkins workers then these steps should be run on each Jenkins node.

# mkdir /var/lib/anchore/data

Note: If using Red Hat’s distribution of Docker on a system with SELinux enabled then the following additional step must be performed:

# chcon -Rt svirt_sandbox_file_t /var/lib/anchore/data

Step 3:

Install the Anchore plugin.

The Anchore plugin has been published in the Jenkins plugin registry and is available for installation on any Jenkins server.

From the main Jenkins menu select Manage Jenkins

Next select Manage Plugins

Click on the Available tab and scroll down to select “Anchore Container Image Scanner Plugin” and click on “Install without restart”.

Step 4:

Configure Anchore Plugin.

From the Manage Jenkins menu select Configure System.

Scroll down to the Anchore Configuration section.

Enable Anchore Scanning should be checked.

The default container image name should already contain anchore/jenkins:latest which corresponds to the Anchore Docker image we pulled earlier.

The default container name of “jenkins_anchore” can be left unchanged.

If you wish to use a local data volume, that we created in step 2, the directory name should be entered in the Local Data Volume field.

pasted image 0 (2)

Click on the Apply button and these changes will be saved.

At this point the Anchore plugin is configured on Jenkins and is available to be added to any project.

 

Adding Anchore Analysis to a Jenkins project.

We will now add an extra build step to an existing Jenkins project to perform container inspection and analysis.

I have an existing project that already builds a container for my project.

In this project the container is built using a simple “execute shell” build step.

This project includes a simple Dockerfile in the project’s workspace and as part of the build step a unique tag is constructed based on the current date and time.

We will add one extra line to the build step which will create a file called anchore_images in the project workspace.

The anchore_images file contains a line for each image that should be scanned.

The line has two entries: the image repo:tag and the path to the Dockerfile.

For example:

myapp:10301511102006 /path/to/my/Dockerfile

If the Dockerfile is part of the project’s workspace then the ${WORKSPACE} environmental variable provides a convenient way to specify the full directory path.

Next we need to add a build step to call the Anchore analysis and inspection phase:

Click on Add Build step  and select Anchore Container Image Scanner

After selecting the Anchore Container Image Scanner entry a new dialog will be displayed.

Leave all options as default and save the Project. After the initial build we will cover the other options in more detail.

You are now ready to build your project

Once the build has completed you will  either be presented with a failed build or a successful build.

In the first example the build has successfully completed and the developer will most likely continue on without looking at the output of Anchore’s analysis.

In the next example the build has failed because Anchore policy evaluation module concluded that the build would violate the organization’s policies.

At the bottom of the build report you can see a new option Anchore Report which will allow the developer to view details of the build and policy evaluation.

Clicking on the Anchore report brings up detailed information about the evaluation of the policy.

Here we can see a number of warnings but at the bottom we can see a High level CVE vulnerability that triggered the build to fail.

An Anchore policy uses the concept of Gates which are tests that are performed on the image. A gate can output one of 3 actions “GO”, “WARN” and “STOP”.

By default the Anchore plugins will fail a build on a STOP result but not on a WARN however that can be overridden in the configuration of the Anchore build step.

The default policy contains the following checks:

- DOCKERFILECHECK:NOTAG:STOP
- DOCKERFILECHECK:SUDO:GO
- DOCKERFILECHECK:FROMSCRATCH:WARN
- DOCKERFILECHECK:EXPOSE:STOP:ALLOWEDPORTS=443
- DOCKERFILECHECK:NOFROM:STOP
- SUIDDIFF:SUIDFILEDEL:GO
- SUIDDIFF:SUIDMODEDIFF:STOP
- SUIDDIFF:SUIDFILEADD:STOP
- PKGDIFF:PKGVERSIONDIFF:STOP
- PKGDIFF:PKGADD:WARN
- PKGDIFF:PKGDEL:WARN
- ANCHORESEC:VULNUNKNOWN:GO
- ANCHORESEC:VULNHIGH:STOP
- ANCHORESEC:VULNMEDIUM:WARN
- ANCHORESEC:VULNLOW:GO
- ANCHORESEC:VULNCRITICAL:STOP
- ANCHORESEC:UNSUPPORTEDDISTRO:WARN

The Anchore documentation contains extensive details about gates and policies.

You can place your own custom policy in the workspace of the project and configure the name of this file in the Anchore Build step dialog. By default Anchore will look for a file named anchore_policy if this is not found the default policy is used.

 

The default policy will issue a warning if any medium level CVEs are found in the image

ANCHORESEC:VULNMEDIUM:WARN

We can modify this to issue a STOP if a medium level CVE is found.

ANCHORESEC:VULNMEDIUM:STOP

Another default is to STOP the build if a package version is changed

PKGDIFF:PKGVERSIONDIFF:STOP

We may want to change this to a warning if this is expected behaviour within your build.

PKGDIFF:PKGVERSIONDIFF:WARN

This is a simple example of what can be done with Anchore’s Jenkins plugin and Anchore’s open source project, where a simple CVE scan can now be added for free to every image that you are building in Jenkins. But this simple level of scanning address just the tip of the iceberg, an image may contain no operating system packages with known vulnerabilities but may still be insecure, mis-configured or in some other way out of compliance.

Please contact us by filling out the form below to learn more about how you can add detailed more detailed analysis, control and governance into your container infrastructure.

If you have any questions or concerns, we are available to chat on our irc channel #anchore on Freenode.com

Enjoy!

 

>

Sign up for news and updates

Get notified of new features, tips & tricks, articles and more.  Just enter your name and email and we will take care of the rest.

Leave a Comment

Your email address will not be published. Required fields are marked *