Jenkins is one of the most popular Continuous Integration/Continuous Delivery platforms in production today. Jenkins has over a million active users, and according to the CloudBees State of Jenkins survey last year, 95% of Jenkins users are already using or plan to start using Docker within 12 months. A CI/CD build system is a very important part of any organization’s automation toolkit, and Anchore has some clear integration points with these tools. In this blog post, I’ll describe and illustrate a simple way to manually integrate Anchore’s open source container image validation engine into a Jenkins-based CI/CD environment. It’s worth noting that this is only one possible method integration between Anchore and Jenkins, and a different approach may be more suitable for your environment. We’d love to hear from you if you find a new way to use Anchore in your CI/CD pipeline!
Anchore allows you to specify “gates” -- checks that are performed on a container image before it moves to the next stage of the development. These gates range from things like required or disallowed packages, properties of the image’s Dockerfile, presence of known vulnerabilities, and so on. The gate subsystem is easily extended to add your own conditions--perhaps application configuration, versioning requirements, etc.
Gates have been designed to run as part of an automated CI/CD pipeline. A popular workflow is to have an organization’s CI/CD pipeline respond to newly-committed Dockerfiles, building images, running tests, and so on. A good place to run Anchore’s Gates would be in between the build of the image and the next phase: whether it’s a battery of tests, or maybe a promotion of an application to the next stage of production. The workflow looks like this:
- Developer commits an updated Dockerfile to Git
- A Jenkins job is triggered based on that commit
- A new container image is built as part of the Jenkins job
- Anchore is invoked to analyze the image
- The status of that image’s gates are checked
At this point, the CI pipeline can make a decision on whether to allow this newly-created and analyzed image to the next stage of development. Gates have three possible statuses: GO, WARN, STOP. They are fairly self-explanatory: an image whose gates all pass GO should be promoted to the next stage. Images with any WARN statuses may need further inspection but may be allowed to continue. An image with a gate that returns a STOP status should not move forward in the pipeline.
Let’s walk through a simplified example. For clarity, I’ve got my Docker, Anchore, and Jenkins instances all on the same virtual machine. Production configurations will likely be different. (I’m running Jenkins 2.7.1, Docker 1.11.2, and the latest version of Anchore from PIP.)
The first thing we need to do is create a Build Job. This is not intended to be a general-purpose Jenkins tutorial, so drop by the Jenkins Documentation if you need some help. Our Jenkins job will poll a GitHub repository containing our very simple Dockerfile, which looks like this:
The relevant section of our Jenkins build job looks like this:
These commands do the following:
docker build -t anchore-test.
This command instructs Docker to build a new image based on the Dockerfile in the directory of the cloned Git repository. The image’s name is “anchore-test”.
anchore analyze --image anchore-test --dockerfile Dockerfile
This command calls Anchore to analyze the newly-created image.
anchore gate --image anchore-test
This command runs through the Anchore “gates” to determine if the newly-generated image is suitable for use in our environment.
Let’s look at the output from this build:
Whoops! Our build failed. Looks like we triggered a couple of gates here. The first one, “PKGDIFF”, is reporting an action of “STOP”. If you look at the “CheckOutput” column, it says: “Package version in container is different from baseline for pkg - tzdata”. This means that along the way the package version of tzdata has changed; probably because our Dockerfile does a “yum update -y”. Let’s try removing that command--maybe we should instead stick to the baseline image that our container team has provided.
So let’s edit the Dockerfile, remove that line, commit the change, and re-run the build. Here’s the output from the new build:
Success! We’ve passed all of the gates. You can change which gates apply to which images and how they are configured by running:
anchore gate --image anchore-test --editpolicy
(You’ll be dropped into the editor specified by the VISUAL or EDITOR environment variables, usually vim.)
Our policy currently looks like this:
DOCKERFILECHECK:NOTAG:STOP
DOCKERFILECHECK:SUDO:GO
DOCKERFILECHECK:EXPOSE:STOP:ALLOWEDPORTS=22
DOCKERFILECHECK:NOFROM:STOP
SUIDDIFF:SUIDFILEDEL:GO
SUIDDIFF:SUIDMODEDIFF:STOP
SUIDDIFF:SUIDFILEADD:STOP
PKGDIFF:PKGVERSIONDIFF:STOP
PKGDIFF:PKGADD:WARN
PKGDIFF:PKGDEL:WARN
ANCHORESEC:VULNHIGH:STOP
ANCHORESEC:VULNLOW:GO
ANCHORESEC:VULNCRITICAL:STOP
ANCHORESEC:VULNMEDIUM:WARN
ANCHORESEC:VULNUNKNOWN:GO
You can read all about gates and policies in our documentation. Let’s try one more thing: let’s change the “PKGDIFF:PKGVERSIONDIFF” policy to “WARN” instead of “STOP”, and re-enable our yum update command in the Dockerfile.
In the policy editor, we’ll change these lines:
PKGDIFF:PKGVERSIONDIFF:STOP
PKGDIFF:PKGADD:WARN
To this:
PKGDIFF:PKGVERSIONDIFF:GO
PKGDIFF:PKGADD:GO
And save and exit. We’ll also edit the Dockerfile, re-add the “RUN yum update -y” line, and commit and push the change. Then let’s run the Jenkins job again and see what happens.
Now you can see that although Anchore still detects an added package and a changed version, because we’ve reconfigured those gates, it’s not a fatal error and the build completes successfully.
This is just a very simple example of what can be done with Anchore gates in a CI/CD environment. We are planning on implementing a full Jenkins plugin for a more streamlined integration, so stay tuned for that. There are also more gates to explore, and you can extend the system to add your own. If you have questions, comments, or want to share how you’re using Anchore, let us know!
