Year: 2017

Democratizing Container Certification

Posted on by Christian Wiens

Today Red Hat announced a new certification program for container images. Key to this announcement is the concept of a container health index that is used to grade a container which is “determined by Red Hat’s evaluation of the level of critical or important security errata that is missing from an image”.

Certifications are certainly not a new thing for Red Hat, it could be said that Red Hat built their enterprise business on top of an industry-leading certification program. Enterprises need to have confidence in their deployments, to know that when they deploy an application it will work, it will be secure, it can be maintained, and it will be performant. In the past, this confidence came through certification. In the early days of Linux, Red Hat really set the standard and worked with hardware and software vendors on certification programs to give a level of assurance to end-users that the operating system would run reliably on their hardware and also offer insurance in the form of enterprise-grade commercial support if they encountered issues.

One Size Doesn’t Fit All

Today the problem is more complex and there can no longer be just a single certification. For example, the requirements of a financial services company are different from the requirements of a healthcare company handling medical records, and these are different from the needs of a federal institution and so on. Even the needs of individual departments within any given organization may be different.

What is needed now is for IT operations and security to be able to define their own certification requirements, which may differ even from application to application, allowing them to define these policies and evaluate them before applications are deployed into production.

What we are talking about is the democratization of certification.

Rather than placing certification in the hands of a small number of vendors or standards bodies, organizations need to define what certification means to them.

Anchore’s goal is to provide a toolset that allows developers, operations, and security teams to maintain full visibility of the ‘chain of custody’ as containers move through the development lifecycle while providing the visibility, predictability, and control needed for production deployment.

At the heart of Anchore’s solution is the concept of users certifying container images based on rules that they define. In the past, certifications for applications typically came from operating systems vendors who defined their own standards and worked with independent software vendors (ISVs) on certification programs to give a level of assurance to end users that the application was compatible with the underlying operating system. Other organizations have created standards and certification tests to cover various forms of compliance validation, especially in the government sector or regulated industries.

Container Certification on Your Terms

Today the baseline feature set for container security is a CVE scan and that’s certainly required but it’s just the first step. An image may contain no operating system CVEs but may still be insecure, misconfigured or in some other way out of compliance. Container images typically contain hundreds, often thousands of files – some coming from operating system packages, some from configuration files, some from 3rd party software libraries such as Node.JS NPMs, Ruby GEMs, Python modules, Java Archives, and some may be supplied by the user. Each one of these artifacts should undergo the same level scrutiny as the operating system packages.

I’m sure that the policies you have in place today for your traditional deployments are more than just ensuring that you’ve updated all operating system packages. While these policies should cover security, starting with the ubiquitous CVE scan, they should go further to analyze the configuration of key security components, for example, you could have the latest version of the Apache or NGINX web server but have configured the wrong set of TLS Ciphers suites leading to insecure communication. Outside of security, certification policies should cover application-specific configurations to comply with best practices or to enable consistency and predictability. With Anchore organizations can define policies and certify containers on their terms – applying the specific policies that matter to them which can even be workload specific and these policies can be applied to any operating system.

As we move away from traditional IT models toward cloud, PaaS, containers and hybrid deployments, the operating system becomes less visible and applications become the focus. However, the operating system is still critical whether as part of a container or underpinning your PaaS platform, and as such it should be secure and well maintained. Some of Anchore’s users have policies requiring that containers should only be built on top of Red Hat Enterprise Linux (RHEL) since this is their corporate standard. Others may use different base operating systems but apply a consistent set of policies to all of these images. Tthis becomes especially important as organizations may consume containers from many sources, including freely available containers on public registries as well as containers provided by software vendors.

With Anchore you own the certification.

Watching Images for Updates

Posted on by Christian Wiens

The majority of Docker users do not build their images from scratch, instead, they are built on top of base images that have been created and published by others. Usually, these are official images that have been created by an organization or community and submitted to Docker Inc. and the community for official review.

Images should be regularly updated by their publishers to include the latest content like the latest release of operating system packages to add new features or fixes to security vulnerabilities or new versions of an application or software library. As a developer how do I know when an image has been pushed?

DockerHub supports the concept of webhooks that allow a user to receive a notification via an HTTP message when a new image has been pushed. This feature can be used in a number of ways, most commonly it’s used to trigger builds or deployments of applications based on a specific image. This feature has a major limitation: It only supports webhooks for images owned by a user, meaning you can trigger webhooks for images you have created but not for other images such as a base image from an official publisher.

Yesterday the Debian team updated their base image; you can inspect the image here using the Anchore Navigator. But how would you know that the image has been updated? The most common approach is just to try and pull the image to see if a new version has been published.

# docker pull debian:latest

If an updated image is present the docker client will download the newer image.

Trying to pull repository docker.io/library/debian ...
sha256:476959f29a17423a24a17716e058352ff6fbf13d8389e4a561c8ccc758245937: Pulling from docker.io/library/debian
10a267c67f42: Pull complete
Digest: sha256:476959f29a17423a24a17716e058352ff6fbf13d8389e4a561c8ccc758245937
Status: Downloaded newer image for docker.io/debian:latest

If you already have the latest image then the docker client will report that your image is up to date.

Trying to pull repository docker.io/library/debian ...
sha256:476959f29a17423a24a17716e058352ff6fbf13d8389e4a561c8ccc758245937: Pulling from docker.io/library/debian
Digest: sha256:476959f29a17423a24a17716e058352ff6fbf13d8389e4a561c8ccc758245937
Status: Image is up to date for docker.io/debian:latest

One of the most popular free features of the Anchore Navigator is the ability to subscribe to images in order to receive notifications when images are updated.

In the search results you will see a list of repositories. Anchore Navigator can search through all public images on DockerHub. You will see two types of repositories: Analyzed and Preview.

Repositories and TAGs that Anchore is monitoring. For these repositories and TAGS any time a new image is pushed Anchore will download the image and perform detailed inspection including image metadata, package manifests, file lists, security vulnerabilities and policies.

Repositories that are publicly available on DockerHub but that Anchore has not yet downloaded images.

For example, searching for debian gives the following initial results.

As you can see the first two repositories in the results list have already been analyzed and you can select the repository to view a list of tags and inspect individual images.

If the repository and tag that you wish to monitor has not yet been analyzed you can press the button to submit this TAG to Anchore to be analyzed.

All the official repositories and several hundred of the most popular public repositories are already analyzed by Anchore so the chances are you’ll find the image you are looking for right away.

Here you can see the overview page for the official Debian image. If you want to receive notifications from Anchore when the image is updated press the Subscribe button and Anchore will notify you when the image has been updated.

You can unsubscribe from the image from the image’s overview page and you can see a list of your image subscriptions and favorited images from the “My Images” page accessible from the left navigation menu.

Here’s an example notification email including details of which subscribed images have been updated. From here you can click on the links to be taken to the overview for the new images:

This is just one example of the features available for free to all Anchore Navigator users.

A Snapshot of the Container Ecosystem

Posted on by Christian Wiens

Over the last 2 months, we ran a short survey to collect information about Container usage. The survey was slightly shorter than the one we performed in conjunction with DevOps.com and Redmonk 6 months ago but provides deep insight into how the container ecosystem has shifted and continued to evolve over a short period of time. Running multiple surveys gives us the ability to see trends develop and as we review the results of each survey we think of new questions to ask in the next survey to dig deeper.

One of the most interesting data points we extracted, which backs up what we’ve seen in the field, is who is paying for container infrastructure: how much of the container infrastructure is paid -vs- free. In our next survey, we’ll dig deeper into this topic to see where organizations are financially investing in their container infrastructure.

Another interesting finding from the survey data was that many companies/container users still lack the necessary security practices to safely deploy containers in production environments.  Operations and security are still racing to catch up with developers when it comes to the use of containers, but they will need to adapt quickly and put the governance in place to effectively execute and capitalize on the benefits of true microservices architecture.

Anatomy of a CVE

Posted on by Christian Wiens

We often mention CVEs in our blogs but we usually skip over the topic, explaining that while CVE checking is important, it is just the tip of the iceberg and that you need to look deeper into the image to check configuration files, non-packaged files, software artifacts such as Ruby GEMs and Node.JS NPMs.

We recently got a Tweet from Marc Boorshtein from Tremolo Security asking why Anchore reported less CVEs in an image than were reported in scan results from the Docker Store.

So we’re going to take this opportunity to dig into some more details about CVEs to understand what they are, where the data comes from, and how we report on vulnerabilities, and then we’ll use that information to answer Marc’s question. For those who don’t want to read all the way through, the tl;dr here is that Anchore’s results are correct!

The Common Vulnerabilities and Exposures (CVE) system establishes a standard for reporting and tracking vulnerabilities. CVE Identifiers are assigned to vulnerabilities to make it easier to share and track information about these issues. The identifier takes the following form: CVE-YEAR-NUMBER, for example CVE-2014-0160.

The CVE identifier is the official way to track vulnerabilities, however, in some cases, well-known vulnerabilities are given names and even logos such as the famous Heartbleed vulnerability. Whether this trend of naming and branding vulnerabilities is a good thing is debatable. Some argue that this branding helps raise awareness others feel it’s a distraction. Either way this trend started by Codenomicon with Heartbleed has continued with many new vulnerabilities receiving catchy names such as Dirty Cow and Badlock. Not all serious vulnerabilities get branded and not all branded vulnerabilities are serious.

The CVE database is maintained by the Mitre Corporation under contract from the US Government. While Mitre retains responsibility for maintaining the CVE list there are a number of organizations who, under Mitre’s direction, can issue CVE numbers – these are called CVE Numbering Authorities. As of today, there are 53 organizations participating in this program, usually, these are hardware or software vendors such as Canonical, Google, IBM and Red Hat. Many vendors have their own vulnerability tracking databases and CVE helps by providing the glue that links these databases together, for example, a vulnerability in vendor’s hardware appliance may be traced back to an issue in a software library used by many other applications. Having a common identifier to refer to this issue simplifies tracking and reduces complexity.

Another database that you’ll see referenced frequently is the National Vulnerability Database (NVD) which is run by the National Institute of Science and Technology (NIST). This database builds on top of the CVE database by adding extra information such as severity scores, fix information and vendor-specific details.

Let’s use an example to dig deeper into CVEs:

CVE-2016-5195 is a bug that impacts the Linux kernel. It’s a race condition that if successfully exploited can allow local users to gain root privileges. This vulnerability is better known as “Dirty COW,” since it leverages incorrect handling of a copy-on-write (COW) feature.

Reading the details in the CVE database you can see that this issue impacts Linux kernel versions 2.x, 3.x and 4.x before version 4.8.3. More details can be found in the NVD database here. Listed in the NVD database you will find information about the severity score of the vulnerability and links to vendors advisories.

So, in theory, any Linux kernel prior to version 4.8.3 is vulnerable to this exploit however in practice things are more complicated. Take for example CentOS, where the latest available kernel is version 3.10 (or more accurately 3.10.0-514.10.2).

At first, you might expect this kernel to be vulnerable to “Dirty Cow” however Red Hat backported the fix from 4.8.3 into an older kernel version. Backporting is a popular practice for enterprise focussed software products whose users want to keep a well known, stable version of a software platform but still take advantage of security fixes or new features. Backporting selective features and fixes minimize the risk of adopting a completely new release of a software platform.

As you can see from this example, while the practice of backporting has many advantages for end-users it complicates the process of auditing installed software using the CVE database.

For this reason, many vendors produce their own vulnerability tracking feeds that link back to the CVE database but provides vendor-specific information. For example, Red Hat issues Red Hat Security Advisories (RHSAs) which are publicly available and can also be used to map between RHSAs and CVEs. Other distributions such as Debian, Oracle, SUSE and Oracle provide similarly detailed feeds.

These vendor-specific feeds contain valuable information that may not be easily obtained from the NVD database. For example, a Linux distributor may discern that while the upstream project that they are using for a given software package may be impacted with a certain CVE, the way that the package is configured and compiled on their platform may not be impacted by this CVE. A good example can be seen here in Debian’s security tracker. For this reason, a combination of using a vendor’s specific feed and whitelists will provide more accurate information in a security scan.

We started this blog by referencing a Tweet that compared Anchore’s scan results to Docker Store’s scan result. The image is question was based off CentOS so we will use this as the foundation for comparison.

You can view Anchore’s analysis of the latest CentOS image in the Anchore Navigator here:
Selecting the Security tab will show known vulnerabilities.

Inspecting the same image in DockerHub or DockerStore will show significantly more vulnerabilities. With at least 12 packages with critical vulnerabilities.

For example, here we see that the bash package has 3 critical vulnerabilities two of which date back to 2014.

Let’s review the first vulnerability: CVE-2014-6277:

While the CVE database does show that bash prior to version 4.3 is vulnerable to this CVE Red Hat’s analysis of the fixes applied in their release states the following:

Red Hat no longer considers this bug to be a security issue. The change introduced in bash errata RHSA-2014:1306, RHSA-2014:1311 and RHSA-2014:1312 removed the exposure of the bash parser to untrusted input, mitigating this problem to a bug without security impact.

Using Red Hat’s data feed allows us to benefit from their detailed analysis and provide more accurate and relevant information. While this requires that Anchore needs to add distribution specific features to our codebase the benefits far outweigh the cost.

Looking in detail through the results for CentOS in DockerStore and in Anchore we are confident that we are displaying the correct results, however, this leads to a very interesting question – how would you know if we were making a mistake?

In a previous blog, we discussed Hanlon’s Razor which states: “Never attribute to malice that which is adequately explained by stupidity.” Mistakes can obviously be made in security scanners, those mistakes could be deliberate – to purposely hide an issue for some nefarious reasons, but more likely they are just innocent mistakes. At Anchore we believe that security and analysis tools should be open source so that anyone can inspect the code to validate that the results are fair and accurate. In short, we live by the mantra “trust but verify.” But that all said, remember that CVEs are just the tip of the iceberg and you need to look far deeper into your images.

Whitelisting CVE’s

Posted on by Christian Wiens

In last week’s blog, we covered how to create custom policies that can be used to evaluate your container images as part of your CI/CD pipeline or at any time during their lifetime. We explained that you should always perform a CVE scan of your container but that this is only the first step, in fact security vulnerabilities in the operating system packages are just the tip of the iceberg in terms of the tests that you should be performing.

Today we want to dig a little deeper into CVEs. Let’s start by looking at the NGINX image in the Anchore Navigator. NGINX is one of the most downloaded images on DockerHub and the chances are that if you are reading this blog you are running NGINX somewhere in your environment.

The following link will take you directly to the Security tab for the image to show you the CVEs found in this image.

At first glance, looking at the summary, you will probably be concerned: 18 high-level CVEs. The chances are that your policies are configured to fail the image and stop the deployment if any high-level CVEs are found but it’s not that simple.

Let’s drill down into the details of the first two entries in the list.

Here you can see that two packages include the following vulnerability: CVE-2016-7943 and that, currently, there is no fix available in the version of Debian that this container is built from. You may be tempted to ignore any CVE for which there is not yet a fix, however, there are may not be a fix because one is not required or perhaps there is a real vulnerability but the vendor has not yet released a patch. So let’s click on the link for CVE-2016-7943 to dig a little deeper.

This CVE has been issued against the libx11 library for X.org the graphical display server.
Libx11 is present in the image as it was pulled in by dependencies (libx11 -> libxpm4 -> nginx-module-image-filter).
The National Vulnerability Database (NVD), maintained by NIST, has this categorized as high severity. As you can see from the description this vulnerability may allow remote X servers to gain higher-level privileges. At the bottom of the page you will see the following notes:

Here you can see that the Debian security team has described it as a “minor issue”.
Given the Debian security team’s assessment, not to mention the fact that there will be no remote X window connections to this server we can safely ignore this high vulnerability.
That’s 1 down, 17 more to go …….

For the sake of this example, let’s presume that all the 18 high-level CVEs aren’t exploitable in our image. While that is certainly reassuring it’s also a lot of work for a DevOps engineer to review all CVEs in an image – there’s just too much noise and mistakes are bound to be made.

At Anchore we don’t believe that we should be producing lists of issues for engineers to review, the result of a policy evaluation should be a decision, does the image pass or fail? You will see that result on the Policy tab for this image.

We have evaluated this image with our default policy and with no whitelist, hence the failure.

The Anchore engine supports the notion of whitelists that allow users to define exceptions that should be ignored. For example, if we included CVE-2016-7943 in our whitelist then those first two high vulnerabilities would not have been shown in our policy evaluation.

Let’s move the command line to continue.

We’ll start by pulling down the latest nginx image and performing an analysis using Anchore.

# docker pull nginx:latest
# anchore analyze --image=nginx:latest

Next, we’ll create a very simple policy file called mypolicy, containing only two checks – for High and Critical CVEs

ANCHORESEC:VULNCRITICAL:STOP
ANCHORESEC:VULNHIGH:STOP

And then run a gate check against the image

# anchore gate --image=nginx:latest --policy=mypolicy
+--------------+------------------------+------------+----------+-----------------------------+-------------+
| Image Id     | Repo Tag               | Gate       | Trigger  | Check Output                | Gate Action |
+--------------+------------------------+------------+----------+-----------------------------+-------------+
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libx11-data       |             |
|              |                        |            |          | (CVE-2016-7943 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-7943)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libx11-6          |             |
|              |                        |            |          | (CVE-2016-7943 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-7943)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libc-bin          |             |
|              |                        |            |          | (CVE-2014-9761 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2014-9761)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - multiarch-support |             |
|              |                        |            |          | (CVE-2014-9761 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2014-9761)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libc6             |             |
|              |                        |            |          | (CVE-2014-9761 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2014-9761)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libx11-data       |             |
|              |                        |            |          | (CVE-2016-7942 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-7942)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libx11-6          |             |
|              |                        |            |          | (CVE-2016-7942 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-7942)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libxml2           |             |
|              |                        |            |          | (CVE-2016-4448 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-4448)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libvpx1           |             |
|              |                        |            |          | (CVE-2015-1258 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2015-1258)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libtiff5          |             |
|              |                        |            |          | (CVE-2015-7554 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2015-7554)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libtiff5          |             |
|              |                        |            |          | (CVE-2016-9535 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-9535)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libxml2           |             |
|              |                        |            |          | (CVE-2016-1761 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2016-1761)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability found in | STOP        |
|              |                        |            |          | package - libtiff5          |             |
|              |                        |            |          | (CVE-2017-5225 - https      |             |
|              |                        |            |          | ://security-tracker.debian. |             |
|              |                        |            |          | org/tracker/CVE-2017-5225)  |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | FINAL      | FINAL    |                             | STOP        |
+--------------+------------------------+------------+----------+-----------------------------+-------------+

For the sake of this example let’s presume that all of these CVEs have been analyzed and based on the results whitelisted. We’ll create a file called mywhitelist that contains a line for each unique CVE along with the name of the gate: ANCHORESEC

Eg.

ANCHORESEC CVE-2014-9761
ANCHORESEC CVE-2015-1258
ANCHORESEC CVE-2015-7554
ANCHORESEC CVE-2016-1761
ANCHORESEC CVE-2016-2779
ANCHORESEC CVE-2016-3881
ANCHORESEC CVE-2016-4448
ANCHORESEC CVE-2016-6711
ANCHORESEC CVE-2016-6712
ANCHORESEC CVE-2016-7942
ANCHORESEC CVE-2016-7943
ANCHORESEC CVE-2016-9535
ANCHORESEC CVE-2017-0393
ANCHORESEC CVE-2017-5225

Now if we run the gate analysis passing the whitelist we’ll see a very different result.

# anchore gate --image=nginx:latest --policy=mypolicy --global-whitelist=mywhitelist
+--------------+------------------------+-------+---------+--------------+-------------+
| Image Id     | Repo Tag               | Gate  | Trigger | Check Output | Gate Action |
+--------------+------------------------+-------+---------+--------------+-------------+
| 5e69fe4b3c31 | docker.io/nginx:latest | FINAL | FINAL   |              | GO          |
+--------------+------------------------+-------+---------+--------------+-------------+

The gate command supports a –show-whitelisted flag that shows which allows a user to see which items were whitelisted and from which whitelist.

# anchore gate --image=nginx:latest --policy=mypolicy --global-whitelist=mywhitelist --show-whitelisted
+--------------+------------------------+------------+----------+-------------------------+-------------+-------------+
| Image Id     | Repo Tag               | Gate       | Trigger  | Check Output            | Gate Action | Whitelisted |
+--------------+------------------------+------------+----------+-------------------------+-------------+-------------+
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libx11-data             |             |             |
|              |                        |            |          | (CVE-2016-7943 - https  |             |             |
|              |                        |            |          | ://security-tracker.deb |             |             |
|              |                        |            |          | ian.org/tracker/CVE-201 |             |             |
|              |                        |            |          | 6-7943)                 |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libx11-6 (CVE-2016-7943 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-7943)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libvpx1 (CVE-2017-0393  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2017-0393)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libc-bin (CVE-2014-9761 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2014-9761)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | multiarch-support       |             |             |
|              |                        |            |          | (CVE-2014-9761 - https  |             |             |
|              |                        |            |          | ://security-tracker.deb |             |             |
|              |                        |            |          | ian.org/tracker/CVE-201 |             |             |
|              |                        |            |          | 4-9761)                 |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libc6 (CVE-2014-9761 -  |             |             |
|              |                        |            |          | https://security-tracke |             |             |
|              |                        |            |          | r.debian.org/tracker/CV |             |             |
|              |                        |            |          | E-2014-9761)            |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libx11-data             |             |             |
|              |                        |            |          | (CVE-2016-7942 - https  |             |             |
|              |                        |            |          | ://security-tracker.deb |             |             |
|              |                        |            |          | ian.org/tracker/CVE-201 |             |             |
|              |                        |            |          | 6-7942)                 |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libx11-6 (CVE-2016-7942 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-7942)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libvpx1 (CVE-2016-6711  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-6711)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libvpx1 (CVE-2016-6712  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-6712)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libxml2 (CVE-2016-4448  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-4448)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libvpx1 (CVE-2015-1258  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2015-1258)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libvpx1 (CVE-2016-3881  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-3881)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libtiff5 (CVE-2015-7554 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2015-7554)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libtiff5 (CVE-2016-9535 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-9535)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | util-linux              |             |             |
|              |                        |            |          | (CVE-2016-2779 - https  |             |             |
|              |                        |            |          | ://security-tracker.deb |             |             |
|              |                        |            |          | ian.org/tracker/CVE-201 |             |             |
|              |                        |            |          | 6-2779)                 |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libxml2 (CVE-2016-1761  |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2016-1761)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | ANCHORESEC | VULNHIGH | High Vulnerability      | STOP        | global      |
|              |                        |            |          | found in package -      |             |             |
|              |                        |            |          | libtiff5 (CVE-2017-5225 |             |             |
|              |                        |            |          | - https://security-trac |             |             |
|              |                        |            |          | ker.debian.org/tracker/ |             |             |
|              |                        |            |          | CVE-2017-5225)          |             |             |
| 5e69fe4b3c31 | docker.io/nginx:latest | FINAL      | FINAL    |                         | GO          | none        |
+--------------+------------------------+------------+----------+-------------------------+-------------+-------------+

Whitelisting with Jenkins Plugin

The latest version of the Anchore plugin for Jenkins (version 1.0.9) includes the ability to pass a whitelist file to Anchore to the custom policy file. Using this mechanism you can include a whitelist in the workspace of your project that will be automatically picked up at analysis time.

To update to the latest version of Anchore login to the Jenkins web interface and select:

Manage Jenkins -> Manage Plugins

Press the “Check now button” to ensure that you have the latest plugin metadata.

From the “Updates” tab ensure that you’re upgrading to the latest Anchore plugin – at least version 1.0.9.

To use whitelists within your Jenkins project go to the Anchore Container Image Scanner build step where you will see a similar screen as shown above including the new “Global White list file” entry field.

From this screen press the “Save” button the new setting will not be honored until Save is pressed once.

The upcoming Anchore 2.0 release will support the graphical creation of whitelists and policies along with the ability to define a mapping file that allows the user to define which policies and whitelists are used for any given image based on its registry, repo name and tag.

In our next blog, we’ll dig deeper into advance policy and whitelist options as well as discussing curated whitelists.

Becoming a Container Security Champion

Posted on by Joel Stags

Since we released Anchore’s open source project almost a year ago we’ve seen fast-growing adoption by users who want to perform detailed inspection and analysis of their container images. By far the most common use case we see with our users is deploying Anchore within their continuous integration and deployment pipelines (CI/CD) especially with Jenkins.

In some of the recent events we’ve attended it’s been great to talk to end-users who are already using Anchore. We’ve heard a pretty consistent message in the conversations we’ve had:

Developers love Docker and it’s already a vital part of their development process and they are either already deploying Docker in production or are planning to do so. What we hear from operations and security folks is often a little different! We hear talks of ‘Shadow IT’ and unmanaged deployments. Right now it seems like the security and operations teams are racing to catch up with development.

Rather than trying to slow things down, most of the operations teams that we talk to want to just “get out of the way and let developers innovate” but they need to balance this with their organization’s needs around security and compliance.

Based on the experience we have already built with organizations in addressing these issues, today we are launching a new offering that we call Anchore Champion that provides a combination of services and support to jumpstart the process of securing an organization’s CI/CD pipeline and adding compliance and governance into their DevOps environment.

The Anchore Champion service begins with a container policy and compliance working session where we, virtually, get together with all the stakeholders: developers, operations and security to work through their requirements for compliance and then build a set of sample policies and whitelists that encompass their needs.

Next, we provide support for architecting a secure container build environment – helping integrate Anchore into a Jenkins or other CI/CD pipeline.

And we provide ongoing support for creating policies, configuration and general operation of Anchore.

Creating Policies

Posted on by Christian Wiens

At the heart of Anchore’s solution is the concept of users certifying container images based on rules that they define. In the past certifications for applications typically came from operating systems vendors who defined their own standards and worked with independent software vendors (ISVs) on certification programs to give a level of assurance to end users that the application was compatible with the underlying operating system. Other organizations have created standards and certification tests to cover various forms of compliance validation, especially in the government sector or regulated industries.

Today the problem is more complex and there can no longer be just a single certification. For example, the requirements of a financial services company are different from the requirements of a healthcare company handling medical records and these are different from the needs of a federal institution and so on.

Anchore believes that rather than having certification in the hands of a small number of vendors or standards bodies, we want to allow organizations to define what certification means to them. In effect, we want to democratize certification.

Today we are seeing the baseline feature set for container security is a CVE scan and that’s certainly required but it’s just the first step.I’m sure the policies and you have in place today for your traditional deployments are more than just ensuring that you’ve updated all operating system packages.

These policies could cover security, starting with the ubiquitous CVE scan but then going further to analyze the configuration of key security components, for example, you could have the latest version of the apache webserver but have configured the wrong set of TLS Ciphers suites leading to insecure communication. Outside of security, policies could cover application-specific configurations to comply with best practices or to enable consistency and predictability.

In this blog, we will walk through some sample policies and cover how users can customize these policies as well as create and share their own policies.

Let’s start by looking at the policy evaluation of a test image. We will be using the anchore gate command. In Anchore’s terminology gates are checks that are run on images as they pass through the CI/CD pipeline or later when performing an evaluation on existing images.

Instead of using the default policy, we will use a customized policy called “basic-policy” that will be loaded by the CLI.

# anchore gate --image=testimage --policy=basic-policy
+--------------+------------------+-----------------+---------------+-------------------------+-------------+
| Image Id     | Repo Tag         | Gate            | Trigger       | Check Output            | Gate Action |
+--------------+------------------+-----------------+---------------+-------------------------+-------------+
| 9ebc746ba558 | testimage:latest | DOCKERFILECHECK | NOHEALTHCHECK | Dockerfile does not     | WARN        |
|              |                  |                 |               | contain any HEALTHCHECK |             |
|              |                  |                 |               | instructions            |             |
| 9ebc746ba558 | testimage:latest | PKGBLACKLIST    | PKGNAMEMATCH  | Package is blacklisted: | STOP        |
|              |                  |                 |               | openssh-server          |             |
| 9ebc746ba558 | testimage:latest | ANCHORESEC      | VULNMEDIUM    | Medium Vulnerability    | WARN        |
|              |                  |                 |               | found in package -      |             |
|              |                  |                 |               | bind-license            |             |
|              |                  |                 |               | (RHSA-2017:0276 - https |             |
|              |                  |                 |               | ://rhn.redhat.com/errat |             |
|              |                  |                 |               | a/RHSA-2017-0276.html)  |             |
| 9ebc746ba558 | testimage:latest | ANCHORESEC      | VULNMEDIUM    | Medium Vulnerability    | WARN        |
|              |                  |                 |               | found in package -      |             |
|              |                  |                 |               | openssl-libs            |             |
|              |                  |                 |               | (RHSA-2017:0286 - https |             |
|              |                  |                 |               | ://rhn.redhat.com/errat |             |
|              |                  |                 |               | a/RHSA-2017-0286.html)  |             |
| 9ebc746ba558 | testimage:latest | ANCHORESEC      | VULNMEDIUM    | Medium Vulnerability    | WARN        |
|              |                  |                 |               | found in package - vim- |             |
|              |                  |                 |               | minimal (RHSA-2016:2972 |             |
|              |                  |                 |               | - https://rhn.redhat.co |             |
|              |                  |                 |               | m/errata/RHSA-2016-2972 |             |
|              |                  |                 |               | .html)                  |             |
| 9ebc746ba558 | testimage:latest | ANCHORESEC      | VULNHIGH      | High Vulnerability      | STOP        |
|              |                  |                 |               | found in package -      |             |
|              |                  |                 |               | bind-license            |             |
|              |                  |                 |               | (RHSA-2017:0062 - https |             |
|              |                  |                 |               | ://rhn.redhat.com/errat |             |
|              |                  |                 |               | a/RHSA-2017-0062.html)  |             |
| 9ebc746ba558 | testimage:latest | IMAGECHECK      | BASEOUTOFDATE | Image base image        | WARN        |
|              |                  |                 |               | (docker.io/acathrow     |             |
|              |                  |                 |               | /aic-test:1a) ID is     |             |
|              |                  |                 |               | (9ebc746ba558), but the |             |
|              |                  |                 |               | latest ID for           |             |
|              |                  |                 |               | (docker.io/acathrow     |             |
|              |                  |                 |               | /aic-test:1a) is        |             |
|              |                  |                 |               | (f3e982542816)          |             |
| 9ebc746ba558 | testimage:latest | FINAL           | FINAL         |                         | STOP        |
+--------------+------------------+-----------------+---------------+-------------------------+-------------+

Here the output is formatted in a tabular view for a command line user to read however if you want to automate the processing of the output then the anchore command supports a --json or --plain command line option to output the results in a format that is easily parsed by other tools.

The most important part of the output is the last line that indicates that the final policy evaluation is “STOP”. Anchore gates will output one of three actions:

GO: The gate is open and the image should be allowed to pass through to the next stage.
STOP: The gate is closed and the image should not proceed
WARN: The gate is open and should proceed to the next step however warnings have been raised that should be reviewed.

If you are automating the use of Anchore from the command line then the return code from the anchore command can be used to evaluate the status: 0 = Go, 1 = Stop, 2 = Warn.

Looking at the output of the policy evaluation we can see that two policy checks outputted a “STOP” action. The first was due to a blacklisted package being present in the image and the second was due to a high-level CVE vulnerability.

Next, we’ll take a look at the policy.

DOCKERFILECHECK:NOTAG:STOP
DOCKERFILECHECK:NOFROM:STOP
DOCKERFILECHECK:NOHEALTHCHECK:WARN
DOCKERFILECHECK:EXPOSE:STOP:DENIEDPORTS=22
SUIDDIFF:SUIDFILEDEL:GO
SUIDDIFF:SUIDMODEDIFF:STOP
SUIDDIFF:SUIDFILEADD:STOP
IMAGECHECK:BASEOUTOFDATE:WARN
PKGBLACKLIST:PKGNAMEMATCH:STOP:BLACKLIST_NAMEMATCH=openssh-server
ANCHORESEC:FEEDOUTOFDATE:STOP:MAXAGE=2
ANCHORESEC:UNSUPPORTEDDISTRO:STOP
ANCHORESEC:VULNCRITICAL:STOP
ANCHORESEC:VULNHIGH:STOP
ANCHORESEC:VULNMEDIUM:WARN

The policy file lines are of the following format:

Gate name : Trigger : Action : Optional Parameters

You can consider a Gate as a family of checks that can be performed.

Checks can raise triggers that may have parameters.

Once a trigger is raised then an action (GO, STOP or WARN) is defined.

For example, looking at the following snippet from this simple policy:

DOCKERFILECHECK:NOHEALTHCHECK:WARN

Within the Dockerfilecheck gate there is a that looks for a healthcheck statement in the Dockerfile. If no healthcheck statement is found then the NOHEALTHCHECK trigger is raised.

In this example, we have configured Anchore to raise a warning if the Dockerfile does not include a health check.
In the next example we’ll blacklist two packages:

PKGBLACKLIST:PKGNAMEMATCH:STOP:BLACKLIST_NAMEMATCH=openssh-server,foolib

Here we have configured the PKGNAMEMATCH trigger to issue a STOP action if either openssh-server or foolib is present in the image.

The PKGBLACKLIST gate has two triggers : PKGFULLMATCH which matches both a package name and version and PKGNAMEMATCH which matches just the name of the package.

Anchore comes pre-installed with a number of policy modules that can be extended by the user and we regularly add new modules.

You can retrieve a full list of available policy options by running the following command:

# anchore gate --show-policytemplate

This will output a sample policy including all available gates including all available parameters.

The policy template option is useful for providing a policy file that you can further customize.

To get more detailed descriptions of gates, triggers and configuration options you can run the
following command:

# anchore gate --show-gatehelp
PKGCHECK:
   PKGNOTPRESENT:
     description: 'triggers if the package(s) specified in the params are not installed
       in the container image.  PKGFULLMATCH param can specify an exact match (ex:
       "curl|7.29.0-35.el7.centos").  PKGNAMEMATCH param can specify just the package
       name (ex: "curl").  PKGVERSMATCH can specify a minimum version and will trigger
       if installed version is less than the specified minimum version (ex: zlib|0.2.8-r2)'
     params: PKGFULLMATCH,PKGNAMEMATCH,PKGVERSMATCH

In this snippet, you can see the configuration options for the Package Check gate which allows a policy to specify that certain packages should be installed in the image. In addition to checking for the presence of a package, the user can configure minimum required versions.

We have set up a git repository to make it easier to share sample policies and allow the community to collaborate. You can access the repository here.

In our next blog, we’ll cover whitelists and show how you can reduce some of the noise that is often seen in CVE scans by whitelisting vulnerabilities that are not exploitable in your image.

Microservices -vs- MicroVMs

Posted on by Christian Wiens

At Anchore we spend a whole lot of time looking at container images to provide detailed analysis and certification. Most of the discussions we hear in the industry around image analysis focus on CVE scanning: how many CVEs are in an image, what severity, etc. As we’ve mentioned before, we see CVE scanning as just the tip of the iceberg and that it’s possible to have all the latest operating system packages but still have an image that has security vulnerabilities or is otherwise not compliant with your operational, security or business policies.

There is another common issue in the tens of thousands of images that we’ve analyzed which we feel is more fundamental. As an industry we are moving to an architecture based on microservices and containers are really the key to enabling this. While the containers we’ve seen are often designed to run microservices, I’d argue that the majority of containers we see (both on DockerHub as well as our customer’s private images) are more like MicroVMs than Microservices. These images typically have a hundred or more packages and several thousand files.

In most cases, the images are general-purpose operating system images and differ only from their virtual machine brethren by not having a kernel installed. There has been much debate in the industry about image size and how smaller is better, allowing images to be rapidly deployed over the network. Others argue that size doesn’t matter and that the layered nature of Docker’s image format and caching largely mitigates this issue, but looking just at the size of the image doesn’t give you the complete picture.

While it’s certainly an important point to consider the real concern should be not the size of the image, but the content of the images.

Let’s take Alpine as an example. Let’s use the Anchore Navigator to view the contents and select the files tab to drill down further. Filtering this list to show the files in /bin highlights just how many executables are in the image. In a microservice, why does my image need utilities for process or file management? These come from having the busybox package in the image. While that may certainly be useful in some use cases I’d argue that having these kinds of binaries in an image that never directly calls them is an accident waiting to happen. I don’t mean to pick on Alpine which weighs in at 4MB (twenty-five times smaller than most base operating system images and certainly has less attack surface) but the point you must consider is that you must ensure that every artifact in your image serves a purpose and goes through some form of quality control to ensure that the final image is secure and meets your operational best practices.

Last month Oracle released a slimmed down Oracle Linux image which reduced the footprint down from 225MB to 114MB, you can read our analysis here, this week Red Hat upped the ante when they announced a slimmed down Red Hat Enterprise Linux Atomic Base Image.

The new RHEL image weighs in at 75MB, compared to 192MB for the standard RHEL image. In this image, Red Hat has removed a number of packages that are deemed not necessary for container deployments, two of the most interesting removals are systemd and Python. Traditionally all RHEL installs have included python since the YUM package manager is written in Python. To get around this Red Hat has created a new mini package manager called microdnf. While microdnf is not as functional as YUM or as DNF, the next generation package manager for RHEL based distributions, it does just what is needed: install, remove and update packages.

I wanted to look at what else changed in the image so I pulled the RHEL Atomic image from Red Hat’s registry. If you don’t have access to the RHEL registry, you can take a look at the analysis of the image using the Anchore navigator here:

Note: This image is not available publicly on DockerHub.

For the rest of the analysis, I’m going to use Anchore’s command line tools.

First I need to analyze the image.

# anchore analyze --image=registry.access.redhat.com/rhel7-atomic

I’ve already analyzed the standard rhel 7 image so now I want to run a query to compare the packages installed in the RHEL Atomic image with the standard RHEL image using the show-pkg-diffs query.

# anchore query --image=registry.access.redhat.com/rhel7 show-pkg-diffs registry.access.redhat.com/rhel7-atomic

 

Package RHEL 7 RHEL Atomic
python-chardet 2.2.1-1.el7_1 Not Installed
librhsm Not Installed 0.0.1-1.el7
yum-plugin-ovl 1.1.31-40.el7 Not Installed
libuser 0.60-7.el7_1 Not Installed
json-glib Not Installed 1.0.2-1.el7
python-urlgrabber 3.10-8.el7 Not Installed
libblkid 2.23.2-33.el7 Not Installed
audit-libs 2.6.5-3.el7_3.1 Not Installed
libsolv Not Installed 0.6.20-5.el7
xz 5.2.2-1.el7 Not Installed
file-libs 5.11-33.el7 Not Installed
rpm-build-libs 4.11.3-21.el7 Not Installed
python-libs 2.7.5-48.el7 Not Installed
qrencode-libs 3.4.1-3.el7 Not Installed
gdbm 1.10-8.el7 Not Installed
cryptsetup-libs 1.7.2-1.el7 Not Installed
dbus-libs 1.6.12-17.el7 Not Installed
tar 1.26-31.el7 Not Installed
dbus-glib 0.100-7.el7 Not Installed
cracklib-dicts 2.9.0-11.el7 Not Installed
kmod 20-9.el7 Not Installed
systemd 219-30.el7_3.7 Not Installed
subscription-manager 1.17.15-1.el7 Not Installed
libpwquality 1.2.3-4.el7 Not Installed
pygpgme 0.3-9.el7 Not Installed
python-dmidecode 3.10.13-11.el7 Not Installed
pyliblzma 0.5.3-11.el7 Not Installed
device-mapper 1.02.135-1.el7_3.3 Not Installed
kmod-libs 20-9.el7 Not Installed
shadow-utils 4.1.5.1-24.el7 Not Installed
python-pycurl 7.19.0-19.el7 Not Installed
libcap-ng 0.7.5-4.el7 Not Installed
python-rhsm-certificates 1.17.9-1.el7 Not Installed
kpartx 0.4.9-99.el7_3.1 Not Installed
python-iniparse 0.4-9.el7 Not Installed
microdnf Not Installed 2-3.el7.1.1
pam 1.1.8-18.el7 Not Installed
cracklib 2.9.0-11.el7 Not Installed
procps-ng 3.3.10-10.el7 Not Installed
pyxattr 0.5.1-5.el7 Not Installed
vim-minimal 7.4.160-1.el7_3.1 Not Installed
python 2.7.5-48.el7 Not Installed
python-rhsm 1.17.9-1.el7 Not Installed
python-ethtool 0.8-5.el7 Not Installed
cpio 2.11-24.el7 Not Installed
libutempter 1.1.6-4.el7 Not Installed
device-mapper-libs 1.02.135-1.el7_3.3 Not Installed
systemd-libs 219-30.el7_3.7 Not Installed
dmidecode 3.0-2.el7 Not Installed
m2crypto 0.21.1-17.el7 Not Installed
hardlink 1.0-19.el7 Not Installed
rpm-python 4.11.3-21.el7 Not Installed
yum-utils 1.1.31-40.el7 Not Installed
dbus-python 1.1.1-9.el7 Not Installed
python-dateutil 1.5-7.el7 Not Installed
librepo Not Installed 1.7.16-1.el7
util-linux 2.23.2-33.el7 Not Installed
usermode 1.111-5.el7 Not Installed
yum-metadata-parser 1.1.4-10.el7 Not Installed
pygobject3-base 3.14.0-3.el7 Not Installed
dracut 033-463.el7 Not Installed
rootfiles 8.1-11.el7 Not Installed
ustr 1.0.4-16.el7 Not Installed
elfutils-libs 0.166-2.el7 Not Installed
diffutils 3.3-4.el7 Not Installed
dbus 1.6.12-17.el7 Not Installed
libuuid 2.23.2-33.el7 Not Installed
gdb-gdbserver 7.6.1-94.el7 Not Installed
libmount 2.23.2-33.el7 Not Installed
libxml2-python 2.9.1-6.el7_2.3 Not Installed
yum 3.4.3-150.el7 Not Installed
virt-what 1.13-8.el7 Not Installed
libdnf Not Installed 0.7.4-2.el7.el
libsemanage 2.5-5.1.el7_3 Not Installed
gzip 1.5-8.el7 Not Installed
passwd 0.79-4.el7 Not Installed
python-kitchen 1.1.1-5.el7 Not Installed
libnl 1.1.4-3.el7 Not Installed
binutils 2.25.1-22.base.el7 Not Installed
acl 2.2.51-12.el7 Not Installed

Here you’ll see there are 80 package differences. Six packages have been added to support the new package manager: librhsm, json-glib, libsolv, microdnf, librepo and libdnf.74 packages have been removed leaving just the minimum set of packages.

Out of interest, I wanted to see how this package list differed from Oracle’s slim image.

Package Oracle Linux Slim RHEL Atomic
python-chardet 2.2.1-1.el7_1 Not Installed
nss-tools 3.21.3-2.0.1.el7_3 3.21.3-2.el7_3
python-urlgrabber 3.10-8.el7 Not Installed
libxml2 2.9.1-6.0.1.el7_2.3 2.9.1-6.el7_2.3
audit-libs 2.6.5-3.el7 Not Installed
nss-sysinit 3.21.3-2.0.1.el7_3 3.21.3-2.el7_3
file-libs 5.11-33.el7 Not Installed
rpm-build-libs 4.11.3-21.el7 Not Installed
python-libs 2.7.5-48.0.1.el7 Not Installed
json-glib Not Installed 1.0.2-1.el7
gdbm 1.10-8.el7 Not Installed
nss 3.21.3-2.0.1.el7_3 3.21.3-2.el7_3
pyxattr 0.5.1-5.el7 Not Installed
yum-plugin-ovl 1.1.31-40.el7 Not Installed
basesystem 10.0-7.0.1.el7 10.0-7.el7
pygpgme 0.3-9.el7 Not Installed
coreutils 8.22-18.0.1.el7 8.22-18.el7
shadow-utils 4.1.5.1-24.el7 Not Installed
python-pycurl 7.19.0-19.el7 Not Installed
libcap-ng 0.7.5-4.el7 Not Installed
bash 4.2.46-21.0.1.el7_3 4.2.46-21.el7_3
python-iniparse 0.4-9.el7 Not Installed
microdnf Not Installed 2-3.el7.1.1
librhsm Not Installed 0.0.1-1.el7
kernel-container 3.10.0-0.0.0.2.el7 Not Installed
gobject-introspection Not Installed 1.42.0-1.el7
python 2.7.5-48.0.1.el7 Not Installed
cpio 2.11-24.el7 Not Installed
yum-utils 1.1.31-40.el7 Not Installed
pyliblzma 0.5.3-11.el7 Not Installed
rpm-python 4.11.3-21.el7 Not Installed
librepo Not Installed 1.7.16-1.el7
yum-metadata-parser 1.1.4-10.el7 Not Installed
libsolv Not Installed 0.6.20-5.el7
ustr 1.0.4-16.el7 Not Installed
oraclelinux-release 7.3-1.0.4.el7 Not Installed
diffutils 3.3-4.el7 Not Installed
redhat-release-server 7.3-7.0.1.el7 7.3-7.el7
libxml2-python 2.9.1-6.0.1.el7_2.3 Not Installed
yum 3.4.3-150.0.1.el7 Not Installed
libdnf Not Installed 0.7.4-2.el7.el
libsemanage 2.5-5.1.el7_3 Not Installed
python-kitchen 1.1.1-5.el7 Not Installed
gpg-pubkey ec551f03-53619141 Not Installed

Ignoring the version differences, there are 7 packages in RHEL Atomic not present in the Oracle Slim image which support the new microdnf package manager. There are 28 packages in Oracle Slim that are not in the RHEL Atomic image – unsurprisingly most of these relate to the inclusion of YUM. It will be interesting to see if Oracle Linux and the other RHEL derivatives follow suit and use microdnf in their images.

This is a great step forward for RHEL users, reducing the image size and the attack surface but still leaves a lot of, arguably, unnecessary content in the image. Take a look at the files view in the content tab of the Anchore Navigator for this image here and, as we did for Alpine earlier, filter for /bin to see the utilities and other libraries installed in the image.

At this point, the challenge in reducing the image further is that most of the packages left are required either in whole or more likely in part due to dependencies. For example, you could argue that there is no good reason to have the /bin/chmod command in the image however that is part of the coreutils package which is required by multiple other packages so any further steps forward will require some major changes in packaging.

If you have a Red Hat Enterprise Linux subscription I’d encourage you to check out the new Atomic base image and see how you can reduce the footprint and attack surface of your RHEL based images.

And whether you use RHEL, CentOS, Debian, Ubuntu, Alpine or other distributions you can use Anchore’s image analysis and compliance tools to ensure that the images you deploy meet your security and best practices requirements.

Improved Jenkins Integration

Posted on by Christian Wiens

Today we have released an update to our popular open source Jenkins plugin adding a number of powerful new features.

Using Anchore’s freely available and open source Jenkins plugin you can secure your Jenkins pipeline in less than 30 minutes adding image scanning including not just CVE based security scans but policy-based scans that can include checks around security, compliance and operational best practices.

The first new feature to highlight is an updated user interface that improves both the aesthetic and the functionality of the UI. In the first screenshot below you can see that while the build has succeeded we have raised a number of warnings.

  •     The container was built from a base image with the tag latest rather than from a specific named tag
  •     The Dockerfile does not include any health check instructions which would simplify ongoing monitoring of the service.
  •     The acme-logging package has not been installed which is a recommended package for this organization.

Policies are customizable, along with whitelists, and are typically defined by the Security or Operations team.

The Policy evaluation summary is always produced by the Anchore plugin, however, there are other reports that a user can define to be run during the CI/CD pipeline.

In the first example, you can see a package manifest that has been produced – both in the form of a searchable web interface but also as a JSON file in the Jenkins project workspace that contains machine-readable output.

In the final example, we see a report detailing the difference in packages between the base image and the final image produced by the build.


Select:   Manage Jenkins > Manage Plugins > Updates
If you are already running the Anchore Jenkins plugin then you can update the Anchore plugin directly from the Jenkins web interface. At the time of writing the latest version of the plugin is version 1.0.7.

If you are not running Anchore’s plugin, there is detailed instructions on the following page.

The second interesting new feature is support for Jenkins Pipelines. In our previous examples, we have illustrated the use of Anchore within a Jenkins Freestyle Project which is the traditional way of architecting a Jenkins build, using the Jenkins web interface to define projects, adding build steps, scripts, etc.

In the Pipeline model, the entire build process is defined as code in a Jenkinsfile. This file can be created, edited and managed in the same way as any other artifact of your software project. For example, you can check your pipeline definition into your source control system, dynamically create the build instructions based on the configuration of your application or perform countless other forms of automation.

Pipeline builds can be more complex including forks/joins and parallelism. The pipeline is more resilient and can survive the master node failure and restarts. Pipelines are written in Groovy scripts and to add an Anchore scan you need to add the following simple code snippet.

node {
def imageLine = IMAGE + ' ' + env.WORKSPACE + '/DockerFile'
writeFile file: 'anchore_images', text: imageLine
anchore name: 'anchore_images', policyName: 'anchore_policy', bailOnFail: false, inputQueries: [[query: 'list-packages all'], [query: 'cve-scan all']]
}

Here the IMAGE is the ID of the container image that was just created. This could be in the form of an image ID (short or long-form), for example, 67591570dd29. Or the REPO/TAG can be used -for example, webapp/frontend:123456.

This code snippet writes out the anchore_images file that is used by the plugin to define which images are to be scanned.

The Dockerfile is read from the project workspace as is the file containing the policy that you wish to evaluate against the image, in this case, we have called the policy file anchore_policy and have stored this file in the project’s workspace.

This code snippet can be crafted by hand or built using the Jenkins UI.

Select:  Pipeline Syntax from the Project

This will launch the Snippet Generator where you can enter the required parameters and press the Generate Pipeline Script button which will produce the required snippet.

It’s quick and easy to add Image scanning and policy to your Jenkins project and we’re here to help.

If you have any questions or would like to learn more you can join our slack channel by clicking the button below or fill out the form to send us a direct message.

Updates to Anchore Open Source Project

Posted on by Tim Gerla

What’s going on in the world of Anchore’s open source platform? As you might know, Anchore has an online container image navigator that provides unique visibility into the contents of container images–our system is constantly watching for updates to public container repositories, and runs a series of comprehensive analyses for every new revision. You can see how containers change over time, what packages and files have been installed, and if any known security vulnerabilities have been fixed or introduced. In our most recent update, we’ve added features to let you subscribe to images you are particularly interested in and request that we scan specific images that we may not already be processing.

Underneath the web UI that we host, that functionality is driven by our open source Anchore Engine, which you can run locally to do a wide variety of queries on your on-premise container images. There is a lot of functionality built into the tools: package queries for a number of different packaging formats including RPM, dpkg, Ruby Gem, and Node packages, and scans for known security vulnerabilities. There is also a “multitool” called anchore-toolbox that can show you a variety of other information about your containers, including the Dockerfile used to create the image, its family tree relative to other containers, and it can simply unpack a container into a directory on the filesystem for troubleshooting or examination by other tools.

We are also working on improvements to the packaging and deployment of our OSS tools. We are going to be expanding API access for easier integration into different deployment pipelines, and shipping a pre-built container image for multi-user environments. Stay tuned for updates on this effort: you can watch our progress on our GitHub page. If you’re interested in contributing to the project, here are some ways to get started:

Finally, we are planning on continuing our efforts to build useful integrations with third-party tools such as Jenkins. We know that a successful production container pipeline is made up of at least several components, and we want it to be as easy as possible to connect Anchore’s analysis and gating functionality into your own environment and make sure that every container you ship to production is safe, secure, and configured appropriately. Check out a detailed intro to our Jenkins plugin for more info.

We know that an open source project’s success depends on its community, so we want to hear from you. Stop by and say Hi, and we hope you enjoy using Anchore for all of your container analysis needs!