Many users have already implemented Anchore to secure their CI/CD pipeline, to ensure that only images that are compliant with their security policies are pushed to their production registries. While this is a crucial process to implement on the path to implementing strong governance of container environments this only the first step. It is not enough to just trust that the images in your production registry are compliant since while they may have passed policy checks at build time, images may fall out of compliance over time due to changes in policy or due to new vulnerabilities being discovered after the images were built and scanned. While many users rely on image signing to prove the provenance of images this does not guarantee that an image which was compliant when signed still complies with your security policies, while signing ensures that the image has not been changed policies and vulnerability data may change. The next step in securing your container pipeline is to implement a system that ensures that only images that comply with your policies are deployed on your Kubernetes cluster, or other orchestration platform. One approach to address this issue that we have seen is to provide a plugin for the Docker Daemon that is configured to communicate with a backend policy system. Commands issues to the Docker Daemon will be filtered through this system and in the case of a failed image policy the command to launch the container will be rejected. There are number of challenges with this approach. Firstly this requires installing, configuring and maintaining plugins on every node, while this is certainly possible for Kubernetes systems where you have access and control of the underlying nodes, it it becomes more complicated for hosted Kubernetes platforms where all that is exposed to the end user is the API, such as Amazon’s upcoming EKS platform. With the growing list of OCI compatible runtimes available plugins will have to be built for each runtime. The main challenge with this plugin approach is that it takes control out of the hands of the orchestration platform rather than cooperating with the orchestrator. Imagine the following scenario:
- Your application is built on a Debian base image and deployed into production
- A new high severity vulnerability is found in openssl is reported upstream.
- A CVE is created and filed against the openssl package which is present in your image.
- At this point the image you use contains a vulnerability for which a fix may not be currently available
- Your deployment needs to scale to add new replicas to handle load or perhaps compensate for a failure
- Kubernetes tries to launch a container in a new pod however the container launch is blocked by the plugin.