Lately it seems like a new company building hardened container images is popping up every other day. What’s the deal with this, why the sudden influx of hardened images?
A previous blog article titled “Navigating the New Compliance Frontier” discussed some of the new trends with compliance. But it’s not as simple as just claiming “because compliance”–that’s the easy answer that doesn’t tell us much. Compliance doesn’t say anything about using hardened container images. But here’s the thing: many compliance standards do have things to say about configuration management, attack surface, and vulnerabilities. We’re not being told to use hardened images, but hardened images solve many of these problems, so they are getting used. It’s one of those situations where the practical solution emerges not because someone mandated it, but because it actually works.
When you operate in a regulated space (as we all will be soon, thanks to the CRA) you have to justify configuration changes, software changes, and every vulnerability. The idea behind the hardened images is to have only the software you absolutely need and nothing else. That translates into you only having a list of the vulnerabilities that directly affect you.
There’s an additional reason that’s giving small hardened images extra attention. The ability to scan your software for vulnerabilities is better than it’s ever been. Historically trying to scan software for vulnerabilities wasn’t very reliable or even a good use of anyone’s time. Scanners of the past missed a lot of things. It was a lot harder to figure out what all the software installed on bare metal, VMs, or containers. And by harder we really mean it was nearly impossible. It was common to just fall back to spreadsheets with people doing this all manually.
Scanning for vulnerabilities was basically out of the question, nobody had a decent inventory of their software supply chain. The scanners also had incomplete data if they even had data at all. The false positive rate made the results useless and the false negative rate made those useless results dangerous. It was also common to scan a system and just get no results at all. Not because there are no vulnerabilities, but because the scanners couldn’t figure anything out.
That’s not true anymore. If you scan any modern Linux distribution you’re going to get results. Language ecosystems like Python, Java, and Go are well supported. A lot of scanners can even figure out what pre-built binaries are in a container image. Scanners have gotten pretty good at finding the software in a container.
Interested to learn how Anchore built Syft, a modern software composition analysis (SCA) scanner & SBOM generator, to overcome the challenges that plagued legacy scanners? Check out our behind the scene blog series:
This also means that if we have a reliable list of software, we can also scan that list for vulnerabilities. Something you’ll notice quickly is a hardened minimal image has fewer findings than a more traditional container image. This doesn’t mean a minimal container image is better than a more traditional one. They solve different problems.Here’s an example, let’s look at an Alpine image and a Debian image. We can scan the container image with the Syft SBOM scanner.
➜ ~ syft alpine:latest
✔ Parsed image
sha256:9234e8fb04c47cfe0f49931e4ac7eb76fa904e33b7f8576aec0501c085f02516
✔ Cataloged contents
eafc1edb577d2e9b458664a15f23ea1c370214193226069eb22921169fc7e43f
├── ✔ Packages [16 packages]
├── ✔ File metadata [81 locations]
├── ✔ Executables [17 executables]
└── ✔ File digests [81 files] ➜ ~ syft debian:latest
✔ Parsed image
sha256:999ffdddc1528999603ade1613e0d336874d34448a74db8f981c6fae4db91ad7
✔ Cataloged contents
56b68c54f22562e5931513fabfc38a23670faf16bbe82f2641d8a2c836ea30fc
├── ✔ Packages [78 packages]
├── ✔ Executables [655 executables]
├── ✔ File metadata [4,148 locations]
└── ✔ File digests [4,148 files] The Alpine image has 16 packages in it. The Debian image has 78. That’s a pretty big difference. It’s pretty clear that an Alpine image would fall under the umbrella of a minimal hardened image while Debian does not.If we scan these for vulnerabilities the results are similar. The Grype vulnerability scanner does a great job when scanning both Alpine and Debian container images.
➜ ~ grype alpine:latest
✔ Vulnerability DB [updated]
✔ Parsed image
sha256:9234e8fb04c47cfe0f49931e4ac7eb76fa904e33b7f8576aec0501c085f02516
✔ Cataloged contents
eafc1edb577d2e9b458664a15f23ea1c370214193226069eb22921169fc7e43f
├── ✔ Packages [16 packages]
├── ✔ Executables [17 executables]
├── ✔ File metadata [81 locations]
└── ✔ File digests [81 files]
✔ Scanned for vulnerabilities [6 vulnerability matches]
├── by severity: 0 critical, 0 high, 0 medium, 6 low, 0 negligible➜ ~ grype debian:latest
✔ Parsed image
sha256:999ffdddc1528999603ade1613e0d336874d34448a74db8f981c6fae4db91ad7
✔ Cataloged contents
56b68c54f22562e5931513fabfc38a23670faf16bbe82f2641d8a2c836ea30fc
├── ✔ Packages [78 packages]
├── ✔ Executables [655 executables]
├── ✔ File metadata [4,148 locations]
└── ✔ File digests [4,148 files]
✔ Scanned for vulnerabilities [46 vulnerability matches]
├── by severity: 0 critical, 0 high, 1 medium, 5 low, 40 negligibleWe see 0 vulnerabilities in the Alpine image and 46 vulnerabilities in the Debian image. Now this doesn’t make Debian worse, they have limited time and fix the most dangerous vulnerabilities first. It’s 1 medium, 5 low, and 40 negligible. It’s pretty clear the Debian security folks are staying on top of all this.
Keep in mind that Debian has a lot more software than you get in an Alpine image. There are many problems that are easier to solve with Debian than Alpine because it has more software available. There are also problems that are easier to solve with Alpine because it has less software.
In the case of hardened container images, if our primary problem is justifying security findings, you’re going to have less work to do if you have fewer vulnerabilities. Less software means fewer vulnerabilities. This seems like a pretty easy logical conclusion. Even when a security vulnerability is marked as “negligible” as we see in the above Debian report, security and compliance auditors will want a justification. Those justifications come with a cost. We can remove that cost by just not having those vulnerabilities present. One way to get fewer vulnerabilities is to ship less software. This is that case where Alpine solves a problem Debian doesn’t.
Of course we could try to build our own tiny variant of Debian. Is that a problem we want to solve? The care and feeding of container base images has been solved by many other companies and open source projects. If you need a giant image full of stuff, or a small image that only contains what you need. This is one of those classic situations where you can use an existing solution to solve a problem. You don’t generate your own electricity, you pay a utility. Why are you trying to manage the operating system in your container images?
As we see more compliance standards becoming the reality for software producers. We are going to see more hardened container images emerge, these things are here to stay. They solve real problems. The thing to keep an eye on in the future is what other common problems that mandatory compliance creates?
Hardened container images aren’t the end goal of meeting your compliance requirements, they are the first step in a long journey. There are many stages to release software and services. We’re going to see increasing attention to how we build our software, what the dependencies are, what the vulnerabilities in those dependencies are. There won’t be a single simple answer for any of these new requirements. However, now that we have some requirements we can start to figure out the best way to solve these new problems.
This is exactly why Anchore recently announced a strategic partnership with Chainguard, one of the leading companies in the hardened container space. The partnership recognizes something important: starting with secure, hardened container images is just the beginning. You also need continuous scanning, policy enforcement, and compliance monitoring as you build your own code on top of those secure foundations. It’s a “Start Safe, Stay Secure and Compliant” approach that acknowledges hardened images solve the base layer problem, but you still need to manage everything else in your software supply chain.
At Anchore we don’t think we can solve every problem, but we do think the combination of starting with hardened images and applying continuous security practices like SBOMs and policy evaluations throughout your development lifecycle is the most practical way forward in this new compliance world.
Learn how Chainguard’s hardened images and Anchore Enterprise’s SBOM-powered, vulnerability scanning and policy enforcement reduce audit effort and accelerate entry into new markets.
