Author: Alex Goodman

Build Your Own Grype Database

Posted on by Alex Goodman

When running vulnerability scans against your software dependencies it’s important to have the most up to date vulnerability information that’s been published. New vulnerabilities are found all the time, the data goes stale quickly. For current Grype users, we have a daily pipeline that builds and publishes a Grype database with the latest vulnerability data. Up until now the tooling that drives this pipeline has not been available as open source since it was originally designed as an embedded aspect of Anchore’s commercial products. Today that’s changing! 

How does this help the average Grype user? By making the framework and code that are used to prepare vulnerability data sources open, the entire open source community (even you!) can contribute improvements and new vulnerability data sources, enhancing both the breadth and quality of vulnerability scanning for all.

We’re happy to announce two new open source projects: Vunnel and Grype-DB.

Vunnel (short for “vulnerability data funnel”) understands how to pull and process vulnerability from various upstream data sources, such as NVD, Github Security Advisories, and multiple Linux distribution providers. This allows you to prepare a data directory with indexed and normalized vulnerability data. This sounds simple, but all of this vulnerability data is different and varies widely in its quality and composition. Vunnel gives us some control to normalize this data in a way that gives better consistency.

Demo of Vunnel

Grype-DB builds an SQLite database that Grype can use based off of the data that Vunnel outputs. Even more, Grype-DB can invoke Vunnel in order to prepare a data directory for multiple providers, allowing you to orchestrate and tailor which providers you want to include in the database.

Running Grype-DB

This puts the entire Grype vulnerability data pipeline and surrounding tooling into the open source! This includes all of the providers that drive Grype today: Alpine, Amazon Linux, Centos, Debian, GitHub Security Advisories, NVD, Oracle Linux, RedHat Enterprise Linux, SUSE Linux Enterprise, Ubuntu, and Wolfi. Anyone can now fully participate in the data processing for the Grype ecosystem, expanding the vulnerability matching capabilities of Grype (for example, adding support for new Linux distributions in Grype). 

We’re excited to see what community contributions arise from this effort! Stay tuned for a tutorial to show you how to implement a new Vunnel provider. 

If you’d like to learn more feel free to reach out to us on our Discourse forum, drop into our community meetings for live Q&A (every other Thursday), or see the docs:

Meet Quill: A cross platform code signing tool for macOS

Posted on by Alex Goodman

We generate a lot of tooling at Anchore. We chose to write most of these tools in Go for a few reasons: the development process is delightful, cross-platform builds are easy, and the distribution of artifacts is very simple (curl the binary). 

Since we target releasing tools for macOS we are beholden to the requirements put forth by Apple, something we’ve written about at length in the past. Tools like gon have made the process of signing and notarizing our releases much easier by wrapping xcrun and codesign utilities and hiding some of the inherent complexities. However, since gon shells out to these tools you still must be on a mac to sign and notarize your binaries. This nullifies one of the reasons why we chose Go in the first place: having simple cross-platform builds from any platform. 

We’ve reworked our release process a few times over to account for this, all with unpleasant tradeoffs. It seems to come down to a couple of points:

  1. Running macOS in CI is more expensive than running on linux.
  2. Using Docker on macOS in CI is annoying. Due to licensing restrictions Docker is not included on the default mac runners. This is problematic since we use goreleaser to perform the build and release steps in one shot, which means we need to be able to sign/notarize our binaries at the same time as we package and release them for all platforms. This has only very recently been alleviated with the addition of colima on the default mac runner, but before this it has caused us to slice-and-dice up our release pipeline in awkward ways.

After a while we started to wonder to ourselves: Is it intrinsically necessary to require the signing and notarization steps to run on a mac? The more we looked the more we were certain the answer was “no”.

What’s in a signed binary anyway?

When you run codesign to sign your binary a new payload is added at the end of the binary with (usually) the following sections:

  • A Code Directory: essentially a table of hashes. Each hash is a digest of each page in the binary before the new payload. The code directory is “what” gets signed.
  • A PKCS7 (CMS) envelope: contains the cryptographic signature made against the Code Directory.
  • A Set of Requirements (optional): expressions that are evaluated against the signature that should hold true. “Designated Requirements” are a special set of requirements that describe how to determine the identity of the code being signed.
  • A Set of Entitlements (optional): a list of key-value pairs in XML that represent privileges an executable can request, e.g. com.apple.developer.avfoundation.multitasking-camera-access=true request for camera access while running alongside another foreground app.

There is nothing inherent about any of these payload elements that require the signing process to run on a mac.

What about notarization? What’s involved to get your binary notarized by Apple? This is an easier answer:

  1. Put your binary in a zip
  2. Upload the zip to Apple via their Notarization API
  3. Poll their Notarization API until there is a result

It seems that the only reason why we are signing and notarizing our releases on macOS is because Apple does not yet provide cross-platform tooling to do so…

Introducing Quill

We created a new tool called Quill to sign and notarize your macOS binary from any platform. 

This works quite well with goreleaser as a post-build step:

using quill with goreleaser

In this way you can use a single goreleaser file for local builds and production builds:

  • Signing and notarization are performed for production builds.
  • Ad-hoc signing is done for snapshot builds (notarization is skipped). This means that no cryptographic material is needed as input, so the Code Directory is added to the binary but there is no signature attached.

You can additionally use Quill to:

  • View your previous notarization submissions with “quill submission list”
  • Get the logs from Apple about the details of a submission result with “quill submission logs <submission-id>”
  • Parse and describe a macOS binary (including all signing details) with “quill describe ./path/to/binary”

We are now using quill for our production releases of Syft and Grype and have room to implement more features in the future to expand the capabilities of quill to match that of codesign. Quill is an open source project — we would love feedback, bug reports, and pull requests!