Skip to content

[ANE-3098] Read Go buildinfo from binaries during fossa analyze - #1757

Draft
saramaebee wants to merge 3 commits into
masterfrom
ane-3098-go-buildinfo-analyze
Draft

[ANE-3098] Read Go buildinfo from binaries during fossa analyze#1757
saramaebee wants to merge 3 commits into
masterfrom
ane-3098-go-buildinfo-analyze

Conversation

@saramaebee

@saramaebee saramaebee commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Overview

Go binaries embed the module list go version -m prints. #1740 taught millhone to read it, but only from container image layers, so Go shipped as a bare binary stayed invisible to fossa analyze.

This adds an opt-in gobinary discovery strategy that finds those binaries on the filesystem and reports their modules as regular go+ dependencies. Motivating case: a gomobile SDK shipping jni/<abi>/lib<name>.so inside an AAR — no go.mod, no Go source, and a third-party notice file missing a package demonstrably linked into the shipped binary.

Four decisions a reviewer can't read off the diff:

  • Opt-in behind --enable-go-binary-analysis. Per #cli-and-analysis: on by default would add dependencies to existing projects unasked, and could newly fail builds on code nobody meant to scan. Not experimental- prefixed — v4 is far enough out that a rename later costs more than a stable name now.
  • A discovery strategy, not an extension of --experimental-enable-binary-discovery. --unpack-archives re-runs discoverFuncs over extracted contents; the binary-deps path runs once on the scan root and never enters archives. The motivating binary is inside an AAR, so hooking binary-deps would have missed it. The flags stay independent — a nested binary needs both.
  • A project is a directory, not a binary. Source units are named after their directory (Srclib.Converter.toSourceUnit), so one project per binary emits colliding Name/Manifest whenever a directory holds two. Binaries in a directory now combine, each still visible as an origin path. The container path can name units per binary only because it hand-builds source units outside the strategy framework.
  • Parsing stays in Rust. New analyze-go-binaries subcommand reads candidate paths from stdin (a large repo exceeds the argv limit). DiscoveredGoBinary moved out of the container command into the shared parser module, so there's one implementation.

Acceptance criteria

Without the flag, nothing changes. With it, fossa analyze reports the modules linked into compiled Go binaries as go+ dependencies, and with --unpack-archives reaches binaries nested inside archives.

Testing plan

cargo test -p millhone — 27 pass, 3 new covering scan_file/scan_files: a real Go buildinfo fixture on disk, and rejection of too-small files, non-binaries, directories, and missing paths. Existing container tests still pass, confirming the shared-type move is behavior-preserving.

cabal run unit-tests — 20 GoBinary examples, 10 new: millhone output decoding, version normalization, and directory grouping (same dir combines, different dirs stay separate, unusable binaries dropped).

End to end, fossa analyze --output against real Go binaries:

Case Result
No flag 0 source units
--enable-go-binary-analysis, bare binary in a tree 1 unit, 165 go+ deps
Plus --unpack-archives, binary inside an AAR 2 units; origin path renders <archive>/jni/arm64-v8a/<lib>.so
Two Go binaries in one directory 1 unit, both origin paths, 168 deps (165 + 3, deduped)

Not run locally: fourmolu (couldn't install in this environment), so please let CI arbitrate Haskell formatting. 31 unit tests fail in my checkout both before and after this change, all in suites needing vendored binaries (themis, lernie, berkeleydb, circe) that aren't present locally.

Risks

  • Main module reported as a dependency. Dropped when unversioned ((devel), the normal case), kept when it carries a real version, which happens for go install module@version builds. Kept for consistency with the container path — happy to drop it.
  • Layering. Container.Types now imports Strategy.Go.GoBinary to re-export the shared types. No cycle, and the direction already exists (Container.Sources.GoBinary imports Strategy.Go.Gomod). A neutral Strategy/Go/BuildInfo/Types.hs is a mechanical follow-up if you'd prefer.
  • vendor/ is excluded by default, so a vendored Go binary is skipped unless --include-path vendor is passed. Documented as a limitation rather than special-casing the default filters.
  • --only-target gobinary does nothing without the enabling flag — consistent with binary-deps today.

Metrics

Not tracked today. Adoption of the flag would be the number worth having; there's no hook for it in this change.

References

Checklist

  • I added tests for this PR's change (or explained in the PR description why tests don't make sense).
  • If this PR introduced a user-visible change, I added documentation into docs/.
  • If this PR added docs, I added links as appropriate to the user manual's ToC in docs/README.ms and gave consideration to how discoverable or not my documentation is.
  • If this PR did not mark a release, I added my changes into an ## Unreleased section at the top of Changelog.md.
  • If I made changes to .fossa.yml or fossa-deps.{json.yml}, I updated docs/references/files/*.schema.json AND I have updated example files used by fossa init command.
  • If I made changes to a subcommand's options, I updated docs/references/subcommands/<subcommand>.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VnoktDR9eVW7yH5pciJFh6

saramaebee and others added 3 commits August 25, 2026 13:47
Go binaries embed the list of modules linked into them (the data
`go version -m` prints). millhone could already read this, but only from
container image layers, so Go code shipped as a binary with no manifest
next to it stayed invisible to `fossa analyze`.

Add an opt-in `gobinary` discovery strategy that finds those binaries on
the filesystem and reports their modules as regular `go+` dependencies.

Opt-in via `--enable-go-binary-analysis`, or `enableGoBinaryAnalysis` in
`.fossa.yml`. `fossa analyze` models package-manager scanning, so reading
binaries by default would add dependencies to existing projects without
the user asking, and could newly fail builds on vulnerabilities in code
the user never intended to scan.

Implemented as a normal discovery strategy rather than an extension of
`--experimental-enable-binary-discovery`, because `--unpack-archives`
re-runs only the strategy list over extracted contents. That is what
reaches a binary nested inside an AAR or JAR; the binary-deps path is
invoked once on the scan root and never sees archive contents. The two
flags stay independent - neither implies the other.

A project is a directory rather than a single binary: source units are
named after their directory, so one project per binary emitted colliding
units whenever a directory held more than one Go binary. Binaries in a
directory are now combined, each still visible as an origin path.

millhone gains an `analyze-go-binaries` subcommand reading candidate
paths from stdin (a large repo can exceed the argument-length limit).
The CLI pre-filters with the existing `contentIsBinary` check and
millhone applies the precise magic/size checks, so the buildinfo parser
stays the single Rust implementation shared with container analysis.
`DiscoveredGoBinary` moves out of the container command into the shared
parser module for the same reason.

Verified end to end against real Go binaries: reported bare in a tree,
nested inside an AAR under `--unpack-archives` (origin path renders as
`<archive>/jni/<abi>/<lib>.so`), combined when two binaries share a
directory, and absent entirely when the flag is not passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VnoktDR9eVW7yH5pciJFh6
The unused import broke every build job (and the integration-test job,
which builds the library first); the rest are lint-only.

- Drop the now-unused Data.Text import left behind when normalizeVersion
  moved out of App.Fossa.Container.Sources.GoBinary. CI builds with
  -Werror, so this was an error there and only a warning locally.
- Apply fourmolu and cabal-fmt using the toolchain image CI runs
  (ghc-lib-parser 9.8.4), rather than formatting by hand.
- Move the dependency guard ahead of the path lookup in toProjects, per
  hlint. Also avoids the lookup for binaries with no usable dependency.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VnoktDR9eVW7yH5pciJFh6
Go binary analysis is now opt-in solely via --enable-go-binary-analysis.
Removes the enableGoBinaryAnalysis config-file key, its parser, and the
config-file fallback in collectStrategyConfig, plus the field from test
config fixtures and the docs/schema entries.

Also adds the missing --enable-go-binary-analysis row to the analyze
subcommand flag table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AjBgXmB2ZCmd23oUudRibG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant