Skip to content

Add THIRD_PARTY_NOTICES.md and a generator for it - #111

Merged
abrarshivani merged 1 commit into
NVIDIA:mainfrom
abrarshivani:third-party-notices
Aug 14, 2026
Merged

Add THIRD_PARTY_NOTICES.md and a generator for it#111
abrarshivani merged 1 commit into
NVIDIA:mainfrom
abrarshivani:third-party-notices

Conversation

@abrarshivani

@abrarshivani abrarshivani commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Adds THIRD_PARTY_NOTICES.md for this module's third-party content, the script
that generates it, and a CI check that fails when deps change without the notices
being refreshed.

The same change is already merged in gpu-operator, mig-parted, k8s-device-plugin,
k8s-driver-manager and nvidia-container-toolkit.

The existing NOTICE file is unchanged.

What to review

Hand-written:

File Lines
hack/generate-third-party-notices.sh 529
hack/notices/licenses/BSD-3-Clause.txt 26
Makefile 16
.github/workflows/golang.yml 16
versions.mk 3
.gitignore 1

Generated: THIRD_PARTY_NOTICES.md (394 lines).

hack/ is new. The pin lives in versions.mk, this repo's existing version
mechanism, since there is no tools.go or tools module.

How it works

make third-party-notices:

  1. Use ./bin/go-licenses so the pinned version runs
  2. Run go-licenses save and csv once per platform, then merge
  3. Join all licenses per package instead of picking one
  4. Add the module path from vendor/modules.txt
  5. Add bundled non-Go files described in the root NOTICE
  6. Write to a temp file, then move it into place

make check-third-party-notices regenerates and diffs. It runs as a job in
golang.yml on every build, with no changed-paths filter: the inventory is the
import closure of ./..., so it moves whenever an ordinary .go file changes
its imports, not only when go.mod or vendor/ change.

Implementation notes

  • Scope is ./..., not ./cmd/.... This is a library with no cmd/ and no
    main.go. Nothing releases a binary, so the module itself is the distributed
    artifact.
  • pkg/pciids/default_pci.ids is covered. This is the trap in this repo: a
    bundled 1.5 MB third-party database that go-licenses cannot see. NOTICE
    already attributes it, but a file named THIRD_PARTY_NOTICES.md is what a
    compliance process picks up, and an omission there is invisible precisely
    because the document looks complete. Both files now carry it.
  • NOTICE is the single source of truth for that file. The election is
    stated there in prose and read back from there, so the two cannot drift apart.
    A reworded NOTICE fails the run rather than silently dropping an entry.
  • The pci.ids notice is extracted from the data file itself, not copied into
    a second place. make update-pcidb overwrites that file wholesale, so a
    hand-maintained copy would keep asserting the old version's notice.
  • --ignore is not used at all. It is a raw strings.HasPrefix match on the
    package path, not a path-segment match, and an ignored package is dropped
    silently with a zero exit. --ignore=go would take golang.org/x/* with it,
    and --ignore=github.com/NVIDIA/go-nvlib would also swallow a real dependency
    published as go-nvlib-extra/.... This repo's own packages are dropped in a
    separate pass that compares whole path segments.
  • Output must match on every machine. LC_ALL=C on every sort and grep.
    Whole-line sort -u rather than sort -t, -k1,1 -u. The awk counts instead of
    using in, which mawk and BSD awk disagree on. grep -a so a NUL byte cannot
    change the fence width.
  • Moved with mv, not cp, so an interrupted run cannot leave a
    half-written file.

Scope

Three third-party modules are linked: go-nvml, google/uuid and
golang.org/x/sys. One bundled non-Go file: default_pci.ids.

The header states the boundary in three parts: Go module dependencies, bundled
non-Go files, and what is excluded (the Go standard library, and NVIDIA's own
code under LICENSE).

testify, go-spew, go-difflib and yaml.v3 are vendored for tests and are
correctly absent, with no filter needed.

Guards: a NOTICE that describes no bundled files is fatal, an entry whose file
is missing is fatal, and a declared SPDX id with no license text is fatal, since
an identifier on its own is not a notice.

Shipping

Committed to the repo. NOTICE is byte-for-byte unchanged and remains part of
the distribution.

Testing

  • Determinism. Repeat runs on macOS byte-identical, sha256 9682e099…,
    matching the committed file.
  • Cross-host. Same sha256 from golang:1.25.6-bookworm, the Go version
    versions.mk pins, on both linux/amd64 and linux/arm64. /usr/bin/awk is
    mawk there, so the license join is exercised on the awk that breaks the naive
    form, against BWK awk on macOS.
  • Completeness against go list -deps ./... over all four platforms in the
    matrix: 4 linked third-party packages, 3 index rows, 0 uncovered, 0 unused.
  • Content. 4 index rows across the two tables, 4 sections, 5 license-text
    blocks, 0 Unknown, 0 missing text, 0 unresolved module paths. A row with no
    license text is fatal rather than rendered as unavailable, since every dep here
    is vendored.
  • The awk paths this repo does not reach. Driven directly, on both mawk and
    BWK awk with identical output: a package carrying two licenses joins as
    BSD-3-Clause / MPL-2.0 with no leading separator and duplicates collapsed, a
    => replace resolves to the replacement's module path, a replace pointing at a
    local path exits 1, and the local-module filter keeps a go-nvlib-extra
    sibling while dropping go-nvlib/pkg/....
  • Error paths. A missing NOTICE, a NOTICE whose prose cannot be parsed, a
    NOTICE electing a license this generator does not know, a NOTICE naming a
    file that does not exist, an elected SPDX id whose text is absent, a missing
    vendor/modules.txt, a modules.txt with no module lines, and an absent or
    wrong-architecture go-licenses. Each exits non-zero with a clear message,
    leaves the committed file untouched, and writes no partial file.
  • The recommended fix works on a pristine tree. With no bin/,
    make third-party-notices installs the pinned go-licenses and produces the
    identical document. With a macOS binary planted in bin/ and the generator run
    under linux/amd64, it fails with that guidance rather than an exec error.
  • Lint. shellcheck clean. actionlint reports nothing on the new job that
    it does not already report identically on the three existing ones. make build
    and make test pass in the container.

@abrarshivani
abrarshivani marked this pull request as ready for review August 11, 2026 03:13
@abrarshivani
abrarshivani force-pushed the third-party-notices branch 9 times, most recently from 18b9739 to 49cb4c0 Compare August 14, 2026 19:22
Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
@abrarshivani abrarshivani self-assigned this Aug 14, 2026
@abrarshivani
abrarshivani merged commit 3c16156 into NVIDIA:main Aug 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants