You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add affinity and nodeSelector to the common DaemonSet config in ClusterPolicy
(spec.daemonsets), surfaced in the Helm chart as daemonsets.affinity / daemonsets.nodeSelector (default empty). When set, apply them to all operand
DaemonSets (driver, container-toolkit, device-plugin, dcgm-exporter, gfd,
mig-manager, …), on top of the operator's existing NFD-based node selection.
Motivation
spec.daemonsets already exposes tolerations but has no affinity/nodeSelector.
That's an asymmetry: tolerations only permit scheduling onto tainted nodes — they
can't restrict where operands land. On a cloud cluster (EKS/GKE/AKS) with several
autoscaled node pools where only one has GPUs, I want to declaratively pin operands to
that pool via a label it already carries (e.g. karpenter.sh/nodepool=gpu).
The documented alternatives don't cover this well:
nvidia.com/gpu.deploy.<operand>=disabled labels / node taints (how Deploy nvidia-device-plugin-daemonset to only certain nodes #547 was
closed) are subtractive: you must label or taint every node you want to exclude.
In autoscaled clusters (Karpenter, managed nodegroups) where nodes churn constantly,
that doesn't scale and is race-prone.
NFD gating keeps operands off non-GPU nodes, but can't target a specific GPU
pool and doesn't apply when NFD is disabled.
A first-class affinity/nodeSelector is declarative, applies automatically to new
nodes, and is consistent with the operator's own NvidiaDriver CRD, which already
exposes spec.affinity and spec.nodeSelector.
Prior art
#547 requested essentially this and was closed pointing at the subtractive workarounds
above; a community MR to add nodeSelector to the common daemonset config was referenced
there but never landed.
Feature description
Add
affinityandnodeSelectorto the common DaemonSet config inClusterPolicy(
spec.daemonsets), surfaced in the Helm chart asdaemonsets.affinity/daemonsets.nodeSelector(default empty). When set, apply them to all operandDaemonSets (driver, container-toolkit, device-plugin, dcgm-exporter, gfd,
mig-manager, …), on top of the operator's existing NFD-based node selection.
Motivation
spec.daemonsetsalready exposestolerationsbut has noaffinity/nodeSelector.That's an asymmetry: tolerations only permit scheduling onto tainted nodes — they
can't restrict where operands land. On a cloud cluster (EKS/GKE/AKS) with several
autoscaled node pools where only one has GPUs, I want to declaratively pin operands to
that pool via a label it already carries (e.g.
karpenter.sh/nodepool=gpu).The documented alternatives don't cover this well:
nvidia.com/gpu.deploy.<operand>=disabledlabels / node taints (how Deploy nvidia-device-plugin-daemonset to only certain nodes #547 wasclosed) are subtractive: you must label or taint every node you want to exclude.
In autoscaled clusters (Karpenter, managed nodegroups) where nodes churn constantly,
that doesn't scale and is race-prone.
pool and doesn't apply when NFD is disabled.
A first-class
affinity/nodeSelectoris declarative, applies automatically to newnodes, and is consistent with the operator's own
NvidiaDriverCRD, which alreadyexposes
spec.affinityandspec.nodeSelector.Prior art
#547 requested essentially this and was closed pointing at the subtractive workarounds
above; a community MR to add nodeSelector to the common daemonset config was referenced
there but never landed.
Proposed API
User-supplied values should be merged (AND) with the operator's NFD-based selection so
GPU detection still applies.