-
Notifications
You must be signed in to change notification settings - Fork 82
[docs] Document default fsGroup for Kubernetes persistent volumes #1502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: release/13.5
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -185,6 +185,71 @@ await api.withKubernetesPersistentVolumeMount(media, '/srv/media'); | |
| Any workload bound to a persistent volume is rendered as a `StatefulSet` instead of a `Deployment`, regardless of its default workload kind. There is no opt-out. A `Deployment` with a `ReadWriteOnce` claim and more than one replica is broken by design because the second pod can't mount the same volume. Therefore, binding to a durable volume automatically selects the `StatefulSet` path, which gives pods stable identity and ordered rollout. | ||
| </Aside> | ||
|
|
||
| ## Default pod security context | ||
|
|
||
| A Kubernetes access mode such as `ReadWriteOnce` controls how a volume can be attached and mounted; it doesn't grant the container's Linux process permission to write to the mounted filesystem. Without further configuration, a freshly provisioned volume is typically owned by `root:root`, so a workload that runs as a non-root user fails on its first write even though the PVC is `Bound` and the pod is `Running`. | ||
|
|
||
| To avoid that out-of-the-box failure, both `WithPersistentVolume(...)` overloads add the following pod security context to the workload automatically: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we also update the generated StatefulSet excerpt below to include this |
||
|
|
||
| ```yaml | ||
| securityContext: | ||
| fsGroup: 2000 | ||
| fsGroupChangePolicy: OnRootMismatch | ||
| ``` | ||
|
|
||
| `fsGroup` adds a supplemental group to the pod's processes and, for supported volume types, instructs Kubernetes or the CSI driver to make the mounted volume accessible to that group. It doesn't change the image-defined UID or primary GID, so Aspire doesn't need to know which identity the image uses. Aspire always uses the same group ID, `2000`, rather than choosing a new value on each deployment, because group ownership is persisted on the volume. `OnRootMismatch` avoids an unnecessary recursive ownership change when the volume already has the expected group. | ||
|
|
||
| This default only applies to workloads bound through `WithPersistentVolume(...)`. Ordinary workloads and legacy PVC generation through the Kubernetes environment's default storage type are unaffected, and read-only mounts remain read-only. | ||
|
|
||
| ### Overriding or removing the default | ||
|
|
||
| The default security context is applied before any `PublishAsKubernetesService` callback runs, so a workload or cluster that needs a different group can replace it: | ||
|
|
||
| <Tabs syncKey="aspire-lang"> | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Phase B (doc-tester, blind to source) — warning + knowledge gap. This Knowledge gap: the page doesn't state whether overriding/removing the pod security context is available to TypeScript AppHosts. Please add a TypeScript tab, or a one-line note that this callback is C#-only. |
||
| <TabItem id="csharp" label="C#"> | ||
|
|
||
| ```csharp title="AppHost.cs" | ||
| builder.AddProject<Projects.WebFrontend>("webfrontend") | ||
| .WithPersistentVolume(data, "/data") | ||
| .PublishAsKubernetesService(resource => | ||
| { | ||
| var podSpec = resource.Workload?.PodTemplate.Spec | ||
| ?? throw new InvalidOperationException("The Kubernetes workload was not generated."); | ||
|
|
||
| podSpec.SecurityContext ??= new(); | ||
| podSpec.SecurityContext.FsGroup = 3000; | ||
| }); | ||
|
Comment on lines
+212
to
+221
|
||
| ``` | ||
|
|
||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| The generated `OnRootMismatch` change policy is retained unless the callback also replaces it. | ||
|
|
||
| To remove the generated pod security context entirely — for example, when ownership is managed by the image, an admission controller, or storage-specific configuration — set it to `null` in the same callback: | ||
|
|
||
| <Tabs syncKey="aspire-lang"> | ||
| <TabItem id="csharp" label="C#"> | ||
|
|
||
| ```csharp title="AppHost.cs" | ||
| builder.AddProject<Projects.WebFrontend>("webfrontend") | ||
| .WithPersistentVolume(data, "/data") | ||
| .PublishAsKubernetesService(resource => | ||
| { | ||
| var podSpec = resource.Workload?.PodTemplate.Spec | ||
| ?? throw new InvalidOperationException("The Kubernetes workload was not generated."); | ||
|
|
||
| podSpec.SecurityContext = null; | ||
| }); | ||
| ``` | ||
|
|
||
| </TabItem> | ||
| </Tabs> | ||
|
|
||
| <Aside type="caution"> | ||
| The group ID `2000` is an Aspire-managed default, not a Kubernetes-reserved value. Some storage drivers don't support `fsGroup`, some CSI drivers apply the group at mount time themselves, and cluster admission policies can restrict allowed group ranges. Use the customization callback above as an escape hatch in those environments. | ||
| </Aside> | ||
|
|
||
| ## Unbound volumes and default storage | ||
|
|
||
| Volumes on a workload that aren't bound to a `KubernetesPersistentVolumeResource` continue to use the environment's default storage type. You can mix a first-class persistent volume and an unbound ephemeral volume on the same workload — each is resolved independently at publish time. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Phase A — unverifiable on the source-of-truth branch (version skew).
This PR targets
aspire.devrelease/13.5, so the matching source of truth ismicrosoft/aspirerelease/13.5(@8ab6999850). On that branch the automaticfsGroup: 2000/fsGroupChangePolicy: OnRootMismatchdefault does not exist:WithPersistentVolume(...)overloads add only annotations — no security context (KubernetesPersistentVolumeExtensions.cs:253and:293).PodSecurityContextV1is ever constructed insrc/Aspire.Hosting.Kubernetes, the literal2000appears nowhere there, andOnRootMismatchappears only in a doc comment (PodSecurityContextV1.cs:114).The behavior is present on
microsoft/aspiremain(13.6):KubernetesResource.cs:24-25defineDefaultPersistentVolumeFsGroup = 2000andDefaultPersistentVolumeFsGroupChangePolicy = "OnRootMismatch"— matching the source PR's13.6milestone (microsoft/aspire#19374).So every claim about the automatic default — the
2000/OnRootMismatchvalues (L196‑197, L200), "this default only applies toWithPersistentVolume" (L202), "applied before anyPublishAsKubernetesServicecallback runs" (L206), and "the generatedOnRootMismatchchange policy is retained" (L227) — can't be verified againstrelease/13.5. As written, a 13.5 reader is told about a default that doesn't ship in 13.5. Please confirm the intended target branch (e.g. wait for arelease/13.6docs branch, or ensure the source change is present on 13.5) before this lands on the 13.5 docs.Note: the override/removal API used below (
PublishAsKubernetesService,resource.Workload?.PodTemplate.Spec,podSpec.SecurityContext,FsGroup) is valid onrelease/13.5— see the ✅ list in the review summary.