A Helm chart for deploying S3Proxy - Access other storage backends via the S3 API
| Repository | Name | Version |
|---|---|---|
| oci://ghcr.io/comet-ml | comet-common | 0.3.0 |
- Kubernetes 1.19+
- Helm 3.2.0+
- PV provisioner support in the underlying infrastructure (if using filesystem backend with persistence)
The chart tracks S3Proxy (andrewgaul/s3proxy) through appVersion (currently 3.3.0); override it with image.tag. Minimum supported version is 2.7.0.
S3Proxy 3.0.0 deprecated the Apache jclouds storage backends (s3, aws-s3, azureblob, filesystem, transient) in favor of SDK / NIO2 providers. Upstream has announced that 3.3.0 is the last release to bundle jclouds and that future releases "will lack its Atmos and B2 storage backends" (no jclouds-free release has shipped yet; 3.3.0 remains the latest). The jclouds providers still work on 3.x, but are deprecated:
filesystem/transient: already default to the non-deprecated*-nio2variants (nio2: true).azureblob: defaults toprovider: azureblob-sdk(the Azure SDK provider, which signs correctly against custom endpoints such as Azurite). The legacy jcloudsazureblobprovider is deprecated and mis-signs against custom endpoints; setprovider: azureblobonly if you specifically need it. On real Azure,azureblob-sdkmay requireconfig.backends.azureblob.regionsfor bucket creation.s3,googleCloudStorage,openstackSwift: SDK providers exist upstream (aws-s3-sdk,google-cloud-storage-sdk,openstack-swift-sdk).rackspaceCloudfilesis OpenStack-Swift-compatible and may be served byopenstack-swift-sdk. Migrating the chart defaults to the SDK providers is tracked separately.b2(and Atmos, if ever added) are jclouds-only with no SDK successor. These are the backends upstream has said future releases will drop.
- Through S3Proxy
3.3.0(chart0.x): all current backends, including the jclouds-onlyb2, are supported. This is where the chart sits today. - Guardrail: the chart hard-fails at render time if
b2(oratmos) is enabled while the effective S3Proxy version (image.tag, elseappVersion) is greater than3.3.0, so an upgrade past the jclouds sunset cannot silently ship a broken backend. Pinimage.tagto3.3.0or earlier, or disable the backend. - When S3Proxy ships its first jclouds-free release: the chart moves to
1.x.x(major bump), migrates the remaining backends to their SDK providers, and dropsb2/atmos. Tracked in the SDK-migration follow-up.
# Install with default values (filesystem backend)
helm install my-s3proxy oci://ghcr.io/comet-ml/s3proxy
# Install with custom values
helm install my-s3proxy oci://ghcr.io/comet-ml/s3proxy -f override-values.yamlThe following section lists the configurable parameters of the s3proxy chart and their default values.
Scroll sideways to see all columns.
| Key | Description | Type | Default |
|---|---|---|---|
affinity |
Affinity for pod assignment | object |
{} |
autoscaling.enabled |
Enable HPA | bool |
false |
autoscaling.maxReplicas |
Maximum number of replicas | int |
100 |
autoscaling.minReplicas |
Minimum number of replicas | int |
1 |
autoscaling.targetCPUUtilizationPercentage |
Target CPU utilization percentage | int |
80 |
config.auth.identity |
S3 Access Key ID for client authentication | string |
"" |
config.auth.secret |
S3 Secret Access Key for client authentication | string |
"" |
config.auth.type |
Authorization type (none, aws-v2, aws-v4, aws-v2-or-v4) | string |
"aws-v4" |
config.backends.azureblob.account |
Storage account name | string |
"" |
config.backends.azureblob.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.azureblob.enabled |
Enable Azure Blob Storage backend | bool |
false |
config.backends.azureblob.endpoint |
Azure endpoint | string |
https://{{ .Values.config.backends.azureblob.account }}.blob.core.windows.net |
config.backends.azureblob.key |
Storage account key configuration | object |
{"existingSecret":"","secretKey":"accountKey","value":""} |
config.backends.azureblob.key.existingSecret |
Name of existing secret containing the storage account key | string |
"" |
config.backends.azureblob.key.secretKey |
Key in the existing secret containing the storage account key | string |
"accountKey" |
config.backends.azureblob.key.value |
Storage account key value | string |
"" |
config.backends.azureblob.provider |
Provider type. Defaults to azureblob-sdk (Azure SDK): it signs correctly against custom endpoints (Azurite, Azure Gov/China, private endpoints) and is the non-deprecated provider on S3Proxy 3.x. The legacy jclouds azureblob provider is deprecated upstream and mis-signs against custom endpoints; on real Azure azureblob-sdk may require regions to be set for bucket creation. |
string |
"azureblob-sdk" |
config.backends.azureblob.regions |
jclouds region(s) for the backend, emitted as jclouds.regions=. The azureblob-sdk provider requires this on real Azure to create buckets (without it aws s3 mb fails with InvalidLocationConstraint / "no jclouds.regions configured"). Comma-separated for multiple. Not needed against Azurite. Leave empty to omit. |
string |
"" |
config.backends.azureblob.sasToken |
SAS token configuration | object |
{"existingSecret":"","secretKey":"sasToken","value":""} |
config.backends.azureblob.sasToken.existingSecret |
Name of existing secret containing the SAS token | string |
"" |
config.backends.azureblob.sasToken.secretKey |
Key in the existing secret containing the SAS token | string |
"sasToken" |
config.backends.azureblob.sasToken.value |
SAS token value | string |
"" |
config.backends.b2.account |
B2 account ID | string |
"" |
config.backends.b2.applicationKey |
B2 application key configuration | object |
{"existingSecret":"","secretKey":"applicationKey","value":""} |
config.backends.b2.applicationKey.existingSecret |
Name of existing secret containing the application key | string |
"" |
config.backends.b2.applicationKey.secretKey |
Key in the existing secret containing the application key | string |
"applicationKey" |
config.backends.b2.applicationKey.value |
Application key value | string |
"" |
config.backends.b2.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.b2.enabled |
Enable Backblaze B2 backend | bool |
false |
config.backends.filesystem.basedir |
Base directory for filesystem backend | string |
"/data/s3proxy" |
config.backends.filesystem.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.filesystem.credential |
jclouds credential. S3Proxy requires jclouds.credential in every backend properties file; the filesystem backend ignores the value. An empty value falls back to "local". | string |
"local" |
config.backends.filesystem.enabled |
Enable filesystem backend | bool |
true |
config.backends.filesystem.identity |
jclouds identity. S3Proxy requires jclouds.identity in every backend properties file; the filesystem backend ignores the value, so the "local" placeholder is fine. An empty value falls back to "local". | string |
"local" |
config.backends.filesystem.nio2 |
Use NIO2 implementation (filesystem-nio2) instead of standard filesystem | bool |
true |
config.backends.googleCloudStorage.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.googleCloudStorage.clientEmail |
Service account email or user email | string |
"" |
config.backends.googleCloudStorage.enabled |
Enable Google Cloud Storage backend | bool |
false |
config.backends.googleCloudStorage.privateKey |
Private key configuration | object |
{"existingSecret":"","secretKey":"private.key","value":""} |
config.backends.googleCloudStorage.privateKey.existingSecret |
Name of existing secret containing the private key | string |
"" |
config.backends.googleCloudStorage.privateKey.secretKey |
Key in the existing secret containing the private key (in PEM format) | string |
"private.key" |
config.backends.googleCloudStorage.privateKey.value |
Private key value (in PEM format) | string |
"" |
config.backends.googleCloudStorage.projectID |
GCP project ID | string |
"" |
config.backends.openstackSwift.authURL |
Authentication URL | string |
"" |
config.backends.openstackSwift.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.openstackSwift.enabled |
Enable OpenStack Swift backend | bool |
false |
config.backends.openstackSwift.password |
Password configuration | object |
{"existingSecret":"","secretKey":"password","value":""} |
config.backends.openstackSwift.password.existingSecret |
Name of existing secret containing the password | string |
"" |
config.backends.openstackSwift.password.secretKey |
Key in the existing secret containing the password | string |
"password" |
config.backends.openstackSwift.password.value |
Password value | string |
"" |
config.backends.openstackSwift.region |
Region | string |
"" |
config.backends.openstackSwift.tenantName |
Tenant name | string |
"" |
config.backends.openstackSwift.userName |
Username | string |
"" |
config.backends.rackspaceCloudfiles.apiKey |
API key configuration | object |
{"existingSecret":"","secretKey":"apiKey","value":""} |
config.backends.rackspaceCloudfiles.apiKey.existingSecret |
Name of existing secret containing the API key | string |
"" |
config.backends.rackspaceCloudfiles.apiKey.secretKey |
Key in the existing secret containing the API key | string |
"apiKey" |
config.backends.rackspaceCloudfiles.apiKey.value |
API key value | string |
"" |
config.backends.rackspaceCloudfiles.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.rackspaceCloudfiles.enabled |
Enable Rackspace Cloud Files backend | bool |
false |
config.backends.rackspaceCloudfiles.region |
Region (uk or us) | string |
"us" |
config.backends.rackspaceCloudfiles.userName |
Username | string |
"" |
config.backends.s3.accessKeyID |
S3 Access Key ID for backend | string |
"" |
config.backends.s3.aws |
Use AWS-specific S3 provider (aws-s3) instead of generic S3 provider | bool |
true |
config.backends.s3.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.s3.enabled |
Enable S3 backend | bool |
false |
config.backends.s3.endpoint |
S3 endpoint | string |
"" |
config.backends.s3.region |
AWS region | string |
"" |
config.backends.s3.secretAccessKey |
S3 Secret Access Key configuration | object |
{"existingSecret":"","secretKey":"secretAccessKey","value":""} |
config.backends.s3.secretAccessKey.existingSecret |
Name of existing secret containing the secret access key | string |
"" |
config.backends.s3.secretAccessKey.secretKey |
Key in the existing secret containing the secret access key | string |
"secretAccessKey" |
config.backends.s3.secretAccessKey.value |
Secret access key value | string |
"" |
config.backends.transient.bucketLocators |
Buckets routed to this backend (S3Proxy bucket-locator; glob patterns supported). Only relevant when multiple backends are enabled; a bucket matching no backend's list falls through to the first-enabled backend. | list |
[] |
config.backends.transient.credential |
jclouds credential. S3Proxy requires jclouds.credential in every backend properties file; the transient backend ignores the value. An empty value falls back to "local". | string |
"local" |
config.backends.transient.enabled |
Enable transient (in-memory) backend | bool |
false |
config.backends.transient.identity |
jclouds identity. S3Proxy requires jclouds.identity in every backend properties file; the transient backend ignores the value, so the "local" placeholder is fine. An empty value falls back to "local". | string |
"local" |
config.backends.transient.nio2 |
Use NIO2 implementation (transient-nio2) instead of standard transient | bool |
true |
config.buckets.alias |
Map virtual bucket names to actual backend buckets | object |
{} |
config.cors.allowCredential |
Allow credentials | bool |
false |
config.cors.allowHeaders |
Allowed headers | list |
["Accept","Content-Type"] |
config.cors.allowMethods |
Allowed methods | list |
["GET","PUT","POST","HEAD","DELETE"] |
config.cors.allowOrigins |
Allowed origins (e.g., ["https://example.com", "https://.+\\.example\\.com"]) | list |
[] |
config.cors.enabled |
Enable CORS support | bool |
false |
config.logLevel |
Log level for S3Proxy (DEBUG, INFO, WARN, ERROR) | string |
"INFO" |
config.middlewares.eventualConsistency |
Enable eventual consistency modeling | bool |
false |
config.middlewares.largeObjectMocking |
Enable large object mocking | bool |
false |
config.middlewares.readOnly |
Make backend read-only | bool |
false |
config.middlewares.shardedBackend |
Enable sharded backend containers | bool |
false |
config.tls.enabled |
Enable native in-pod HTTPS (S3Proxy secure-endpoint). When enabled, S3Proxy serves HTTPS only on service.targetPort (the plaintext endpoint is not bound), so TLS is terminated in the pod rather than at the ingress. Requires a PKCS12 (or JKS) keystore and its password. tcpSocket health probes are unaffected (they do not perform a TLS handshake). |
bool |
false |
config.tls.keystore.existingSecret |
Name of an existing Secret holding the keystore file (binary PKCS12/JKS). Takes precedence over value. Use this for a customer-provided keystore or a cert-manager-issued one (Certificate.spec.keystores.pkcs12). |
string |
"" |
config.tls.keystore.secretKey |
Key within keystore.existingSecret (or, when value is used, within the chart's own Secret) that holds the keystore file |
string |
"keystore.p12" |
config.tls.keystore.value |
Inline base64-encoded keystore, stored in the chart's own Secret and mounted as a file. Used only when keystore.existingSecret is empty. Convenient for testing; prefer existingSecret in production. |
string |
"" |
config.tls.keystorePassword.existingSecret |
Name of an existing Secret holding the keystore password. Takes precedence over value. |
string |
"" |
config.tls.keystorePassword.secretKey |
Key within keystorePassword.existingSecret that holds the keystore password |
string |
"keystore-password" |
config.tls.keystorePassword.value |
Inline keystore password, stored in the chart's own Secret and merged into the backend properties by the secret-merge initContainer (kept out of the ConfigMap). Used only when keystorePassword.existingSecret is empty. |
string |
"" |
config.virtualHost |
Virtual Host configuration | string |
"" |
configMergeImage.pullPolicy |
Config merge container image pull policy | string |
"IfNotPresent" |
configMergeImage.repository |
Config merge container image repository | string |
"busybox" |
configMergeImage.tag |
Config merge container image tag | string |
"1.36" |
extraEnvVars |
Additional environment variables | list |
[] |
extraVolumeMounts |
Additional volume mounts | list |
[] |
extraVolumes |
Additional volumes | list |
[] |
fullnameOverride |
String to fully override s3proxy.fullname template | string |
"" |
image.pullPolicy |
Image pull policy | string |
"IfNotPresent" |
image.repository |
S3Proxy image repository | string |
"andrewgaul/s3proxy" |
image.tag |
Overrides the image tag whose default is the chart appVersion | string |
"" |
imagePullSecrets |
Image pull secrets | list |
[] |
ingress.annotations |
Ingress annotations | object |
{} |
ingress.className |
Ingress class name | string |
"" |
ingress.enabled |
Enable ingress | bool |
false |
ingress.hosts |
Ingress hosts configuration | list |
[] |
ingress.tls |
TLS configuration | list |
[] |
nameOverride |
String to partially override s3proxy.fullname template | string |
"" |
nodeSelector |
Node selector for pod assignment | object |
{} |
persistence.accessMode |
PVC Access Mode | string |
"ReadWriteOnce" |
persistence.annotations |
PVC annotations | object |
{} |
persistence.enabled |
Enable persistence using PVC | bool |
true |
persistence.existingClaim |
Use existing PVC | string |
"" |
persistence.size |
PVC Storage Request | string |
"10Gi" |
persistence.storageClass |
Storage Class | string |
"" (uses default StorageClass) |
podAnnotations |
Annotations to add to the pod | object |
{} |
podLabels |
Extra labels to add to the pod (merged with the chart's selector labels) | object |
{} |
podSecurityContext |
Pod security context | object |
{} |
replicaCount |
Number of S3Proxy replicas | int |
1 |
resources |
Resource limits and requests | object |
{} |
securityContext |
Container security context | object |
{} |
service.annotations |
Service annotations | object |
{} |
service.port |
Service port | int |
9000 |
service.targetPort |
Target port (controls both the container port and S3Proxy bind port) | int |
9000 |
service.type |
Kubernetes service type | string |
"ClusterIP" |
serviceAccount.annotations |
Annotations to add to the service account | object |
{} |
serviceAccount.create |
Specifies whether a service account should be created | bool |
false |
serviceAccount.name |
The name of the service account to use. If not set and create is true, a name is generated using the fullname template | string |
"" |
terminationGracePeriodSeconds |
Pod termination grace period. Covers the 5s preStop sleep plus the in-flight request drain (up to 30s) that S3Proxy releases after 3.3.0 perform on SIGTERM. | int |
40 |
tolerations |
Tolerations for pod assignment | list |
[] |
# values-filesystem.yaml
config:
auth:
type: "aws-v4"
identity: "myaccesskey"
secret: "mysecretkey"
backends:
filesystem:
enabled: true
nio2: true
filesystem:
basedir: "/data/s3proxy"
persistence:
enabled: true
size: 20GiInstall:
helm install s3proxy-fs ./s3proxy -f values-filesystem.yaml# values-aws-s3.yaml
config:
auth:
type: "aws-v4"
identity: "proxy-access-key" # For clients connecting to s3proxy
secret: "proxy-secret-key"
backend:
provider: "aws-s3"
awsS3:
region: "us-west-2"
accessKeyID: "aws-access-key-id" # For s3proxy to connect to AWS
secretAccessKey: "aws-secret-access-key"
persistence:
enabled: false # Not needed for S3 backendInstall:
helm install s3proxy-s3 ./s3proxy -f values-aws-s3.yaml# values-azure.yaml
config:
auth:
type: "aws-v4"
identity: "myaccesskey"
secret: "mysecretkey"
backend:
provider: "azureblob"
azureblob:
account: "mystorageaccount"
key: "storageaccountkey"
persistence:
enabled: false # Not needed for Azure backendInstall:
helm install s3proxy-azure ./s3proxy -f values-azure.yaml# values-gcs.yaml
config:
auth:
type: "aws-v4"
identity: "myaccesskey"
secret: "mysecretkey"
backend:
provider: "google-cloud-storage"
googleCloudStorage:
projectID: "my-project"
clientEmail: "service-account@my-project.iam.gserviceaccount.com"
privateKey: |
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
persistence:
enabled: false # Not needed for GCS backend# values-anonymous.yaml
config:
auth:
type: "none"
backends:
transient:
enabled: true
nio2: true # In-memory storage
persistence:
enabled: falseOnce deployed, you can test S3Proxy using the AWS CLI:
# Get the service endpoint
kubectl get svc
# Port-forward for local testing
kubectl port-forward svc/my-s3proxy 8080:8080
# Configure AWS CLI (if authentication is enabled)
export AWS_ACCESS_KEY_ID=myaccesskey
export AWS_SECRET_ACCESS_KEY=mysecretkey
# Test S3 operations
aws --endpoint-url http://localhost:8080 s3 ls
aws --endpoint-url http://localhost:8080 s3 mb s3://test-bucket
aws --endpoint-url http://localhost:8080 s3 cp test.txt s3://test-bucket/
aws --endpoint-url http://localhost:8080 s3 ls s3://test-bucket/By default S3Proxy binds plain HTTP and TLS is expected to be terminated at the
ingress. Set config.tls.enabled=true to have S3Proxy itself terminate HTTPS in
the pod (S3Proxy's secure-endpoint). When enabled:
- S3Proxy serves HTTPS only on
service.targetPort(the plaintext endpoint is not bound). The container/Service port is namedhttps. - The health probes are
tcpSocket, which only check the TCP accept (no TLS handshake), so they keep working unchanged against the TLS port. - A PKCS12 keystore is expected (Jetty's default keystore type; JKS also works if supplied). S3Proxy exposes only the keystore path and password, so the chart does not set a keystore type.
- The keystore password is never written to the ConfigMap; it is merged into the backend properties from the Secret by the config-merge initContainer.
Reference a Secret that already holds the keystore file and the password. This is
also how a cert-manager Certificate with spec.keystores.pkcs12 delivers a
keystore (point keystore.existingSecret at that Secret, and
keystorePassword.existingSecret at the password Secret it references):
config:
tls:
enabled: true
keystore:
existingSecret: my-tls # holds the PKCS12 archive
secretKey: keystore.p12
keystorePassword:
existingSecret: my-tls # holds the password (may be the same Secret)
secretKey: keystore-passwordProvide the base64-encoded keystore and the password inline; both are stored in the chart's own Secret. Convenient for testing; prefer Option A in production.
config:
tls:
enabled: true
keystore:
value: "<base64-encoded PKCS12 keystore>"
keystorePassword:
value: "changeit"Create a PKCS12 keystore, for example:
# From an existing cert + key:
openssl pkcs12 -export -inkey tls.key -in tls.crt \
-out keystore.p12 -passout pass:changeit
# Inline value:
base64 -w0 keystore.p12
# Or as an existing Secret (Option A):
kubectl create secret generic my-tls \
--from-file=keystore.p12=keystore.p12 \
--from-literal=keystore-password=changeitTesting over HTTPS (the CI keystore above is self-signed, so skip verification):
aws --endpoint-url https://localhost:9000 --no-verify-ssl s3 lsTo enable CORS support:
config:
cors:
enabled: true
allowOrigins:
- "https://example.com"
- "https://.+\\.example\\.com"
allowMethods:
- "GET"
- "PUT"
- "POST"
- "HEAD"
- "DELETE"
allowHeaders:
- "Accept"
- "Content-Type"
allowCredential: trueS3Proxy supports various middlewares:
config:
middlewares:
readOnly: false # Make backend read-only
eventualConsistency: true # Enable eventual consistency modeling
shardedBackend: true # Enable sharded backend containers
largeObjectMocking: false # Enable large object mockingMap virtual bucket names to actual backend buckets:
config:
buckets:
alias:
virtual-bucket: "real-backend-bucket"
another-bucket: "actual-bucket-name"When more than one backend is enabled, assign buckets to a specific backend with
that backend's own bucketLocators list. S3Proxy reads bucket-locators per
backend, so each list is emitted only into that backend's properties file. Glob
patterns are supported. A bucket that matches no backend's list falls through to
the first-enabled backend (the default).
config:
backends:
s3:
enabled: true
# ... credentials ...
bucketLocators:
- "prod-*"
- "customer-data"
filesystem:
enabled: true
bucketLocators:
- "scratch-*" # everything else also lands here (first-enabled default)Check S3Proxy logs:
kubectl logs deployment/my-s3proxyhelm upgrade my-s3proxy ./s3proxy -f my-values.yamlhelm uninstall my-s3proxyThis will remove all resources created by the chart. If using persistence, the PVC will be retained by default.
-
Authentication failures: Ensure
config.auth.identityandconfig.auth.secretare set correctly for client authentication. -
Backend connection issues: Verify backend credentials are correctly configured in the appropriate section (e.g.,
config.backend.awsS3.*). -
Persistence issues: Check that your cluster has a default StorageClass or specify one explicitly.
-
Port conflicts: If port 8080 is already in use, change
service.portandservice.targetPort.
This Helm chart is provided as-is. S3Proxy itself is licensed under the Apache License 2.0.
Autogenerated from chart metadata using helm-docs v1.14.2