feat(ExternalStorage) Allow the use of 'prefix' in S3 buckets by mbartsch · Pull Request #61585 · nextcloud/server

mbartsch · 2026-06-25T08:25:06Z

Summary

Adds an optional Object key prefix parameter to the S3-Compatible Object Storage external storage backend.

This allows scoping a mount to a sub-path within a shared bucket rather than requiring exclusive use of the entire bucket. For example, setting the
prefix to nextcloud/$user/ will store each user's files under their own namespace within the bucket, with $user being substituted automatically at
mount time.

What changed:

Backend/AmazonS3.php: new optional prefix field exposed in the admin UI
Storage/AmazonS3.php: prefix is parsed and normalized in the constructor (leading slashes stripped, trailing slash enforced), then applied
transparently to all S3 key operations (read, write, delete, list, copy, presigned URLs). Keys returned from S3 listing responses have the prefix
stripped before being stored in the in-memory cache or yielded as directory entries. The storage ID includes the prefix so two mounts to the same
bucket with different prefixes are treated as distinct storages.

Variable substitution: $user (and any other registered placeholder) works in the prefix field automatically, since all backend options pass through
the existing applyConfigHandlers() pipeline before the storage is constructed.

TODO

Add unit tests for addPrefix / stripPrefix helpers and constructor normalization
Add integration test covering mount with a prefix set

Checklist

Code is properly formatted (https://docs.nextcloud.com/server/latest/developer_manual/digging_deeper/continuous_integration.html#linting)
Sign-off message (https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md) is added to all commits
Tests (unit (https://docs.nextcloud.com/server/latest/developer_manual/app_development/tutorial.html#unit-tests), integration
(https://docs.nextcloud.com/server/latest/developer_manual/app_development/tutorial.html#integration-tests), api and/or acceptance) are included
Screenshots before/after for front-end changes (admin external storage form shows new optional field)
Documentation (manuals (https://github.com/nextcloud/documentation/) or wiki) has been updated or is not required
Backports requested (https://github.com/nextcloud/backportbot/#usage) where applicable
Labels added (https://github.com/nextcloud/server/labels) — enhancement, 3. to review, files_external
Milestone added (https://github.com/nextcloud/server/milestones) for target branch/version

AI (if applicable)

The content of this PR was partly or fully generated using AI

Assisted-by: CloudeCode:claude-sonnet-4-6

susnux · 2026-06-25T12:41:55Z

Thank you for your pull request!
Please add the disclosure to your commits as per policy:
https://github.com/nextcloud/.github/blob/master/AI_POLICY.md#disclosure

Also you need to use conventional commit messages (e.g. feat(...): ...):
https://www.conventionalcommits.org/en/v1.0.0/

mbartsch · 2026-06-25T13:11:04Z

Hope now is OK, if not, please let me know.

github-actions · 2026-07-10T02:14:59Z

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

Signed-off-by: Marcelo H. Bartsch <marcelo@bartsch.cl> Signed-off-by: Carl Schwan <carlschwan@kde.org>

szaimen

🐘

provokateurin · 2026-09-09T13:31:58Z

+		$rawPrefix = ltrim(trim($parameters['prefix'] ?? ''), '/');
+		$this->prefix = $rawPrefix !== '' ? rtrim($rawPrefix, '/') . '/' : '';


This logic should go in to parseParams. It also looks a bit convoluted and I would suggest something like this instead:

Suggested change

$rawPrefix = ltrim(trim($parameters['prefix'] ?? ''), '/');

$this->prefix = $rawPrefix !== '' ? rtrim($rawPrefix, '/') . '/' : '';

$prefix = trim(trim(parameters['prefix'] ?? ''), '/');

$this->prefix = $prefix !== '' ? $prefix . '/' : '';

Merging the two trim calls would also be possible, but this variant is easier to understand IMO.

provokateurin · 2026-09-09T13:34:41Z

 						'Delete' => [
 							'Quiet' => true,
 							'Objects' => array_map(fn (array $object) => [
 								'Key' => $object['Key'],


I first thought that the prefix would need to be added here too, but I suppose the response will already contain the prefix. Can you just add a comment here explaining this, so the next person doesn't stumble over it?

provokateurin · 2026-09-09T13:35:56Z

+	private function batchDelete(string $path = ''): bool {
 		// TODO explore using https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.S3.BatchDelete.html
+		// an empty path means the whole storage: the bucket prefix, or the entire bucket when none is set
 		$params = [
-			'Bucket' => $this->bucket
+			'Bucket' => $this->bucket,
+			'Prefix' => $this->addPrefix($path === '' ? '' : $path . '/'),
 		];
-		if ($path !== null) {
-			$params['Prefix'] = $path . '/';
-		}


I'm not sure about this change, shouldn't it only ever allow deleting objects within the prefix?

susnux · 2026-09-09T16:42:26Z

@mbartsch thank you for the pull request again, would you still be interested in getting this merged?
In this case I think incorporate the review comments would be enough to get this merged :)

…rride Set $this->prefix inside a parseParams() override (called via the S3ConnectionTrait alias) instead of the constructor, so the prefix is available wherever parseParams() runs. Also clarifies batchDelete() comments on prefix scoping. Assisted-by: ClaudeCode:claude-sonnet-5 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Marcelo H. Bartsch <marcelo@bartsch.cl>

mbartsch force-pushed the feature/s3-external-prefix branch from 6a2538d to 7e55e24 Compare June 25, 2026 08:27

mbartsch mentioned this pull request Jun 25, 2026

feat(ExternalStorage) Allow the use of 'prefix' in S3 buckets nextcloud/documentation#15222

Open

4 tasks

mbartsch marked this pull request as ready for review June 25, 2026 09:12

mbartsch requested a review from a team as a code owner June 25, 2026 09:12

mbartsch requested review from ArtificialOwl, come-nc, provokateurin and salmart-dev and removed request for a team June 25, 2026 09:12

susnux added enhancement 3. to review Waiting for reviews feature: object storage labels Jun 25, 2026

susnux added this to the Nextcloud 35 milestone Jun 25, 2026

susnux added the community pull requests from community label Jun 25, 2026

mbartsch changed the title ~~Allow the use of 'prefix' in S3 buckets~~ feat(ExternalStorage) Allow the use of 'prefix' in S3 buckets Jun 25, 2026

CarlSchwan approved these changes Jun 29, 2026

View reviewed changes

CarlSchwan force-pushed the feature/s3-external-prefix branch 2 times, most recently from cc7a490 to c5e4cce Compare June 30, 2026 13:35

github-actions Bot added the feedback-requested label Jul 10, 2026

icewind1991 reviewed Jul 17, 2026

View reviewed changes

Comment thread apps/files_external/lib/Lib/Storage/AmazonS3.php Outdated

This was referenced Aug 12, 2026

35.0.0 beta 2 #63182

Merged

35.0.0 beta 3 #63375

Merged

nextcloud-bot mentioned this pull request Aug 20, 2026

35.0.0 beta 4 #63436

Merged

skjnldsv mentioned this pull request Aug 25, 2026

[stable35] 35.0.0 RC1 #63569

Merged

nextcloud-bot mentioned this pull request Aug 27, 2026

[stable35] 35.0.0 RC2 #63702

Merged

AndyScherzinger modified the milestones: Nextcloud 35, Nextcloud 36 Sep 4, 2026

feat: Allow the use of 'prefix' in S3 buckets

2c2daba

Signed-off-by: Marcelo H. Bartsch <marcelo@bartsch.cl> Signed-off-by: Carl Schwan <carlschwan@kde.org>

susnux force-pushed the feature/s3-external-prefix branch from 60abd45 to 2c2daba Compare September 9, 2026 13:23

szaimen approved these changes Sep 9, 2026

View reviewed changes

provokateurin requested changes Sep 9, 2026

View reviewed changes

mbartsch added 2 commits September 10, 2026 13:06

Merge branch 'master' into feature/s3-external-prefix

763acdb

susnux requested review from icewind1991 and provokateurin September 10, 2026 14:05

Navigation Menu

Uh oh!

Uh oh!

feat(ExternalStorage) Allow the use of 'prefix' in S3 buckets - #61585

feat(ExternalStorage) Allow the use of 'prefix' in S3 buckets#61585
mbartsch wants to merge 3 commits into
nextcloud:masterfrom
mbartsch:feature/s3-external-prefix

mbartsch commented Jun 25, 2026 •

edited

Loading

Uh oh!

susnux commented Jun 25, 2026

Uh oh!

mbartsch commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

Uh oh!

szaimen left a comment

Uh oh!

provokateurin Sep 9, 2026

Uh oh!

provokateurin Sep 9, 2026

Uh oh!

provokateurin Sep 9, 2026

Uh oh!

susnux commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

		$rawPrefix = ltrim(trim($parameters['prefix'] ?? ''), '/');
		$this->prefix = $rawPrefix !== '' ? rtrim($rawPrefix, '/') . '/' : '';

Uh oh!

Uh oh!

Conversation

mbartsch commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

susnux commented Jun 25, 2026

Uh oh!

mbartsch commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

Uh oh!

szaimen left a comment

Choose a reason for hiding this comment

Uh oh!

provokateurin Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

provokateurin Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

provokateurin Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

susnux commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mbartsch commented Jun 25, 2026 •

edited

Loading