feat(drivers/alias): add autoMakeDir for missing dest#2817
Conversation
…n folder if it does not exist Co-authored-by: Claude <claude@users.noreply.github.com>
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c3ed6c944f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return nil, errs.PermissionDenied | ||
| } | ||
| return d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.WriteConflictPolicy)) | ||
| return d.getAllObjs(ctx, obj, getWriteAndPutFilterFunc(d.WriteConflictPolicy), isAllWritePolicy(d.WriteConflictPolicy)) |
There was a problem hiding this comment.
When WriteConflictPolicy is all/all_strict, getWriteObjs is also used by Rename and Remove for the object being operated on, not just by MakeDir for a destination parent. If /foo is a directory that exists on backend A but is missing on backend B, renaming it now creates B's /foo and then renames that empty directory to /bar, so the alias gains a new empty destination on a backend that never had the source content. Limit autoMakeDir to destination-parent resolution paths such as upload/copy/move/mkdir, not source-object resolution.
Useful? React with 👍 / 👎.
| } | ||
| } | ||
| // 目标为目录但该后端不存在时，按需自动创建，使新加入的存储也能参与写入 | ||
| if err != nil && autoMakeDir && bObj.IsDir() && errs.IsObjectNotFound(err) { |
There was a problem hiding this comment.
The auto-create branch only has a backend path for *tempObjs, but Alias.Get collapses any missing backend that appears before the first existing copy into nil, so this still misses those roots. For example, with alias paths [new, old] and /album present only on old, /album resolves to the old object plus nil; an all upload/mkdir under /album never sees new/album to create it, and all_strict still fails. The missing backend path needs to be preserved instead of represented only as nil when auto-creation is enabled.
Useful? React with 👍 / 👎.
| return nil, nil, errs.PermissionDenied | ||
| } | ||
| dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy)) | ||
| dstObjs, err := d.getAllObjs(ctx, dstDir, getWriteAndPutFilterFunc(d.PutConflictPolicy), isAllWritePolicy(d.PutConflictPolicy)) |
There was a problem hiding this comment.
For PutConflictPolicy all/all_strict, resolving dstDir now creates missing destination parent directories before getMoveObjs checks whether there are enough source copies to move. If a file or folder exists on only one backend but the destination parent is auto-created on two backends, the later len(tmpSrcObjs) < len(dstObjs) check returns ErrNotEnoughSrcObjs, leaving the newly-created destination directory behind even though the move failed. Resolve the source count before performing mkdir side effects, or skip auto-creation for move cases that cannot be satisfied.
Useful? React with 👍 / 👎.
| } | ||
| // 目标为目录但该后端不存在时，按需自动创建，使新加入的存储也能参与写入 | ||
| if err != nil && autoMakeDir && bObj.IsDir() && errs.IsObjectNotFound(err) { | ||
| if mkErr := fs.MakeDir(ctx, o.GetPath()); mkErr == nil { |
There was a problem hiding this comment.
With all_strict, this loop creates each missing backend directory as it is encountered and immediately clears the error, but a later backend can still fail MakeDir; in that case getAllObjs returns before the upload/copy/mkdir runs while any earlier auto-created directories remain. For a destination that is missing on two backends where only the first can be created, a failed strict operation now leaks an empty directory onto the first backend. Preflight the targets or undo directories created during this resolution step when a later strict target cannot be prepared.
Useful? React with 👍 / 👎.
xrgzs
changed the title
autoMakeDir to automatically create the destinatio…autoMakeDir for missing dest
xrgzs
changed the title
autoMakeDir for missing destautoMakeDir for missing dest
3 participants
Summary / 摘要
为
alias驱动的all/all_strict写入策略增加目标目录缺失时自动创建，使新加入的存储也能参与写入。此前，写入操作的「有效路径」仅指目标对象已存在且类型匹配的后端路径。当某个后端（例如新加入的存储）缺少对应的子目录时，会被判定为无效路径而跳过，导致数据不会写入该后端。本次改动在解析写入目标时，对返回
ObjectNotFound的目录后端自动调用fs.MakeDir创建目录后纳入写入目标。实现要点：
getAllObjs()新增autoMakeDir bool参数；当目标为目录、后端返回ObjectNotFound且开关开启时，自动fs.MakeDir创建后继续参与写入。新增
isAllWritePolicy()辅助函数，判断策略是否为all/all_strict。写入目标（Put/Copy/Mkdir/Rename/Remove 的目标目录）在
all/all_strict策略下启用自动建目录；源对象与读操作（Link 等）保持false，不受影响。This PR has breaking changes.
/ 此 PR 包含破坏性变更。
This PR changes public API, config, storage format, or migration behavior.
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
This PR requires corresponding changes in related repositories.
/ 此 PR 需要关联仓库同步修改。
Related repository PRs / 关联仓库 PR:
Testing / 测试
Checklist / 检查清单
/ 我已阅读 CONTRIBUTING。
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
gofmt,go fmt, orprettierwhere applicable./ 我已按适用情况使用
gofmt、go fmt或prettier格式化变更代码。/ 我已在适用情况下请求相关维护者或代码所有者审查。
AI Disclosure / AI 使用声明
/ 此 PR 包含 AI 辅助内容。
Tools used / 使用工具:
Usage scope / 使用范围:
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。
Co-Authored-Byattribution./ 我已确保所有 AI 辅助提交都包含
Co-Authored-By归属信息。/ 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。