Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 58 additions & 58 deletions .plans/files-gateway-asset-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Status

Proposed.
Implemented; awaiting review.

## Objective

Expand All @@ -13,7 +13,7 @@ Exercise one complete authenticated file lifecycle through the API application:
3. a Files SDK `onAction` hook records the object's ownership and metadata in the
`storage.assets` table;
4. another user cannot access the object; and
5. deleting the object marks the asset record as deleted.
5. deleting the object removes the asset record.

Keep the Files SDK gateway at the versionless `/files` transport seam. Reserve
`/v1/assets` for a future application-owned HTTP interface over asset IDs, ownership,
Expand All @@ -33,14 +33,15 @@ organizations, metadata, and references.
- The route does not perform a second Better Auth session lookup inside `authorize`.
- Successful upload completion creates or updates one `storage.assets` row owned by
the authenticated user.
- Successful deletion marks the matching asset row as `deleted`.
- Successful deletion removes the matching asset row owned by the authenticated user.
- `onAction` dispatches successful events to local `handleUpload` and `handleDelete`
functions in the singleton module; there is no persistence service or custom Files
SDK plugin.
- Repeated upload-completion events are idempotent through the existing unique index
on `(bucket, key)`.
- Repeated upload-completion events are idempotent through the unique index on `key`.
- Database failures from the fire-and-forget hook are caught and logged without
producing unhandled promise rejections.
- The template contains exactly one baseline database migration generated from the
current schema.
- Formatting, static checks, dependency analysis, and relevant build verification
pass.

Expand Down Expand Up @@ -91,26 +92,27 @@ Use a switch to keep the hook concise. Single upload and head events call
`handleDelete`. These handlers remain private functions in `shared/files.ts` and
execute the Drizzle operations directly.

### Keep the current asset schema
### Use a Files-SDK-shaped asset record

The existing `storage.assets` table already contains the fields required by this
slice:
The `storage.assets` table stores the stable asset ID, Files SDK metadata, and two
distinct user roles:

- `(bucket, key)` provides storage identity and idempotency;
- `uploaderId` records the authenticated owner for the current model;
- `provider`, `mimeType`, `size`, `name`, and `metadata` describe the object; and
- `status` represents `available` and `deleted` lifecycle states.
- `key`, `name`, `size`, `type`, `etag`, `lastModified`, and `metadata` mirror the
stored file returned by Files SDK;
- `ownerId` records the user to whom the asset belongs; and
- `uploaderId` records the user who placed the asset in managed storage.

Do not rename `uploaderId`, remove organization support, or otherwise redesign the
schema in this change. Those decisions can be made when the application-owned
`/v1/assets` interface is designed.
The current upload flow assigns the authenticated user to both roles. Keeping them
separate preserves creation provenance if ownership changes later. The configured
Files singleton owns one bucket, so the storage key is the unique object identity and
bucket/provider columns are unnecessary.

### Observe upload and head completion

Handle successful `upload` events and successful `head` events. Direct/proxied uploads
can produce an upload result, while completion of a direct or presigned upload can be
confirmed through a follow-up `head`. The `(bucket, key)` conflict target makes both
paths idempotent.
confirmed through a follow-up `head`. The unique `key` conflict target makes both paths
idempotent.

Do not support copy or move until their destination-ownership behavior is explicitly
defined. Do not build the index from list results.
Expand Down Expand Up @@ -196,9 +198,9 @@ export const requireSession = createMiddleware<AuthenticatedAppContext>(async (c
Replace `apps/api/src/shared/files.ts` with this proposed implementation:

```ts
import { assets } from "@init/db/schema"
import { assets, type UserId } from "@init/db/schema"
import * as z from "@init/utils/schema"
import { and, eq, inArray } from "drizzle-orm"
import { helpers } from "@init/db/helpers"
import type { DeleteManyResult, StoredFile, UploadResult } from "files-sdk"
import { createFiles } from "files-sdk"
import { bunS3 } from "files-sdk/bun-s3"
Expand Down Expand Up @@ -238,6 +240,8 @@ export const files = createFiles({
handleDelete(keys)
break
}
default:
break
}
},
},
Expand Down Expand Up @@ -271,27 +275,27 @@ function handleUpload(key: string, file: UploadResult | StoredFile) {
void ctx.var.db
.insert(assets)
.values({
bucket: env.S3_BUCKET,
etag: file.etag,
key,
metadata: file.etag ? { etag: file.etag } : null,
mimeType,
lastModified: file.lastModified,
metadata: "metadata" in file ? file.metadata : undefined,
name,
provider: "bun-s3",
ownerId: ctx.var.session.user.id as UserId,
size: file.size,
status: "available",
uploaderId: ctx.var.session.user.id,
type: mimeType,
uploaderId: ctx.var.session.user.id as UserId,
})
.onConflictDoUpdate({
target: [assets.bucket, assets.key],
set: {
errorMessage: null,
metadata: file.etag ? { etag: file.etag } : null,
mimeType,
etag: file.etag,
lastModified: file.lastModified,
metadata: "metadata" in file ? file.metadata : undefined,
name,
size: file.size,
status: "available",
type: mimeType,
updatedAt: new Date(),
},
target: assets.key,
})
.catch((error: unknown) => {
ctx.var.logger.error(`Failed to record asset: ${String(error)}`)
Expand All @@ -304,20 +308,15 @@ function handleDelete(keys: string[]) {
const ctx = context<AuthenticatedAppContext>()

void ctx.var.db
.update(assets)
.set({
status: "deleted",
updatedAt: new Date(),
})
.delete(assets)
.where(
and(
eq(assets.bucket, env.S3_BUCKET),
inArray(assets.key, keys),
eq(assets.uploaderId, ctx.var.session.user.id)
helpers.and(
helpers.inArray(assets.key, keys),
helpers.eq(assets.ownerId, ctx.var.session.user.id as UserId)
)
)
.catch((error: unknown) => {
ctx.var.logger.error(`Failed to mark asset as deleted: ${String(error)}`)
ctx.var.logger.error(`Failed to delete asset records: ${String(error)}`)
})
}
```
Expand All @@ -328,8 +327,8 @@ an authenticated request.

This intentionally makes action-producing use of this singleton request-bound: a
future caller that invokes it outside Hono context would not have a session to assign
as `uploaderId` and must use a separately configured Files instance or establish an
equivalent ownership context.
as `ownerId` and `uploaderId` and must use a separately configured Files instance or
establish an equivalent ownership context.

### 4. Implement the complete versionless gateway route

Expand Down Expand Up @@ -390,30 +389,32 @@ keys that were successfully removed change status.
`onAction` to read the session from Hono context storage. Remove the old direct `auth`
import, `auth.api.getSession()` call, and `FilesError` authorization branch.

### 5. Declare the direct Drizzle dependency
### 5. Expose Drizzle through the database package

Because the shared Files module constructs Drizzle expressions itself, add the
installed Drizzle version as a direct dependency of `apps/api`:
Keep Drizzle's package identity behind `@init/db` and expose its helpers as a namespace
from `packages/db/src/helpers.ts`:

```diff
"dependencies": {
"@init/workflows": "workspace:*",
"@scalar/hono-api-reference": "^0.9.48",
"@trpc/server": "11.8.1",
+ "drizzle-orm": "0.45.2",
"files-sdk": "2.2.2",
"hono": "4.12.32"
}
```ts
export * as helpers from "drizzle-orm"
```

The API imports `helpers` from `@init/db/helpers`; it does not declare a separate direct
Drizzle dependency.

### 6. Limit the initial operation surface

The concrete `operations` list above permits upload, direct/presigned upload, metadata,
listing, download, URL, and deletion. It excludes `copy` and `move` because the hook
does not update destination ownership, and excludes versioning and trash operations
because the configured Files instance does not install those plugins.

### 7. Verify the change
### 7. Regenerate the baseline migration

Delete the existing migration SQL and metadata, then run `bun run generate` from
`packages/db`. The template deliberately keeps one starting migration and does not
preserve migration history for databases created from earlier template revisions.

### 8. Verify the change

Run the repository-managed static checks:

Expand All @@ -430,7 +431,6 @@ this change.
## Explicitly out of scope

- Implementing `/v1/assets`.
- Renaming `uploaderId` or redesigning the asset schema.
- Making the asset table the source of truth for object existence.
- Transactional guarantees between object storage and Postgres.
- A reconciliation worker for missing or stale asset rows.
Expand All @@ -442,8 +442,8 @@ this change.

- Design the application-owned `/v1/assets` interface around asset IDs rather than raw
storage keys.
- Decide whether ownership should become `userId`, a polymorphic owner, or separate
user/organization ownership records.
- Decide whether ownership should expand from a User owner to a polymorphic owner or
separate user/organization ownership records.
- Add reconciliation if production requirements demand repair of missed hook writes.
- Define database behavior for copy, move, bulk operations, and failed uploads.
- Add domain references from future records to `assets.id` instead of duplicating
Expand Down
11 changes: 11 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,14 @@ _Avoid_: Required backend, backend layer
**Preset**:
A reusable environment or tooling configuration selected by a local project generator.
_Avoid_: Template recipe

**Asset**:
A stored file represented by a stable identity so other domain records can refer to
it. An Asset has one Owner and records the Uploader who placed it in managed storage.

**Asset Owner**:
The User to whom an Asset belongs. Ownership can differ from creation provenance.

**Asset Uploader**:
The User who placed an Asset in managed storage. The Uploader does not necessarily
remain its Owner.
2 changes: 1 addition & 1 deletion apps/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@

API server built with [Hono](https://hono.dev/).

The versioned router includes the authenticated Files SDK gateway at `/v1/files`.
The API server includes the authenticated Files SDK gateway at `/files`.
Provider composition and upload policy live in `src/shared/files.ts`; local development
uses the MinIO service and `assets` bucket from `infra/local/docker-compose.yml`.
40 changes: 40 additions & 0 deletions apps/api/src/routes/files.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { createFilesRouter } from "files-sdk/api"
import { createRouteHandler } from "files-sdk/hono"
import type { AuthenticatedAppContext } from "#shared/types.ts"
import env from "#shared/env.ts"
import { files, FILES_MAX_UPLOAD_SIZE, FILES_MAX_URL_AGE } from "#shared/files.ts"
import { requireSession } from "#shared/middleware.ts"
import { context, factory } from "#shared/utils.ts"

const router = createFilesRouter({
allowedOrigins: env.ALLOWED_API_ORIGINS,
authorize: () => {
const ctx = context<AuthenticatedAppContext>()

return {
disposition: "attachment",
keyPrefix: `users/${ctx.var.session.user.id}/`,
maxExpiresIn: FILES_MAX_URL_AGE,
maxResults: 100,
}
},
files,
maxListLimit: 100,
maxSearchResults: 100,
maxUploadSize: FILES_MAX_UPLOAD_SIZE,
operations: [
"capabilities",
"delete",
"download",
"exists",
"head",
"list",
"search",
"signedUploadUrl",
"upload",
"url",
],
secret: env.FILES_API_SECRET,
})

export default factory.createApp().all("/", requireSession, createRouteHandler(router))
4 changes: 4 additions & 0 deletions apps/api/src/routes/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ import { contextStorage } from "hono/context-storage"
import { cors } from "hono/cors"
import { HTTPException } from "hono/http-exception"
import { secureHeaders } from "hono/secure-headers"
import filesRoutes from "#routes/files.ts"
import healthRoutes from "#routes/health.ts"
import trpcRoutes from "#routes/trpc.ts"
import v1Routes from "#routes/v1/index.ts"
import workflowRoutes from "#routes/workflows.ts"
import { auth } from "#shared/auth.ts"
import env from "#shared/env.ts"
import { files } from "#shared/files.ts"
import { LoggerCategory, logger } from "#shared/logger.ts"
import { withLanguageDetection } from "#shared/middleware.ts"
import { factory } from "#shared/utils.ts"
Expand Down Expand Up @@ -42,6 +44,7 @@ app.use(
app.use(async (c, next) => {
c.set("auth", auth)
c.set("db", database())
c.set("files", files)
c.set("kv", kv())
c.set("logger", logger)
await next()
Expand Down Expand Up @@ -82,6 +85,7 @@ export const router = app
})
)
.route("/health", healthRoutes)
.route("/files", filesRoutes)
Comment thread
adelrodriguez marked this conversation as resolved.
.route("/workflows", workflowRoutes)
.route("/trpc", trpcRoutes)
.route("/v1", v1Routes)
Expand Down
30 changes: 0 additions & 30 deletions apps/api/src/routes/v1/files.ts

This file was deleted.

2 changes: 0 additions & 2 deletions apps/api/src/routes/v1/index.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import * as z from "@init/utils/schema"
import { describeRoute, resolver, validator } from "hono-openapi"
import filesRoutes from "#routes/v1/files.ts"
import { m } from "#shared/internationalization/messages.js"
import { requireSession } from "#shared/middleware.ts"
import { factory } from "#shared/utils.ts"

export default factory
.createApp()
.route("/files", filesRoutes)
.get(
"/hello",
describeRoute({
Expand Down
Loading