Skip to content

Scheduled prune for hosted social media derivatives (all networks) #121

Description

@paulocastellano

Context

Some social networks pull media from a URL instead of accepting an uploaded byte stream (TikTok, Instagram, Threads, Facebook). For those, when we need a processed copy of an image (aspect-ratio crop, or a resized/spec-compliant version), we render a derivative, host it on our public disk, and hand the platform that URL.

Today there are two derivative directories:

Directory Written by Used for Cleanup today
social-crops/ App\Services\Social\Concerns\CropsImageForAspectRatio (:42) Instagram/Facebook aspect-ratio crops None — leaks forever
social-tiktok-photos/ App\Services\Social\TikTokPublisher::renderCompliantPhoto (:333) TikTok photos resized to ≤1080px (PR #120) Inline Storage::delete in a finally after publish

Neither approach is right long-term:

  1. social-crops/ leaks. Every Instagram/Facebook post with an aspect ratio writes a JPEG that is never deleted. These accumulate on disk indefinitely.
  2. social-tiktok-photos/ inline prune has a rare timing risk. The derivative is deleted in a finally right after waitForPublishStatus returns. On the normal PUBLISH_COMPLETE path (and on failure paths) TikTok has already pulled the image, so it's safe. But if the status poll times out (~60s) while TikTok is still pulling, the inline delete removes an image the platform still needs.

Proposal

Replace the per-publisher cleanup strategy with a single scheduled prune command that periodically deletes derivative files older than a safe window (e.g. 1 hour — far longer than any platform needs to pull) from all social-derivative directories.

This is intentionally generic so that every current and future social network that hosts a pull-from-URL derivative is covered by one mechanism.

Tasks

  • Create php artisan make:command PruneSocialMediaDerivatives (sibling of the existing scheduled commands: ProcessScheduledPosts, RefreshExpiringTokens, CheckSocialConnections).
    • Delete files older than a configurable age (default 1h) from the known derivative directories: social-crops/, social-tiktok-photos/.
    • Source the directory list from a single place (e.g. a shared constant/config) so adding a new network's directory is one line.
  • Register it in the scheduler (->hourly() or similar) in routes/console.php / bootstrap/app.php.
  • Remove the inline finally prune in TikTokPublisher::publishPhotos (:278) now that cleanup is centralized — this also removes the timeout risk above.
  • Tests: command deletes old files, keeps recent ones, and is a no-op when the directories are empty/missing.

Notes / future-proofing

  • When a new pull-from-URL platform starts hosting derivatives (e.g. if we ever generalize the TikTok resize to Instagram/Threads/Facebook), it must write into a directory registered with this prune, not invent an uncovered one.
  • Use the application default disk (never force a disk name), consistent with the existing publishers.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions