Add package for converting tar to erofs - #289
Draft
dmcgowan wants to merge 3 commits into
Draft
Conversation
dmcgowan
force-pushed
the
tar-erofs
branch
3 times, most recently
from
July 24, 2026 06:32
5aaabb8 to
a1c62e0
Compare
Adds package github.com/containerd/continuity/tarconv, which converts a tar stream into an EROFS image by translating each tar entry directly into a go-erofs Writer call, without staging an intermediate fs.FS. AUFS/OCI whiteouts are translated to their overlayfs representation by default, matching mkfs.erofs --aufs. WithMerge resolves them structurally instead, and WithPreserveWhiteouts keeps them as plain files. Hard links may appear in any order; links whose target has not yet been seen are queued and replayed as later entries arrive. A hard-link entry's own header metadata is deliberately not applied: every name in a hard-link group refers to one inode, so mode/uid/gid belong to that inode and are already established by the entry the link points to. The fields are also not guaranteed to be populated on a TypeLink header, and because the inode is shared, writing a zero mode through the link name would clear the permission bits for every name in the group. go.mod is raised to go 1.23 and CI to Go 1.23.x, as required by go-erofs. Signed-off-by: Derek McGowan <derek@mcg.dev>
WithTarIndexData enables tar-index mode in Apply. When set, file payload bytes are written to an external data file (via go-erofs WithDataFile) at 512-byte-block-aligned positions, and the EROFS writer records chunk-index entries pointing into that file rather than copying bytes into its internal spool. The resulting EROFS image is metadata-only: filesystem structure, inode table, and chunk-index table. Combining it with the data file produces a blob where a kernel consumer can locate file content directly via DeviceID=1 chunk references. This replaces the mkfs.erofs --tar=i --aufs subprocess invocation with a pure-Go implementation. Block size is set to 512 to match tar's natural granularity (each header block is 512 bytes). Adds four tests: - TestWithTarIndexDataBasic: round-trips a file through the combined blob - TestWithTarIndexDataFileListing: compares file listing with full extraction - TestWithTarIndexDataWhiteouts: verifies OCI whiteout translation - TestWithTarIndexDataXattrs: reads back multi-block payloads whose inodes also carry xattrs, which offsets the chunk-index map within the inode Signed-off-by: Derek McGowan <derek@mcg.dev>
Signed-off-by: Derek McGowan <derek@mcg.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds package
github.com/containerd/continuity/tarconvwhich uses the go-erofs library to convert a tar stream into erofs.