configs: make the firmware filesystems actually readable (squashfs xz/lzo/lz4/zstd, xattrs, UBI/UBIFS, erofs) - #60
Merged
Conversation
Both kernel lines ship CONFIG_SQUASHFS=y without the xz decompressor, so a
guest that mounts a squashfs 4.0/xz image gets
Filesystem uses "xz" compression. This is not supported
mount: mounting /dev/loop0 on /sqfs failed: Invalid argument
Measured on penguin v3.1.7, mipseb, on both the 4.10 and 6.13 kernels; 6.13
prints the same line without the "squashfs:" prefix, which is why grepping a
console log for "squashfs" misses it there. Loop devices themselves are fine —
loop0 is created, capacity is detected and the whole filesystem probe list runs.
xz is the dominant squashfs compressor in firmware, and firmware that keeps a
*nested* squashfs inside its rootfs and loop-mounts it at boot is common in
switch and AP images (a kernel-embedded initramfs whose real userland lives in
a nested .img). Those targets cannot boot as shipped: the mount fails, and
every symlink into the mount point dangles. The failure also does not look like
a kernel-config problem from downstream — it reads as a broken extraction — so
it costs a while to find.
SQUASHFS_XZ depends only on SQUASHFS and selects XZ_DEC, whose per-arch BCJ
filter decoders are all default y, so nothing further is needed and olddefconfig
has no unmet dependency to drop it on. Every target chains to all-common.inc
(directly or via an arch-common.inc), so one line per version covers the matrix;
confirmed with config-explain on 4.10/{armel,x86_64} and 6.13/{mipseb,riscv64,
powerpc}.
Same class of gap as the previous commit, same one-line-per-option fix: lzo is what pre-4.x-era squashfs images use, lz4 and zstd are what recent OpenWrt-derived images use. Each depends only on SQUASHFS and selects its own decompress library, so like SQUASHFS_XZ they cost one line and survive olddefconfig. ZSTD support landed in squashfs in 4.14, so it is 6.13-only here. Unlike xz this is not backed by a measured failure yet — it is the rest of the compressor set the kernels can be asked for, added so the next image that uses one is not a second round of this. Drop this commit if you would rather add them on evidence.
This is the failure class the gate exists for. A kernel without it compiles, links, has the right ELF shape, boots, passes boot-check — and then cannot mount the firmware it was built to run. Nothing in the matrix notices; the symptom surfaces several repos downstream as "that target does not extract", which is where it cost real time. The entry also keeps the fragment honest: all-common.inc is the file everything chains to, so the one line that fixes every cell is also the one line a later config reshuffle can drop for every cell at once. requiredCheck reads the shipped .config, so it catches that. Deliberately only xz, not the other three decompressors added in the previous commit: xz is the one with a measured failure behind it, and the contract is meant to stay the subset whose absence is silent, not the list of options we happen to set.
Both kernels build squashfs and jffs2 without xattr support, so a firmware
image's extended attributes are dropped at mount time. The squashfs case is
visible in the shipped kernels: they carry the string
SQUASHFS error: Xattrs in filesystem, these will be ignored
which is the stub that gets compiled in when SQUASHFS_XATTR=n.
What is actually lost is `security.capability`. Firmware that ships a setcap'd
binary instead of a setuid one — increasingly the norm — gets a binary with no
capabilities under emulation, which does not fail at mount time or at exec: it
fails later, inside the program, as a permission error that looks like a missing
model or a wrong uid. SELinux/Smack labels on firmware images go the same way,
and both LSMs are enabled here.
JFFS2_FS_XATTR pulls POSIX_ACL and SECURITY with it (both default y under it);
SQUASHFS_XATTR has no sub-options. jffs2 is included because a jffs2 *rootfs* is
common on older NOR devices, and a jffs2 config partition alongside a squashfs
rootfs is common on newer ones — the same argument in both places.
Neither kernel has any UBI at all — no MTD_UBI, no UBIFS_FS. Verified on the shipped mipseb images: zero occurrences of UBIFS, ubi_attach or ubi_ctrl in either 4.10 or 6.13, despite MTD, JFFS2 and NAND all being on. UBI is how NAND firmware stores its rootfs. The standard modern layout is a squashfs rootfs on `ubiblock` with a UBIFS overlay for the writable half, and a guest that runs `ubiattach` or `mount -t ubifs` cannot get off the ground without these — the attach fails, so the volume never appears and neither does anything that lives on it. This is the NAND counterpart of the jffs2 support that is already here for NOR. MTD_UBI_BLOCK is what provides `/dev/ubiblock*`, so it is part of the same unit rather than an extra: without it UBIFS mounts but the squashfs-on-UBI half of that layout still cannot. The compressors come free — UBIFS_FS_LZO and UBIFS_FS_ZLIB are default y in both versions, plus UBIFS_FS_ZSTD in 6.13 — and UBIFS_FS depends on MTD_UBI, which is why both lines are needed for either to survive olddefconfig. Not included: MTD_UBI_GLUEBI, the compat layer that re-exposes UBI volumes as MTD devices. Firmware that expects mtdchar on top of UBI would want it, but it changes device enumeration for every target, so it deserves its own change with a target behind it.
erofs is absent from both kernels; the only EROFS hits in the shipped images are the `-EROFS` errno in a jffs2 message. 4.10 predates the filesystem entirely, so this is 6.13-only. It matters going forward rather than today: erofs is what Android images use and it is turning up in newer consumer firmware as the read-only rootfs, in the role squashfs has held. Adding it now costs four lines and means the first such image is a rehost rather than a kernel rebuild. EROFS_FS_ZIP (lz4) and EROFS_FS_XATTR are default y, so the three explicit lines are the compressors that are not: deflate, lzma and zstd. That mirrors what the first two commits do for squashfs — the filesystem being enabled says nothing about which images it can actually read.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Started as one line (squashfs xz) and grew into an audit of the same question across the
rest of the firmware filesystems: the filesystem being enabled says nothing about which
images it can actually read. Each commit stands alone and can be dropped.
The original gap
Both kernel lines ship
CONFIG_SQUASHFS=ywith no decompressor beyond zlib, so a guestmounting a squashfs 4.0/xz image gets:
Measured on penguin v3.1.7,
mipseb, on both 4.10 and 6.13. 6.13 prints the same linewithout the
squashfs:prefix, which is why grepping a console log for "squashfs" misses itthere. Loop devices are fine —
loop0is created, capacity is detected, the whole probelist runs. Only the decompressor is missing.
xz is the dominant squashfs compressor in firmware. The case that fails hard is firmware
holding a nested squashfs inside its own rootfs and loop-mounting it at boot — common in
switch and AP images built around a kernel-embedded initramfs. There the mount fails, every
symlink into the mount point dangles, and no service starts. It is also expensive to
diagnose: from downstream it does not look like a kernel-config problem, it reads as a
broken extraction, and the extractor gets blamed first.
Commits
SQUASHFS_XZon bothSQUASHFS_LZO,_LZ4,_ZSTD(zstd 6.13-only — squashfs gained it in 4.14)SQUASHFS_XZinto therequiredcontractSQUASHFS_XATTR+JFFS2_FS_XATTRXattrs in filesystem, these will be ignoredstubMTD_UBI+MTD_UBI_BLOCK+UBIFS_FSon bothUBIFS/ubi_attach/ubi_ctrlin either shipped kernelEROFS_FS+_ZIP_LZMA/_ZIP_DEFLATE/_ZIP_ZSTD(6.13)EROFShit is the errno in a jffs2 messageTwo of these are worth a second look rather than a rubber stamp:
required, which is otherwise scoped to IGLOO plumbing. The argumentis that this is exactly the class the file documents — builds, links, correct ELF shape,
boots, passes
boot-check, then cannot mount the firmware it exists to run — and thatall-common.incis where one line fixes every cell and where one reshuffle can drop itfor every cell at once. Only xz, not the other three.
(squashfs on
ubiblock+ a UBIFS overlay), and todayubiattachcannot succeed on anytarget. Whether UBI attaches cleanly over penguin's MTD models is untested — but it cannot
be tested at all while the kernel has no UBI.
Deliberately left out:
MTD_UBI_GLUEBI(changes device enumeration everywhere; wants its ownchange with a target behind it),
EXFAT_FS(removable media, not a rootfs), andF2FS_FS_COMPRESSION.Verification
config-explainconfirms fragment resolution for every new option across cells —4.10/{mipseb,armel,x86_64} and 6.13/{mipseb,armel,riscv64,powerpc}:
Every target chains to
all-common.inc(directly or via anarch-common.inc), so one lineper version covers the matrix.
That is fragments only. I did not build the matrix locally — cross-toolchain builds here
run for hours.
config-required,config-redundantandboot-checkare CI's, and they arethe checks that prove an option survived
olddefconfiginto the shipped.config. Thedependency reasoning says every option here will: each depends only on its filesystem (or, for
UBIFS_FS, onMTD_UBI, which is why both lines are in the same commit), and the librariescome in by
select—XZ_DECwith its per-arch BCJ decoders alldefault y,LZ4_DECOMPRESS,LZO_DECOMPRESS,ZSTD_DECOMPRESS,CRC16/CRC32for UBIFS.Not in this PR, but found while looking
HW_RANDOMandHW_RANDOM_VIRTIOare set on 6.13 and not on 4.10. 4.10 is the defaultkernel for most firmware, so on that line there is no entropy source even when qemu attaches
a virtio-rng device — which is the shape of the
getrandom()-wedges-a-daemon problem seenelsewhere. Different axis from filesystems, so it belongs in its own change; flagging it here
because this is where it turned up.