Skip to content

Strip symbol tables from the compiled extensions in the Docker image - #2439

Merged
Flix6x merged 7 commits into
mainfrom
fix/docker-strip-symbols
Aug 26, 2026
Merged

Strip symbol tables from the compiled extensions in the Docker image#2439
Flix6x merged 7 commits into
mainfrom
fix/docker-strip-symbols

Conversation

@Flix6x

@Flix6x Flix6x commented Aug 25, 2026

Copy link
Copy Markdown
Member

Description

Strips the symbol tables from the compiled extensions in the image. Most wheels ship them unstripped; nothing uses them at runtime.

Shares the changelog entry introduced by #2438 rather than adding one of its own, since both shrink the image through the builder stage. #2440 is stacked on this and widens the same entry again.

What changed

One RUN in the builder stage, after the existing rm -rf of sktime's stray payloads:

RUN command -v strip > /dev/null || { \
        echo "strip not found: the builder stage needs binutils" >&2; exit 1; \
    }; \
    find "${VIRTUAL_ENV}" \( -name '*.so' -o -name '*.so.*' \) -type f \
    -exec strip --strip-unneeded {} + 2>/dev/null || true

Placement matters. Because the runtime stage only does COPY --from=builder ${VIRTUAL_ENV}, the unstripped copies never reach a layer that ships — stripping in the runtime stage instead would grow the image, since the fat layer would still sit underneath. strip --strip-unneeded keeps everything dynamic linking needs, which is why the extensions stay loadable.

The explicit strip check came out of review: binutils only reaches this stage incidentally, as a transitive of the gcc install above. Without the check, the trailing || true would swallow strip: not found and turn the whole step into a no-op, growing the image back by ~130 MB with nothing failing and CI staying green. The || true stays on the find, where it is wanted — tolerating individual files strip cannot handle.

Measured

image size
main (with #2438) 1.52 GB
plus this PR 1.39 GB
−130 MB

Against main before #2438 the same step was worth 132.8 MB. The 2.8 MB difference is symbol tables in dev-only packages that --no-dev already removes, mostly mypy's mypyc extensions — the two changes overlap slightly, so the numbers should not be added.

Where it comes from, measured across all 726 shared objects belonging to locked distributions:

distribution saved % of its .so bytes
openturns 53.2 MB 22.6%
llvmlite 18.8 MB 11.2%
fonttools 10.0 MB 88.9%
scipy 9.3 MB 11.4%
kiwisolver, sqlalchemy, pillow, numba, numpy, cryptography, pyyaml, scikit-learn, greenlet ~28 MB 7–96%

How to test

docker build -t fm-strip .
docker run --rm fm-strip flexmeasures --version

Smoke-tested on the built image: the CLI starts (and stops at the missing SQLALCHEMY_DATABASE_URI, as expected without a database), and flexmeasures, timely_beliefs, openturns, scipy, numpy, pandas, sklearn, lightgbm, darts and statsmodels all import. openturns got the closest look, being both the biggest saver and a C++/SWIG library with a large ABI surface: Normal, UserDefined, NormalCopula, IndependentCopula, JointDistribution and Mixture all behave identically stripped.

The guard was checked on /bin/sh, the builder's shell, in both directions: with strip present it exits 0 and strips; with strip off the PATH it prints the message and exits 1.

Further Improvements

The bigger remaining items are about what is in the image rather than how it is packed: darts and its chain (shap/pyodnumballvmlite) is ~310 MB, openturns ~250 MB, vl-convert-python 88 MB. Ranked with their user-facing costs in #2437.

Related Items

Part of #2437. #2440 stacks on this.


Sign-off

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on code under GPL or another incompatible license.

🤖 Generated with Claude Code

TeaDrinkingProgrammer and others added 4 commits August 25, 2026 09:51
…tray docs/examples

Signed-off-by: Stijn van Houwelingen <teadrinkingprogrammer@proton.me>
Context:
- Review on #2438: the entry still carried the XXXX placeholder, in the label and the URL

Change:
- Pointed both at PR #2438

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- Most wheels ship their compiled extensions unstripped. The symbol tables and debug
  info they carry are never used at runtime, and openturns alone accounts for over a
  fifth of the weight

Change:
- Strip the shared objects in the builder stage, so only the stripped virtual
  environment is copied into the runtime image and no layer keeps the fat copy
- --strip-unneeded retains everything dynamic linking needs, so the extensions
  stay loadable

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- Both PRs shrink the image through the builder stage, so one entry reads better
  than two

Change:
- Expanded the entry #2438 introduced to mention the symbol stripping, and
  appended PR #2439 to its reference list

Signed-off-by: F.N. Claessen <felix@seita.nl>
@read-the-docs-community

read-the-docs-community Bot commented Aug 25, 2026

Copy link
Copy Markdown

Documentation build overview

📚 flexmeasures | 🛠️ Build #34238009 | 📁 Comparing 2554df2 against latest (3272084)

  🔍 Preview build  

2 files changed
± changelog.html
± api/v3_0.html

Signed-off-by: F.N. Claessen <felix@seita.nl>

# Conflicts:
#	Dockerfile
#	documentation/changelog.rst

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces the size of the FlexMeasures Docker image by stripping symbol tables from compiled Python extension modules during the builder stage, so only the stripped artifacts are copied into the runtime image. It also updates the changelog entry (stacked on #2438) to reflect the additional image-size reduction.

Changes:

  • Strip symbol tables from all *.so / *.so.* files inside ${VIRTUAL_ENV} during the builder stage to reduce final runtime image size.
  • Expand the existing Infrastructure / Support changelog entry (from #2438) to include symbol-table stripping and link both PRs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
Dockerfile Add builder-stage strip --strip-unneeded pass over shared objects to shrink the runtime image
documentation/changelog.rst Expand the existing Docker image shrinking entry to include symbol-table stripping and link PR #2439

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Dockerfile Outdated
Comment thread Dockerfile Outdated
Flix6x added 2 commits August 25, 2026 23:23
…pping

Context:
- Review: the trailing '|| true' also swallows 'strip: not found', which would
  turn this step into a no-op and let the image quietly grow back by ~130 MB
  with nothing failing. binutils arrives via gcc today, but that is incidental

Change:
- Check for strip up front and exit 1 with a message naming binutils
- Kept '|| true' on the find, which is there to tolerate individual files that
  strip cannot handle
- Also reworded the size note, which read ungrammatically

Signed-off-by: F.N. Claessen <felix@seita.nl>
Context:
- The repo convention is that each physical line of a comment ends at punctuation,
  so review comments and text search stay stable

Change:
- Reflowed the comment block, and tightened it from seven lines to five

Signed-off-by: F.N. Claessen <felix@seita.nl>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@Flix6x Flix6x added this to the 1.1.0 milestone Aug 26, 2026
@Flix6x
Flix6x merged commit f05e892 into main Aug 26, 2026
14 checks passed
@Flix6x
Flix6x deleted the fix/docker-strip-symbols branch August 26, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants