Trim dependencies to shrink the FlexMeasures Docker image

[TeaDrinkingProgrammer]

I was digging into why the FlexMeasures Docker image is ~2.3GB (v1.0.0-rc5), and a good chunk of it comes from timely-beliefs and other packages. I let AI dig through the dependencies, and these are some quick findings. I'll leave it to you to make the call on what is worth it and what is not.

Dependency sizes

flexmeasures

1.2G └─┬ . │█████████████████████ │ 100%
171M ├── llvmlite │███ │ 14%
143M ├── openturns.libs │███ │ 12%
88M ├── openturns │██ │ 7%
83M ├── vl_convert │██ │ 7%
79M ├── scipy │██ │ 7%
44M ├── pandas │█ │ 4%
35M ├── 08ae81f72d5a2b5fa9e0__mypyc.cpython-312-x86_64-linux-gnu.so│█ │ 3%
35M ├── statsmodels │█ │ 3%
33M ├── sklearn │█ │ 3%
32M ├── babel │█ │ 3%
29M ├── scipy.libs │█ │ 2%
28M ├── sktime │█ │ 2%
28M ├── numpy │█ │ 2%
27M ├── numpy.libs │█ │ 2%
24M ├── matplotlib │█ │ 2%
21M ├── docs │█ │ 2%
21M ├── pyomo │█ │ 2%
20M ├── fontTools │█ │ 2%
19M ├── examples │█ │ 2%
16M ├── numba │█ │ 1%
14M ├── cryptography │█ │ 1%
13M ├── pillow.libs │█ │ 1%
13M ├── sqlalchemy │█ │ 1%
11M ├── mypy │█ │ 1%
11M ├── highspy │█ │ 1%
10M ├── psycopg2_binary.libs │█ │ 1%
9.3M ├── lightgbm │█ │ 1%
6.9M ├── xarray │█ │ 1%
6.6M ├── holidays │█ │ 1%

timely-beliefs

542M └─┬ (total) │█████████████████████████████████████████████████████████████ │ 100%
143M ├── openturns.libs │█████████████████ │ 26%
88M ├── openturns │██████████ │ 16%
79M ├── scipy │█████████ │ 15%
44M ├── pandas │█████ │ 8%
33M ├── sklearn │████ │ 6%
29M ├── scipy.libs │████ │ 5%
28M ├── sktime │████ │ 5%
28M ├── numpy │████ │ 5%
27M ├── numpy.libs │████ │ 5%
13M ├── sqlalchemy │██ │ 2%
10M ├── psycopg2_binary.libs│██ │ 2%
2.6M ├── pytz │█ │ 0%
2.5M ├── tzdata │█ │ 0%
2.4M ├── timely_beliefs │█ │ 0%
2.1M ├── greenlet │█ │ 0%
1.3M ├── joblib │█ │ 0%
744K ├── skbase │█ │ 0%
500K ├── psutil │█ │ 0%
476K ├── psycopg2 │█ │ 0%
472K ├── dateutil │█ │ 0%
448K ├── dill │█ │ 0%
400K ├── packaging │█ │ 0%
164K ├── typing_extensions.py│█ │ 0%
84K ├── importlib_metadata │█ │ 0%
72K ├── isodate │█ │ 0%
64K ├── properscoring │█ │ 0%
52K ├── threadpoolctl.py │█ │ 0%
36K ├── six.py │█ │ 0%
32K └── zipp │█ │ 0%

Usage

package	size	In extra	used for
openturns(+libs)	241M	core dependency	only beliefs/probabilistic_utils.py and visualization/utils.py (probabilistic downsampling)
sktime	50M	the timely-beliefs[forecast] extra	only beliefs/classes.py (belief-formation model)
scipy(+libs)	139M	timely-beliefs core, but also darts, lightgbm, scikit-learn, statsmodels	shared numeric core

AI suggestions

1. Make openturns its own extra. This is the big one: 241M for one downsampling utility that most consumers of timely-beliefs (including FlexMeasures's default scheduling/forecasting path) never call. Suggestion: something like timely-beliefs[probabilistic] and importing openturns lazily inside probabilistic_utils.py/visualization/utils.py instead of at module load. Biggest win, but touches actual functionality.

2. Drop [forecast] on the FlexMeasures side. sktime only backs beliefs/classes.py's belief-formation model, and I can't find anywhere in FlexMeasures that touches it. If that's right, FlexMeasures can just depend on plain timely-beliefs instead of timely-beliefs[forecast]>=3.5.5 and drop 50M for free, no change needed here. Flagging it here too in case [forecast] is meant to cover something I'm missing.

3. scipy: not worth it. I already opened a PR removing scipy (and properscoring) as a direct dependency of timely-beliefs, but that will not change the image-size needle: darts, lightgbm, scikit-learn, and statsmodels all pull it in independently, so it stays in the resolved environment either way. Worth doing for the dependency hygiene, not for size.

[TeaDrinkingProgrammer]

New findings:

• Docker container pulls in dev dependencies (will make a PR for that)
• vl-convert-python is used for one CLI command (show-data-model). Move into an extra.
• darts is used extensively, but drags in lot's of dependencies. Would save a lot if it could be moved into an extra.

[TeaDrinkingProgrammer]

@Flix6x @nhoening