Replace flex-context field inflexible-device-sensors with inflexible-consumption and inflexible-production

[Flix6x]

Description

Replaces the flex-context field inflexible-device-sensors with two sign-explicit fields, applying the conventions established by #2190:

• inflexible-consumption: inflexible devices whose data records positive values as consumption.
• inflexible-production: inflexible devices whose data records positive values as production (the FlexMeasures default).

Each entry is a sensor reference ({"sensor": <id>}), optionally with source filters (source-types, exclude-source-types, sources, source-account), which the scheduler's read path now honors — so an inflexible device field can, for example, be pointed at forecaster beliefs only.

Deprecation & validation

• inflexible-device-sensors remains silently supported (no per-request warnings): its entries are read according to each sensor's consumption_is_positive attribute, as before. Metadata/docs mark it deprecated.
• Mixing the deprecated key with the new keys in one flex-context is rejected (check_prices-style steering error).
• A sensor listed under a new key whose explicit consumption_is_positive attribute contradicts the key's sign convention is rejected (same choice as feat: allow setting a consumption/production sensor in the flex-model of an asset #2190's output sensors); listing a sensor under both new keys is rejected too.
• The three keys form one field family when flex-contexts merge across sources: a request-level definition shadows the DB-stored family, and a nearer asset's definition shadows farther ancestors' — so the deprecated and newer keys never mix through merging.

Data migration (behavior-preserving, with downgrade)

Stored asset flex-contexts (incl. nested commodities entries) are migrated: each sensor id under the deprecated key becomes {"sensor": <id>} under inflexible-consumption (attribute True, with asset-attribute fallback) or inflexible-production (otherwise — the old default). Sensor attributes are kept, since trigger messages cannot be migrated. The downgrade merges entries back to bare ids (dropping any source filters — documented as lossy) and pins consumption_is_positive=True explicitly where needed to preserve read behavior.

Also included

• Scheduler support via the device inventory (FlexDevice.consumption_is_positive/sensor_reference; get_power_values gains an explicit sign parameter and source-filter support).
• Sequential scheduling injects previously scheduled sensors into the attribute-appropriate new key (or keeps bare ids for legacy contexts).
• UI flex-context editor offers both new multi-sensor pickers; the deprecated field shows read-only on unmigrated assets.
• Docs: flex-context table rows, sign-convention section, tutorial 2 switched to inflexible-production, API examples, OpenAPI spec regenerated.

How to test

• Core regression guard: pytest -k test_battery_solver_inflexible_key_equivalence (old-key schedule == new-key schedule).
• Sign semantics: pytest -k "test_get_power_values_sign_conventions_and_source_filters or test_battery_solver_inflexible_sign_flip".
• Validation: pytest flexmeasures/data/schemas/tests/test_scheduling.py -k inflexible and pytest -k test_trigger_schedule_inflexible_conflicts.
• Migration: verified in an up→down round trip against Postgres (incl. dangling ids, nested commodities, empty-list shadowing).

Related Items

• Closes Migrate inflexible-device-sensors to inflexible-consumption and inflexible-production on flex-context #2239. Part of Make flex-config more explicit about production vs consumption #2041.
• Supersedes Feat: make inflexible-device-sensors more explicit about production vs consumption (#2239) #2156 (complete redo on top of the device-inventory refactor and feat: allow setting a consumption/production sensor in the flex-model of an asset #2190's sign-convention groundwork).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TaMepbxRYzJxQKFtu6Hq4p

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #33840178 | 📁 Comparing 626da31 against latest (a3fb811)

  🔍 Preview build  

25 files changed · + 3 added · ± 22 modified

+ Added

• _autodoc/INFLEXIBLE_CONSUMPTION.html
• _autodoc/INFLEXIBLE_PRODUCTION.html
• _autosummary/flexmeasures.data.schemas.scheduling.groups.html

± Modified

• changelog.html
• genindex.html
• py-modindex.html
• _autodoc/GROUP.html
• _autodoc/INFLEXIBLE_DEVICE_SENSORS.html
• _autosummary/flexmeasures.api.v3_0.assets.html
• _autosummary/flexmeasures.api.v3_0.sensors.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.base.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.predict.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.train.html
• and 12 more...

[Copilot]

Pull request overview

This PR updates FlexMeasures’ flex-context schema and scheduling stack to replace the legacy inflexible-device-sensors field with two sign-explicit fields (inflexible-consumption and inflexible-production), aligning scheduling inputs with the consumption/production sign conventions introduced earlier (e.g. PR #2190). It also adds migration logic, UI/editor support, OpenAPI/doc updates, and tests to keep legacy behavior working while enabling optional source-filtered sensor references.

Changes:

• Add inflexible-consumption / inflexible-production flex-context fields (sensor references with optional source filters) while keeping inflexible-device-sensors deprecated-but-supported and validating conflicts.
• Extend scheduling/device-inventory plumbing to carry explicit sign conventions and source filters through to belief queries (get_power_values), including sequential-scheduling injection logic.
• Add a data migration to split stored inflexible-device-sensors into the new keys, plus docs/OpenAPI regeneration and test coverage.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
flexmeasures/ui/tests/test_utils.py	Excludes deprecated key from UI schema parity checks while documenting deprecation.
flexmeasures/ui/templates/assets/asset_context.html	Updates UI flex-context editor to treat inflexible device lists as sensor-reference entries and keep list cards active.
flexmeasures/ui/static/openapi-specs.json	Regenerates OpenAPI examples/schemas to reflect new inflexible-device fields and the new InflexibleDevice schema.
flexmeasures/data/tests/test_scheduling_sequential.py	Adds regression test for sequential scheduling injection when using sign-explicit inflexible fields.
flexmeasures/data/tests/test_generic_assets.py	Adds tests for inflexible key “family” shadowing and gathering across keys.
flexmeasures/data/services/sensors.py	Broadens sensor-reference pruning and ensures dict-valued flex-context metadata/status skips all inflexible keys.
flexmeasures/data/services/scheduling.py	Adds _add_inflexible_devices to inject previously scheduled sensors into the appropriate inflexible field family.
flexmeasures/data/schemas/tests/test_scheduling.py	Adds validation/deserialization tests for new inflexible fields and conflict cases.
flexmeasures/data/schemas/tests/conftest.py	Adds fixture sensors for inflexible-field schema tests with varying consumption_is_positive attributes.
flexmeasures/data/schemas/scheduling/metadata.py	Updates metadata for deprecated field and introduces metadata for the new sign-explicit fields.
flexmeasures/data/schemas/scheduling/init.py	Introduces InflexibleDeviceSchema, adds new flex-context fields, and validates conflicts/duplicates/sign-attribute contradictions.
flexmeasures/data/models/planning/utils.py	Extends get_power_values to accept SensorReference + source filters and an explicit sign parameter.
flexmeasures/data/models/planning/tests/test_utils_fresh_db.py	Adds tests for get_power_values sign semantics and source-filter behavior.
flexmeasures/data/models/planning/tests/test_solver.py	Adds solver-level regression tests for legacy/new key equivalence and sign flip behavior.
flexmeasures/data/models/planning/tests/test_device_inventory.py	Adds device-inventory tests for inflexible enumeration ordering and sign conventions, including per-commodity contexts.
flexmeasures/data/models/planning/storage.py	Switches inflexible-device constraint wiring to use inventory inflexible devices (incl. sign + references) rather than bare sensors.
flexmeasures/data/models/planning/devices.py	Defines inflexible-device key families/ordering and extends inventory to retain sign convention + optional sensor references.
flexmeasures/data/models/planning/init.py	Ensures passed flex-context shadows DB-stored inflexible-device key family to prevent cross-source mixing.
flexmeasures/data/models/generic_assets.py	Implements inflexible-device key-family shadowing in provenance merge and updates inflexible-device sensor gathering across keys.
flexmeasures/data/migrations/versions/3c2f9e5a1d47_split_inflexible_device_sensors_by_sign.py	Adds data-only migration (upgrade split, downgrade merge) for stored flex-contexts, including nested commodities.
flexmeasures/api/v3_0/tests/test_sensors_api.py	Extends sensor deletion pruning test coverage to include new inflexible keys.
flexmeasures/api/v3_0/tests/test_sensor_schedules_fresh_db.py	Adds API tests ensuring request-level sign-explicit fields shadow DB-stored legacy inflexible definitions and validate conflicts.
flexmeasures/api/v3_0/sensors.py	Updates API docs/examples to use inflexible-consumption / inflexible-production.
flexmeasures/api/v3_0/assets.py	Updates API docs/examples to use inflexible-consumption / inflexible-production.
documentation/tut/toy-example-expanded.rst	Updates tutorial text/examples to use inflexible-production sensor references.
documentation/tut/scripts/run-tutorial2-in-docker.sh	Updates tutorial runner script to pass the new flex-context JSON structure.
documentation/features/scheduling.rst	Documents the new flex-context fields in the scheduling features table.
documentation/concepts/data-model.rst	Documents sign conventions for scheduling inputs and legacy behavior for the deprecated key.
documentation/changelog.rst	Adds changelog entries for the new fields and the accompanying migration.
documentation/api/change_log.rst	Adds API changelog entry describing new fields, format, and validation/deprecation behavior.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.

[Flix6x]

Release QA ✅

Ran the QA (release) workflow against this branch (head 392ae73) — passed on the first run, no fixes needed.

Coverage:

• Full docker compose stack (server, worker, dev-db).
• Toy tutorials 1–5 via the in-repo runner scripts — including tutorial 2, which on this branch schedules the battery with the new inflexible-production field ('{"inflexible-production": [{"sensor": ...}]}'), so the sign-explicit path is exercised end-to-end through the CLI and scheduler.
• HEMS walkthrough from flexmeasures-client (main), incl. its CLI-based report generation — confirming the deprecated inflexible-device-sensors path clients still use keeps working unchanged.

Per the workflow's own note, this does not replace manual QA (UI login/graph checks, exploratory testing).