Conversation
There was a problem hiding this comment.
Pull request overview
This PR migrates the simulator worker execution path to a Prefect-based flow/deployment model while modernizing development tooling (uv + ruff + ty), CI, and container build strategy.
Changes:
- Introduces a new Prefect flow (
simulator_flow) and deployment script, plus supporting env helpers and local-run entrypoint. - Migrates tooling/config: switches to uv build backend and dependency groups, replaces flake8/mypy with ruff/ty, and updates CI + just tasks accordingly.
- Updates/expands tests and fixtures to exercise the Prefect flow path and deployment job variables.
Reviewed changes
Copilot reviewed 43 out of 46 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| unit_test/test_hello.py | Reformats legacy unittest file (now duplicated by tests/test_hello.py). |
| tests/test_prefect_flow.py | Adds a Prefect flow execution test with MinIO/InfluxDB mocking. |
| tests/test_prefect_deploy_flow.py | Adds coverage for module-level job_variables structure. |
| tests/test_kpi_integration.py | Reworks KPI integration to call Prefect flow and handle Prefect State. |
| tests/test_hello.py | Adds new test suite for config parsing/date indexing under tests/. |
| tests/data/esdl/simulator_tutorial.esdl | Adds ESDL fixture used by Prefect flow tests/local run. |
| src/simulator_worker/worker_task.py | Adds a legacy worker task module, currently fully commented out. |
| src/simulator_worker/utils.py | Refactors utilities typing/logging; adds _parse_datetime_config; tweaks ESDL output naming. |
| src/simulator_worker/simulator_worker.py | Removes previous main worker entrypoint implementation. |
| src/simulator_worker/prefect_flow.py | Adds Prefect flow implementation and artifact writing to MinIO. |
| src/simulator_worker/prefect_deploy_flow.py | Adds Prefect deployment registration script and job variables. |
| src/simulator_worker/env.py | Adds environment access helpers (require_env, EnvSettings). |
| src/simulator_worker/main.py | Removes previous __main__ entrypoint. |
| src/simulator_worker/init.py | Changes init behavior to load dotenv (removes logging + script export). |
| run.sh | Removes old local run script. |
| run.ps1 | Removes old PowerShell helper script. |
| requirements.txt | Removes pip-compile generated runtime requirements. |
| README.md | Updates dev workflow docs for uv/just/ruff/ty and Prefect deploy/run steps. |
| pyproject.toml | Switches to uv build backend, updates deps, and replaces flake8/mypy config with ruff/ty-era config. |
| local_run/run_simulator_flow_function.py | Adds local runner for calling the Prefect flow function without Prefect orchestration. |
| justfile | Adds standardized local/CI task runner commands. |
| Dockerfile | Reworks container build to use uv and BuildKit mounts; removes previous entrypoint-based install flow. |
| dev.Dockerfile | Reworks dev image build to use uv and allow local SDK install. |
| dev-requirements.txt | Removes pip-compile generated dev requirements. |
| ci/win32/update_dependencies.cmd | Removes legacy Windows dependency update script. |
| ci/win32/typecheck.cmd | Removes legacy Windows mypy script. |
| ci/win32/test_unit.cmd | Removes legacy Windows unittest/pytest script. |
| ci/win32/lint.cmd | Removes legacy Windows flake8 script. |
| ci/win32/install_dependencies.cmd | Removes legacy Windows pip-sync script. |
| ci/win32/create_venv.cmd | Removes legacy Windows venv bootstrap script. |
| ci/linux/update_dependencies.sh | Removes legacy Linux dependency update script. |
| ci/linux/typecheck.sh | Removes legacy Linux mypy script. |
| ci/linux/test_unit.sh | Removes legacy Linux pytest runner script (unit_test-based). |
| ci/linux/lint.sh | Removes legacy Linux flake8 script. |
| ci/linux/install_dependencies.sh | Removes legacy Linux pip-sync script. |
| ci/linux/create_venv.sh | Removes legacy Linux venv bootstrap script. |
| ci/linux/build_python_package.sh | Removes legacy package build script. |
| ci/linux/_load_dot_env.sh | Removes legacy dotenv loader script. |
| .vscode/settings.json | Updates editor integration for ruff/ty; switches test discovery to tests/. |
| .vscode/launch.json | Adds debug configs for Prefect deploy, local flow run, and pytest. |
| .gitignore | Renames coverage ignore path from unit_test_coverage/ to tests_coverage/. |
| .github/workflows/publish_container_image.yml | Modernizes GHCR publish workflow using buildx + metadata-action. |
| .github/workflows/ci.yml | Replaces venv-tar pipeline with uv+just jobs (lint/security/test/typecheck). |
| .env.template.local | Updates local env template for Prefect/MinIO/InfluxDB settings. |
| .env.template | Updates env template for Prefect/MinIO/InfluxDB settings. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+83
to
+87
| if minio_host is None or minio_port is None or minio_access_key is None or minio_secret is None: | ||
| raise ValueError( | ||
| f"MinIO credentials are not fully set. MinIO host: {minio_host}, port: {minio_port}, " | ||
| f"access key: {minio_access_key}, secret key: {minio_secret}" | ||
| ) |
Comment on lines
+359
to
+366
| value = config[key] | ||
| if not isinstance(value, str): | ||
| return default | ||
|
|
||
| try: | ||
| return datetime.fromisoformat(value) | ||
| except ValueError: | ||
| return default |
Comment on lines
+61
to
+63
| Returns: | ||
| SimulatorFlowResult | State[Any] | None: Failed state when execution fails; otherwise no value is | ||
| returned from this flow function. |
Comment on lines
+16
to
+20
| # """Main python file for Simulator-worker.""" | ||
| # import logging | ||
| # import math | ||
| # import traceback | ||
| # from datetime import datetime, timedelta |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.