diff --git a/flash/cli/build.mdx b/flash/cli/build.mdx index 53b368844..fd9972dde 100644 --- a/flash/cli/build.mdx +++ b/flash/cli/build.mdx @@ -76,6 +76,21 @@ Flash also respects your `.gitignore` file and excludes any files matching those If you use other environment file variants like `.env.dev` or `.env.staging`, add them to your `.gitignore` to exclude them from deployment artifacts. +### Local modules and the ignore filter + +Local (non-pip) modules that your endpoints import are bundled automatically, provided they pass the ignore filter: your `.gitignore` plus the built-in patterns listed above. During the build, Flash resolves the transitive local-import closure of every shipped Python file and checks it against the files the ignore filter already selected. + +If shipped code imports a local module that an ignore rule excludes (for example, a `test_*.py` sibling or a file under `tests/`), the build fails with a `LocalModuleResolutionError` instead of silently overriding your ignore rules or shipping a broken artifact. Flash names each excluded file and the file that imports it: + +``` +Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude: + utils/helpers.py (imported by endpoint.py) + +Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code. +``` + +If an `@Endpoint` file has a local import Flash can't resolve at all (a broken relative import, or a file outside your project root), the build also fails with a clear error. A file that fails resolution but doesn't define an endpoint is skipped with a warning and the build continues. For details on how local imports are resolved and bundled, see [Import local modules](/flash/create-endpoints#import-local-modules). + ## Build artifacts After running `flash build`: diff --git a/flash/create-endpoints.mdx b/flash/create-endpoints.mdx index ee89bf094..b2ce31eae 100644 --- a/flash/create-endpoints.mdx +++ b/flash/create-endpoints.mdx @@ -206,7 +206,7 @@ dependencies=["transformers==4.36.0", "torch>=2.0.0", "numpy<2.0"] ### Import packages inside the function body -You must import packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. +You must import pip/installed packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. This rule applies to installed packages only; local project modules can be imported at the top of the file because Flash ships their source (see [Import local modules](#import-local-modules)). **Correct:** imports inside the function. ```python @@ -242,6 +242,24 @@ async def process_video(video_data): return {"processed": True} ``` +## Import local modules + +Your endpoint can import local (non-pip) Python modules that live alongside it in your project, such as a sibling `utils.py` file or a `helpers/` package. Flash detects these imports, follows them transitively, and ships the module source to the worker for you, so an import like `import utils` or `from helpers import load` works remotely with no extra configuration. + +Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning, and you're responsible for making those modules available on the worker. + +Flash bundles only local project files. Standard library modules are already present in the worker image, and pip packages must still be declared through the `dependencies` parameter. This applies transitively: if a bundled local module imports a pip package at its top level, that package must still be declared in the `dependencies` of any endpoint that uses the module. + +On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details. + +### Live execution size limit + +When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. See [Local module payload too large](/flash/troubleshooting#local-module-payload-too-large) for the corresponding error. + +### Local modules in a parent directory + +On the live execution path, Flash treats a module imported by absolute name from a parent directory as external and doesn't ship it, which causes a `ModuleNotFoundError` on the worker. For example, this happens with `import shared` when `shared.py` sits above your endpoint file. To avoid it, place your endpoint at or above its local dependencies, or use `flash deploy`, which resolves imports against the whole project directory. + ## Parallel execution Endpoint functions are async. Use Python's `asyncio` to run multiple operations concurrently: diff --git a/flash/troubleshooting.mdx b/flash/troubleshooting.mdx index d9bccdfb9..cb6da5a08 100644 --- a/flash/troubleshooting.mdx +++ b/flash/troubleshooting.mdx @@ -244,6 +244,36 @@ You have three options: Python 3.12 is recommended for best performance with no cold-start overhead. Python 3.10, 3.11, and 3.13 incur additional cold-start overhead on GPU workers because an alternative Python interpreter must be installed. +### Local module could not be resolved + +**Error:** +``` +endpoint.py: relative import (level=1, module='helpers') could not be resolved to a local file under /path/to/project +``` + +Or, for an ignore-excluded import: + +``` +Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude: + utils/helpers.py (imported by endpoint.py) + +Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code. +``` + +**Cause:** This `LocalModuleResolutionError` has two variants: an `@Endpoint` file imports a local module that Flash can't resolve, or shipped code imports a local module that an ignore pattern excludes. + +**Solution:** + +For an unresolvable import: + +1. Fix the import so it points to a file that exists under your project root. +2. Move the imported module under the project directory if it lives outside it. +3. Confirm that package imports have an `__init__.py` file. + +For an ignore-excluded import, remove the matching ignore pattern, or stop importing that module from your shipped code. + +For how Flash resolves and bundles local imports, see [Import local modules](/flash/create-endpoints#import-local-modules). + ## Deployment errors ### Tarball too large @@ -464,6 +494,27 @@ Payload size X MB exceeds limit of 10.0 MB 3. **Split large requests**: Break large datasets into smaller chunks and process them in multiple requests. +### Local module payload too large + +**Error:** +``` +Inline module payload is X bytes, over the 8388608-byte live-serverless cap. +Use `flash deploy` for endpoints with large local dependencies. +``` + +**Cause:** On the live execution path (calling an `@Endpoint` function directly or during `flash dev`), Flash ships your local module source inline with the request, and the combined source exceeds the 8 MiB cap. + +**Solution:** Deploy the app with `flash deploy`, which bundles local modules into the build artifact instead of the request payload: + +```bash +flash deploy + +# If using uv: +uv run flash deploy +``` + +See [Live execution size limit](/flash/create-endpoints#live-execution-size-limit) for details on how local module source is shipped. + ### Deserialization timeout **Error:** diff --git a/release-notes.mdx b/release-notes.mdx index c1213e65b..8818a6347 100644 --- a/release-notes.mdx +++ b/release-notes.mdx @@ -13,6 +13,10 @@ rss: true +**July 28, 2026** + +

New Release [Automatic local module bundling for Flash endpoints](/flash/create-endpoints#import-local-modules)

Flash now automatically bundles the local Python modules your endpoints import (sibling files and packages that aren't installed via pip) and ships their source to the worker. Imports like `import utils` or `from helpers import x` now work remotely with no extra configuration. + **July 23, 2026**

New Release [Runpod API v2 (BETA)](/api-reference-v2/overview)

A new REST API is available in public beta. See the [API v2 reference](/api-reference-v2/overview) to get started. The [GraphQL API](/api-reference/overview) and REST API v1 continue to work for now, but will be deprecated in a future release, so new integrations should build on API v2.