Skip to content
15 changes: 15 additions & 0 deletions flash/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,21 @@ Flash also respects your `.gitignore` file and excludes any files matching those
If you use other environment file variants like `.env.dev` or `.env.staging`, add them to your `.gitignore` to exclude them from deployment artifacts.
</Tip>

### Local modules and the ignore filter

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documents the build-path validation behavior (merged PR #352, not force-include). validate_local_module_imports in cli/commands/build.py is called from run_build AFTER files = get_file_tree(project_dir, spec) (the ignore-filtered shipped set) is computed; it never expands that set. For every shipped .py file it walks the local-import closure via stubs/local_modules.py::resolve_local_modules and checks each resolved file against the already-ignore-filtered files. If a shipped file imports a local module the ignore rules excluded, the build is refused (LocalModuleResolutionError -> typer.Exit(1)) with an actionable message naming the excluded file and its importer — it does NOT force-include the file. Strictness for unresolvable imports is scoped to endpoint files via build_utils/scanner.py::defines_endpoint (recognizing @remote/@Endpoint): an endpoint file with an unresolvable local import fails the build loudly, while a non-endpoint file that fails resolution is skipped with a warning and the build continues.

Source: runpod/flash#352


Local (non-pip) modules that your endpoints import are bundled automatically, provided they pass the ignore filter: your `.gitignore` plus the built-in patterns listed above. During the build, Flash resolves the transitive local-import closure of every shipped Python file and checks it against the files the ignore filter already selected.

If shipped code imports a local module that an ignore rule excludes (for example, a `test_*.py` sibling or a file under `tests/`), the build fails with a `LocalModuleResolutionError` instead of silently overriding your ignore rules or shipping a broken artifact. Flash names each excluded file and the file that imports it:

```
Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude:
utils/helpers.py (imported by endpoint.py)

Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code.
```

If an `@Endpoint` file has a local import Flash can't resolve at all (a broken relative import, or a file outside your project root), the build also fails with a clear error. A file that fails resolution but doesn't define an endpoint is skipped with a warning and the build continues. For details on how local imports are resolved and bundled, see [Import local modules](/flash/create-endpoints#import-local-modules).

## Build artifacts

After running `flash build`:
Expand Down
20 changes: 19 additions & 1 deletion flash/create-endpoints.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ dependencies=["transformers==4.36.0", "torch>=2.0.0", "numpy<2.0"]

### Import packages inside the function body

You must import packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker.
You must import pip/installed packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. This rule applies to installed packages only; local project modules can be imported at the top of the file because Flash ships their source (see [Import local modules](#import-local-modules)).

**Correct:** imports inside the function.
```python
Expand Down Expand Up @@ -242,6 +242,24 @@ async def process_video(video_data):
return {"processed": True}
```

## Import local modules

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documents the local-module bundling feature added in PR #352. stubs/local_modules.py::resolve_local_modules walks the transitive local-import closure (absolute, relative, and literal importlib.import_module imports, at module level and in function bodies), pulls in package __init__.py files, classifies stdlib/installed names as external, and warns on non-literal dynamic imports; stubs/live_serverless.py::build_modules_map ships the resolved source inline via the new FunctionRequest.modules field and enforces MAX_INLINE_MODULE_BYTES = 8 MiB.

Source: runpod/flash#352


Your endpoint can import local (non-pip) Python modules that live alongside it in your project, such as a sibling `utils.py` file or a `helpers/` package. Flash detects these imports, follows them transitively, and ships the module source to the worker for you, so an import like `import utils` or `from helpers import load` works remotely with no extra configuration.

Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning, and you're responsible for making those modules available on the worker.

Flash bundles only local project files. Standard library modules are already present in the worker image, and pip packages must still be declared through the `dependencies` parameter. This applies transitively: if a bundled local module imports a pip package at its top level, that package must still be declared in the `dependencies` of any endpoint that uses the module.

On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details.

### Live execution size limit

When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. See [Local module payload too large](/flash/troubleshooting#local-module-payload-too-large) for the corresponding error.

### Local modules in a parent directory

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documents the parent-directory constraint. The docstring note in stubs/local_modules.py::resolve_local_modules states that on the live path a module imported by absolute name from a parent directory is classified as external and silently omitted (raising ModuleNotFoundError on the worker); only relative imports fail loudly, and the workaround is to keep the endpoint at/above its local deps or use flash deploy.

Source: runpod/flash#352


On the live execution path, Flash treats a module imported by absolute name from a parent directory as external and doesn't ship it, which causes a `ModuleNotFoundError` on the worker. For example, this happens with `import shared` when `shared.py` sits above your endpoint file. To avoid it, place your endpoint at or above its local dependencies, or use `flash deploy`, which resolves imports against the whole project directory.

## Parallel execution

Endpoint functions are async. Use Python's `asyncio` to run multiple operations concurrently:
Expand Down
51 changes: 51 additions & 0 deletions flash/troubleshooting.mdx

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Line 488)

Documents the LocalModulePayloadTooLargeError raised in stubs/live_serverless.py::build_modules_map when the inline module payload exceeds MAX_INLINE_MODULE_BYTES (8 * 1024 * 1024 = 8388608 bytes = 8 MiB), defined in stubs/local_modules.py. The raised message is a single continuous string: "Inline module payload is {total} bytes, over the {MAX_INLINE_MODULE_BYTES}-byte live-serverless cap. Use flash deploy for endpoints with large local dependencies." — no embedded newline; the docs code block wraps it onto two lines for readability. The error points users to flash deploy, which is the documented solution. Anchor moved from line 484 (inside the unrelated pre-existing "Payload too large" section) to line 488, the heading of the new "Local module payload too large" section this citation actually supports.

Source: runpod/flash#352

Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,36 @@ You have three options:
Python 3.12 is recommended for best performance with no cold-start overhead. Python 3.10, 3.11, and 3.13 incur additional cold-start overhead on GPU workers because an alternative Python interpreter must be installed.
</Tip>

### Local module could not be resolved

**Error:**
```
endpoint.py: relative import (level=1, module='helpers') could not be resolved to a local file under /path/to/project
```

Or, for an ignore-excluded import:

```
Shipped code imports local modules that the build ignore rules (.gitignore or built-in defaults) exclude:
utils/helpers.py (imported by endpoint.py)

Shipping them would silently override a deliberate exclusion, and omitting them would break the worker with ModuleNotFoundError. Remove the matching ignore pattern or stop importing these modules from shipped code.
```

**Cause:** This `LocalModuleResolutionError` has two variants: an `@Endpoint` file imports a local module that Flash can't resolve, or shipped code imports a local module that an ignore pattern excludes.

**Solution:**

For an unresolvable import:

1. Fix the import so it points to a file that exists under your project root.
2. Move the imported module under the project directory if it lives outside it.
3. Confirm that package imports have an `__init__.py` file.

For an ignore-excluded import, remove the matching ignore pattern, or stop importing that module from your shipped code.

For how Flash resolves and bundles local imports, see [Import local modules](/flash/create-endpoints#import-local-modules).

## Deployment errors

### Tarball too large
Expand Down Expand Up @@ -464,6 +494,27 @@ Payload size X MB exceeds limit of 10.0 MB

3. **Split large requests**: Break large datasets into smaller chunks and process them in multiple requests.

### Local module payload too large

**Error:**
```
Inline module payload is X bytes, over the 8388608-byte live-serverless cap.
Use `flash deploy` for endpoints with large local dependencies.
```

**Cause:** On the live execution path (calling an `@Endpoint` function directly or during `flash dev`), Flash ships your local module source inline with the request, and the combined source exceeds the 8 MiB cap.

**Solution:** Deploy the app with `flash deploy`, which bundles local modules into the build artifact instead of the request payload:

```bash
flash deploy

# If using uv:
uv run flash deploy
```

See [Live execution size limit](/flash/create-endpoints#live-execution-size-limit) for details on how local module source is shipped.

### Deserialization timeout

**Error:**
Expand Down
4 changes: 4 additions & 0 deletions release-notes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ rss: true

<Accordion title="July 2026" defaultOpen>

**July 28, 2026**

<h4><Badge color="green">New Release</Badge> [Automatic local module bundling for Flash endpoints](/flash/create-endpoints#import-local-modules) </h4> Flash now automatically bundles the local Python modules your endpoints import (sibling files and packages that aren't installed via pip) and ships their source to the worker. Imports like `import utils` or `from helpers import x` now work remotely with no extra configuration.

**July 23, 2026**

<h4><Badge color="green">New Release</Badge> [Runpod API v2 (BETA)](/api-reference-v2/overview) </h4> A new REST API is available in public beta. See the [API v2 reference](/api-reference-v2/overview) to get started. The [GraphQL API](/api-reference/overview) and REST API v1 continue to work for now, but will be deprecated in a future release, so new integrations should build on API v2.
Expand Down
Loading