-
Notifications
You must be signed in to change notification settings - Fork 45
docs: Document Flash local (non-pip) module bundling for endpoints #775
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
c54334a
8cef7f5
a585273
e8b6774
f62240b
aa375ee
f4d34ce
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -206,7 +206,7 @@ dependencies=["transformers==4.36.0", "torch>=2.0.0", "numpy<2.0"] | |
|
|
||
| ### Import packages inside the function body | ||
|
|
||
| You must import packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. | ||
| You must import pip/installed packages **inside the decorated function body**, not at the top of your file. This ensures imports happen on the remote worker. This rule applies to installed packages only; local project modules can be imported at the top of the file because Flash ships their source (see [Import local modules](#import-local-modules)). | ||
|
|
||
| **Correct:** imports inside the function. | ||
| ```python | ||
|
|
@@ -242,6 +242,24 @@ async def process_video(video_data): | |
| return {"processed": True} | ||
| ``` | ||
|
|
||
| ## Import local modules | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Documents the local-module bundling feature added in PR #352. Source: runpod/flash#352 |
||
|
|
||
| Your endpoint can import local (non-pip) Python modules that live alongside it in your project, such as a sibling `utils.py` file or a `helpers/` package. Flash detects these imports, follows them transitively, and ships the module source to the worker for you, so an import like `import utils` or `from helpers import load` works remotely with no extra configuration. | ||
|
|
||
| Flash resolves local imports whether they appear at the top of the file or inside the function body, and it supports absolute imports (`import utils`), relative imports (`from . import helpers`), and dynamic imports with a literal name (`importlib.import_module("plugin")`). It also pulls in the `__init__.py` files for any packages you import. Flash can't resolve dynamic imports whose module name is computed at runtime, so it emits a warning, and you're responsible for making those modules available on the worker. | ||
|
|
||
| Flash bundles only local project files. Standard library modules are already present in the worker image, and pip packages must still be declared through the `dependencies` parameter. This applies transitively: if a bundled local module imports a pip package at its top level, that package must still be declared in the `dependencies` of any endpoint that uses the module. | ||
|
|
||
| On `flash build` and `flash deploy`, local modules are bundled when they pass the ignore filter, and importing a local module that an ignore rule excludes (or one Flash can't resolve) fails the build. See [Local modules and the ignore filter](/flash/cli/build#local-modules-and-the-ignore-filter) for details. | ||
|
|
||
| ### Live execution size limit | ||
|
|
||
| When you run an endpoint live (calling an `@Endpoint` function directly, or during `flash dev`), Flash ships the resolved module source inline with the request. The combined source is capped at 8 MiB. If your local dependencies exceed this limit, deploy the app with `flash deploy` instead, which bundles local modules into the build artifact rather than the request payload. See [Local module payload too large](/flash/troubleshooting#local-module-payload-too-large) for the corresponding error. | ||
|
|
||
| ### Local modules in a parent directory | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Documents the parent-directory constraint. The docstring note in Source: runpod/flash#352 |
||
|
|
||
| On the live execution path, Flash treats a module imported by absolute name from a parent directory as external and doesn't ship it, which causes a `ModuleNotFoundError` on the worker. For example, this happens with `import shared` when `shared.py` sits above your endpoint file. To avoid it, place your endpoint at or above its local dependencies, or use `flash deploy`, which resolves imports against the whole project directory. | ||
|
|
||
| ## Parallel execution | ||
|
|
||
| Endpoint functions are async. Use Python's `asyncio` to run multiple operations concurrently: | ||
|
|
||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (Line 488) Documents the Source: runpod/flash#352 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Documents the build-path validation behavior (merged PR #352, not force-include).
validate_local_module_importsincli/commands/build.pyis called fromrun_buildAFTERfiles = get_file_tree(project_dir, spec)(the ignore-filtered shipped set) is computed; it never expands that set. For every shipped.pyfile it walks the local-import closure viastubs/local_modules.py::resolve_local_modulesand checks each resolved file against the already-ignore-filtered files. If a shipped file imports a local module the ignore rules excluded, the build is refused (LocalModuleResolutionError->typer.Exit(1)) with an actionable message naming the excluded file and its importer — it does NOT force-include the file. Strictness for unresolvable imports is scoped to endpoint files viabuild_utils/scanner.py::defines_endpoint(recognizing@remote/@Endpoint): an endpoint file with an unresolvable local import fails the build loudly, while a non-endpoint file that fails resolution is skipped with a warning and the build continues.Source: runpod/flash#352