Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
77dc250
Update docs to use NVIDIA Sphinx theme (#712)
bdice Jul 18, 2026
22d1e04
feat: add dagster-slurm + RAPIDS topic modeling on HPC/Slurm workflow…
HPicatto Jul 18, 2026
967968a
docs: address tutorial review feedback on the dagster-slurm example
geoHeil Jul 18, 2026
2c57741
Merge pull request #2 from geoHeil/fix/tutorial-review-feedback
HPicatto Jul 19, 2026
f0d2983
docs: address review feedback on the topic modeling example
HPicatto Jul 18, 2026
f13a0b8
docs: add complete-run screenshots to the topic modeling example
HPicatto Jul 18, 2026
6a14dc2
Default the deployment docs to CUDA 13 (#719)
jayavenkatesh19 Jul 28, 2026
6640750
Fixes after testing on Databricks (#721)
jayavenkatesh19 Jul 29, 2026
8698903
Remove stale pip/cupy warning from Vertex AI page (#725)
jayavenkatesh19 Jul 30, 2026
ce6931c
X-ORG-722: Publish API docs to docs.nvidia.com (#724)
josephine-wolf-oberholtzer Jul 30, 2026
a8bbfc1
Update compute engine - add workarounds (#727)
ncclementi Jul 30, 2026
6b86c22
X-ORG-722: Ensure gha-tools are installed (#728)
josephine-wolf-oberholtzer Jul 31, 2026
276d2d6
Update readme, better layout description (#732)
ncclementi Aug 4, 2026
11658b6
added new Brev launchable instructions (#733)
jayavenkatesh19 Aug 4, 2026
f436858
Add kaggle page in platforms (#734)
ncclementi Aug 5, 2026
0c7479f
add Databricks Serverless (#736)
jayavenkatesh19 Aug 5, 2026
567e42e
replace scikit
HPicatto Aug 6, 2026
fd8108d
rewording
HPicatto Aug 6, 2026
e2aa997
make plot nicer
HPicatto Aug 6, 2026
fa070cf
reference example on HPC base
HPicatto Aug 6, 2026
d4976ac
Merge branch 'main' into feat/dagster-slurm-topic-modeling-example
HPicatto Aug 6, 2026
765609b
test if rendering works
HPicatto Aug 6, 2026
1180f8c
Merge branch 'feat/dagster-slurm-topic-modeling-example' of github-hp…
HPicatto Aug 6, 2026
243571e
test
HPicatto Aug 6, 2026
14b791f
check render on web
HPicatto Aug 6, 2026
1621cc0
add metaxy
HPicatto Aug 6, 2026
927612a
remove issue
HPicatto Aug 6, 2026
44bf677
Merge branch 'main' into feat/dagster-slurm-topic-modeling-example
HPicatto Aug 10, 2026
383a8ec
Merge branch 'main' into feat/dagster-slurm-topic-modeling-example
jayavenkatesh19 Aug 14, 2026
4d2dbbe
Merge branch 'main' into feat/dagster-slurm-topic-modeling-example
jayavenkatesh19 Aug 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions source/examples/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,5 @@ rapids-morpheus-pipeline/notebook
fraud-detection-mlops-pipeline/notebook
lulc-classification-gpu/notebook
cuml-ray-hpo/notebook
rapids-topic-modeling-slurm/notebook
```
564 changes: 564 additions & 0 deletions source/examples/rapids-topic-modeling-slurm/notebook.ipynb

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions source/hpc.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,12 @@ python -m cudf.pandas my_script.py
sbatch rapids_job.sh
```

## Orchestrating Pipelines

The workflows above submit one job at a time by hand. For multi-stage pipelines, for example CPU preprocessing followed by GPU training, [dagster-slurm](https://github.com/ascii-supply-networks/dagster-slurm) runs [Dagster](https://dagster.io/) assets as `sbatch` jobs over SSH: it packs your Python environment with [pixi-pack](https://github.com/Quantco/pixi-pack), ships it to the cluster, submits each stage with its own resources (`gpus_per_node`, memory, wall time), and streams logs and results back into the Dagster UI.
It needs nothing on the cluster beyond SSH and `sbatch`, so it works without containers or admin access.
See the [GPU topic modeling on HPC](/examples/rapids-topic-modeling-slurm/notebook) example for a complete RAPIDS pipeline with cuML UMAP and HDBSCAN stages.

```{relatedexamples}

```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading