Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/docs/extraction/audio-video.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ This documentation describes two ways to run [NeMo Retriever Library](overview.m
Supported file types for speech extraction today:

- `mp3`, `wav`
- `mp4`, `mov`, `mkv`, `avi` — common video containers; the audio track is transcribed (same extensions as in [What is NeMo Retriever Library?](overview.md))
- `mp4`, `mov`, `mkv`, `avi` — common video containers; the audio track is transcribed (same extensions as in [NeMo Retriever Library Overview](overview.md))

[NeMo Retriever Library](overview.md) supports extracting speech from audio for Retrieval Augmented Generation (RAG). Similar to how the multimodal document pipeline uses detection and OCR microservices, NeMo Retriever Library uses the [parakeet-1-1b-ctc-en-us ASR NIM](https://docs.nvidia.com/nim/speech/latest/asr/deploy-asr-models/parakeet-ctc-en-us.html) to transcribe speech to text, then embeddings through the NeMo Retriever embedding path.

Expand Down Expand Up @@ -131,7 +131,7 @@ For video assets, NeMo Retriever Library can combine audio or speech processing

For OCR-oriented extract methods on scanned or image-heavy content, refer to [OCR and scanned documents](multimodal-extraction.md#ocr-and-scanned-documents), [text and layout extraction](multimodal-extraction.md#text-and-layout-extraction), and [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse) for advanced visual parsing.

Container formats and early-access video types are listed under [supported file types and formats](multimodal-extraction.md#supported-file-types-and-formats) (refer to [What is NeMo Retriever Library?](overview.md) for the full list).
Container formats and early-access video types are listed under [supported file types and formats](multimodal-extraction.md#supported-file-types-and-formats) (refer to [NeMo Retriever Library Overview](overview.md) for the full list).

For end-to-end RAG stacks that include multimodal ingestion, refer to the [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) and related solution pages on [NVIDIA Build](https://build.nvidia.com/).

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/extraction/getting-started-about.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ This section walks you from **access and prerequisites** through **first deploym
Typical order:

1. [Get your API key](api-keys.md) (NGC / API access as required by your workflow).
2. Confirm the [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) for your OS, GPU, and software stack.
2. Confirm the [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) for your OS, GPU, and software stack. Local GPU inference requires Linux; remote NIM workflows can use the base package on Windows x64 and macOS as well.
3. Choose a path in [Deployment options](deployment-options.md) — local library, hosted NIMs, the Helm chart for Kubernetes, or a standalone Docker service.
4. Explore [Jupyter Notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) for end-to-end examples.

The NeMo Retriever Library and its Helm chart are not supported under NVIDIA AI Enterprise (NVAIE). For more information, refer to [NVIDIA AI Enterprise (NVAIE) support](overview.md#nvidia-ai-enterprise-nvaie-support).

If you are new to the product, read [What is NeMo Retriever Library?](overview.md) and [Concepts](concepts.md) under **Introduction** first.
If you are new to the product, read [NeMo Retriever Library Overview](overview.md) and [Concepts](concepts.md) under **Introduction** first.
8 changes: 4 additions & 4 deletions docs/docs/extraction/multimodal-extraction.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ NeMo Retriever Library classifies and extracts text, tables, charts, infographic

## Supported file types and formats { #supported-file-types-and-formats }

NeMo Retriever Library accepts multiple document and media types. A current list (including PDF, Office formats, HTML, images, audio, and video, some early access) appears in [What is NeMo Retriever Library?](overview.md) under **NeMo Retriever Library supports the following file types**.
NeMo Retriever Library accepts multiple document and media types. A current list (including PDF, Office formats, HTML, images, audio, and video, some early access) appears in [NeMo Retriever Library Overview](overview.md) under **NeMo Retriever Library supports the following file types**.

**Related**

Expand All @@ -31,7 +31,7 @@ For PDFs, NeMo Retriever Library typically uses **pdfium**-based extraction with

**Related**

- [What is NeMo Retriever Library?](overview.md)
- [NeMo Retriever Library Overview](overview.md)
- [OCR and scanned documents](#ocr-and-scanned-documents)
- [Chunking](concepts.md#chunking)

Expand All @@ -41,7 +41,7 @@ NeMo Retriever Library detects tables as structured page elements, processes the

**Related**

- [What is NeMo Retriever Library?](overview.md) for artifact classification
- [NeMo Retriever Library Overview](overview.md) for artifact classification
- [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse) for advanced visual parsing
- [Metadata reference](content-metadata.md)

Expand All @@ -66,7 +66,7 @@ For natural-language infographic descriptions, optionally enable [image captioni

**Related**

- [What is NeMo Retriever Library?](overview.md)
- [NeMo Retriever Library Overview](overview.md)
- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
- [Multimodal embeddings (VLM)](embedding.md) when you treat graphics as images for embedding

Expand Down
15 changes: 7 additions & 8 deletions docs/docs/extraction/overview.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# What is NeMo Retriever Library?
# NeMo Retriever Library Overview { #what-is-nemo-retriever-library }

NVIDIA NeMo Retriever Library (NRL) is a high retrieval accuracy, performant, and scalable framework for content and metadata extraction from various media types (PDFs, HTML, Word docs, Powerpoint, audio, video, and image files). It supports both NVIDIA NIM microservices and a range of models to find, contextualize, and extract text, tables, charts, infographics, and transcripts for use in downstream generative and retrieval-augmented applications.
NVIDIA NeMo Retriever Library (NRL) extracts text, tables, charts, infographics, and transcripts from PDFs, HTML, Office documents, audio, video, and images. Run it as a Python library or Kubernetes deployment, and route inference through NVIDIA NIM microservices or local Nemotron models for downstream RAG and generative applications.

NeMo Retriever Library enables parallelization of splitting documents into pages where sub-page content is classified (such as text paragraphs, tables, charts, and infographics), extracted, and further contextualized through optical character recognition (OCR) into a standard schema. From there, NeMo Retriever Library manages computation of embeddings for the extracted content,
and can store vectors in [LanceDB](https://lancedb.com/) for the recommended embedded path when you pass `vdb_op="lancedb"` to upload (refer to [Vector databases](vdbs.md)).
NeMo Retriever Library splits documents into pages, classifies sub-page content (text, tables, charts, and infographics), extracts it, and applies optical character recognition (OCR) where needed into a standard schema. It can compute embeddings for extracted content and store vectors in [LanceDB](https://lancedb.com/) when you pass `vdb_op="lancedb"` to upload (refer to [Vector databases](vdbs.md)).

## NVIDIA AI Enterprise (NVAIE) support { #nvidia-ai-enterprise-nvaie-support }

Expand All @@ -21,13 +20,13 @@ The following diagram shows the retriever pipeline.

NeMo Retriever Library does the following:

- Accept directories of input files and a series of configurable ingestion tasks to perform on that input
- Allow the extracted content be retrieved from a VDB containing discrete metadata element
- Accept directories of input files and configurable ingestion tasks
- Store extracted content in a vector database (VDB) with discrete metadata elements
- Support multiple extraction methods per document type—for example, PDFs can use **pdfium** or [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse) as an alternate method (`extract_method="nemotron_parse"`)
- Support various types of pre- and post- processing operations, including text splitting and chunking, transform and filtering, embedding generation, and image offloading to storage.
- Apply pre- and post-processing: text splitting and chunking, transforms and filtering, embedding generation, and image offloading to storage

!!! note
To use `extract_method="nemotron_parse"` with PDFs, install the Nemotron Parse client dependencies with the `nemotron-parse` extra, for example `pip install "nemo-retriever[nemotron-parse]"`.
To use `extract_method="nemotron_parse"` with PDFs, install the Nemotron Parse client dependencies with the `nemotron-parse` extra, for example `uv pip install "nemo-retriever[nemotron-parse]"`. You can use the equivalent `pip install` command if you do not use UV.

NeMo Retriever Library supports the following file types:

Expand Down
34 changes: 18 additions & 16 deletions docs/docs/extraction/prerequisites-support-matrix.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,32 @@
# Pre-Requisites & Support Matrix

Before you begin using [NeMo Retriever Library](overview.md), confirm your software stack, deployment hardware, and—if you use them—advanced features (audio and video, Nemotron Parse, VLM image captioning, reranking) against the guidance in this page.
Before you begin using [NeMo Retriever Library](overview.md), confirm your software stack, deployment hardware, and—if you use them—advanced features (audio and video, Nemotron Parse, VLM image captioning, reranking) against the guidance on this page.

!!! note "NVIDIA AI Enterprise (NVAIE) support"
**Platform summary:** Supported **local GPU inference** requires **Linux** and CUDA 13. For **remote NIM inference**, the base Python package also installs on **Windows x64** and **macOS x64/ARM64**; local GPU inference is not supported on those platforms.

The NeMo Retriever Library, including its container image and Helm chart artifacts, is not supported under NVIDIA AI Enterprise (NVAIE), even though some NIM microservices and models it uses may be individually covered by NVAIE. For more information, refer to [NVIDIA AI Enterprise (NVAIE) support](overview.md#nvidia-ai-enterprise-nvaie-support).
> **Note — NVIDIA AI Enterprise (NVAIE) support**
>
> The NeMo Retriever Library, including its container image and Helm chart artifacts, is not supported under NVIDIA AI Enterprise (NVAIE), even though some NIM microservices and models it uses may be individually covered by NVAIE. For more information, refer to [NVIDIA AI Enterprise (NVAIE) support](overview.md#nvidia-ai-enterprise-nvaie-support).

## Software Requirements { #software-requirements }

- Linux operating systems (Ubuntu 22.04 or later recommended)
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) (NVIDIA Driver >= `580`, CUDA >= `13.0`)
- Linux operating systems (Ubuntu 22.04 or later recommended) for supported local GPU inference. For remote NIM inference, the base package can also be installed on Windows x64 and macOS x64/ARM64; local GPU inference is not supported on those platforms.
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) (local GPU inference only; NVIDIA Driver >= `580`, CUDA >= `13.0`)
- [Python](https://www.python.org/downloads/) `3.12` — required to install and run the NeMo Retriever Library Python API, CLI, and related packages from PyPI (for example `pip` or `uv`). Older Python versions will fail dependency resolution without a clear error.
- [UV Python package and environment manager](https://docs.astral.sh/uv/getting-started/installation/) (optional; recommended for creating isolated environments)
- For audio and video, `ffmpeg` and `ffprobe` must be on `PATH` (for example
`sudo apt-get install -y --no-install-recommends ffmpeg` on Debian/Ubuntu).
`ffmpeg-python` and `nemo-retriever[multimedia]` do not install these binaries.
For container and Kubernetes guidance, refer to [Audio and video](audio-video.md).
- For PDF extraction with `extract_method="nemotron_parse"`, install the Nemotron Parse
client dependencies with `pip install "nemo-retriever[nemotron-parse]"` (pulls
client dependencies with `uv pip install "nemo-retriever[nemotron-parse]"` (pulls
`open-clip-torch`, which provides the `open_clip` module required by the Nemotron Parse
NIM client). The base `nemo-retriever` install and `[local]` extra do not include this
package.
package. You can use the equivalent `pip install` command if you do not use UV.

!!! note

When you use UV, create the environment with Python 3.12 — for example, `uv venv --python 3.12`. This matches the `requires-python` metadata in the library packages.
> **Note**
>
> When you use UV, create the environment with Python 3.12 — for example, `uv venv --python 3.12`. This matches the `requires-python` metadata in the library packages.

## Hardware Requirements { #hardware-requirements }

Expand All @@ -40,9 +42,9 @@ For per-feature GPU memory, disk, and co-residency rules, refer to [Model hardwa
- **CPU Cores**: At least 32 CPU cores
- **GPU**: NVIDIA GPU with at least 24 GB VRAM (for example, A100, H100, L40S, or equivalent)

!!! note

Using less powerful systems or lower resource limits is still viable, but performance will suffer.
> **Note**
>
> Using less powerful systems or lower resource limits is still viable, but performance will suffer.

### Resource Consumption Notes

Expand Down Expand Up @@ -73,9 +75,9 @@ Optional advanced features—audio and video transcription, Nemotron Parse, Omni

### Default NIMs { #default-helm-nims }

!!! important "NVAIE support applies to individual NIMs only"

A NIM or model listed in the default and optional NIM rows in the table below might be supported under NVIDIA AI Enterprise (NVAIE) as an individual product. That support does **not** cover its use through NeMo Retriever Library or extend to the library, its container image, its Helm chart, or the end-to-end extraction workflow.
> **Important — NVAIE support applies to individual NIMs only**
>
> A NIM or model listed in the default and optional NIM rows in the table below might be supported under NVIDIA AI Enterprise (NVAIE) as an individual product. That support does **not** cover its use through NeMo Retriever Library or extend to the library, its container image, its Helm chart, or the end-to-end extraction workflow.

The production Helm chart reconciles NIM microservices through `nimOperator.<key>.enabled`. Four core NIMs are **enabled by default** and auto-wired into the retriever service; optional NIMs reconcile only when you opt in. For chart keys, image overrides, and enablement, refer to the [NeMo Retriever Helm chart README](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#nim-operator-sub-stack) and [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md#recommended-minimal-install-2605).

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/extraction/troubleshoot.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ For local GPU inference with Nemotron Parse, combine extras:
pip install "nemo-retriever[local,nemotron-parse]"
```

Also refer to [What is NeMo Retriever Library?](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements).
Also refer to [NeMo Retriever Library Overview](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements).

## Extract method nemotron-parse doesn't support image files

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/extraction/vdbs.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Use this documentation to learn how [NeMo Retriever Library](overview.md) stores
## On this page { #on-this-page }

- [Overview](#overview)
- [Why LanceDB?](#why-lancedb)
- [LanceDB Overview](#why-lancedb)
- [Upload to LanceDB](#upload-to-lancedb)
- [Semantic retrieval](#semantic-retrieval)
- [Metadata and filtering](#metadata-and-filtering)
Expand Down Expand Up @@ -37,7 +37,7 @@ Currently, data upload is not supported through the [CLI](https://github.com/NVI



## Why LanceDB? { #why-lancedb }
## LanceDB Overview { #why-lancedb }

LanceDB is optimized for low-latency retrieval in this stack:

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# What is NVIDIA NeMo Retriever?
# NVIDIA NeMo Retriever Overview { #what-is-nvidia-nemo-retriever }

NVIDIA NeMo Retriever is a collection of microservices
for building and scaling multimodal data extraction, embedding, and reranking pipelines
Expand Down
2 changes: 1 addition & 1 deletion docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ extra_css:
# bar when there is only one top-level tab.
nav:
- "1. Introduction":
- "What is NeMo Retriever?": extraction/overview.md
- "NeMo Retriever Library Overview": extraction/overview.md
- Key concepts: extraction/concepts.md
- Release notes: extraction/releasenotes.md
- "2. Get started":
Expand Down
Loading