-
Notifications
You must be signed in to change notification settings - Fork 1
Chore/dockerize app #21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Ayan-josh-05
wants to merge
9
commits into
joshsoftware:lending_poc/main
Choose a base branch
from
Ayan-josh-05:chore/dockerize-app
base: lending_poc/main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
e36daad
Setup locally
Ayan-josh-05 f12b726
Dockerized the application
Ayan-josh-05 d293b13
fix: keep Ollama model resident to prevent /map timeouts
Ayan-josh-05 83c04ad
Undo some changes
Ayan-josh-05 baa515e
feat: make GPU acceleration opt-in for Surya OCR and Ollama via compo…
Ayan-josh-05 007e5a8
Added .env.example file
Ayan-josh-05 e948b5b
fix: pin torchvision to CPU wheel in OCR Dockerfile
Ayan-josh-05 bee85e0
docs: add top-level README for running the full stack with Docker
Ayan-josh-05 a4e03c0
docs: document OLLAMA_MODEL/OLLAMA_HOST in .env.example
Ayan-josh-05 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| *.sh text eol=lf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| .git | ||
| .venv | ||
| **/.venv | ||
| **/.venv-windows | ||
| **/surya-env | ||
| **/node_modules | ||
| **/__pycache__ | ||
| document_processing | ||
| field_mapping_poc | ||
| gateway | ||
| frontend | ||
| scripts | ||
| docs | ||
| *.log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| POSTGRES_USER=postgres | ||
| POSTGRES_PASSWORD=postgres | ||
| POSTGRES_DB=lending_poc | ||
| POSTGRES_HOST_PORT=55439 | ||
| DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:55439/lending_poc | ||
| ENCRYPTION_KEY=changeme-generate-a-base64-fernet-key | ||
| DEBUG=true | ||
|
|
||
| # Selects the surya-inference/ocr variant docker-compose.yml runs: "cpu" | ||
| # (default, always works) or "gpu" (requires an NVIDIA GPU + NVIDIA | ||
| # Container Toolkit / WSL GPU passthrough on the host). | ||
| COMPOSE_PROFILES=cpu | ||
|
|
||
| # Model used by the translation and field_mapping services via Ollama. | ||
| # Must be pulled into the ollama container first: | ||
| # docker compose exec ollama ollama pull <model> | ||
| OLLAMA_MODEL=gemma4:e4b-it-qat | ||
| # OLLAMA_HOST=http://ollama:11434 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| # Lending POC | ||
|
|
||
| Lending POC — FastAPI backend + PostgreSQL (pgvector), plus a document | ||
| processing pipeline (OCR, translation, field mapping) fronted by a gateway, | ||
| and a React frontend. This guide covers running the **entire stack in | ||
| Docker**. | ||
|
|
||
| For running the `app` service natively against a containerized DB only | ||
| (e.g. for backend development with hot-reload outside Docker), see | ||
| [Database_setup.md](Database_setup.md). | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2 | ||
| (`docker compose version`) | ||
| - Git | ||
| - **Optional, for GPU acceleration**: an NVIDIA GPU, the | ||
| [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html), | ||
| and (on Windows) WSL2 with GPU passthrough enabled | ||
|
|
||
| ## 1. Clone and configure environment | ||
|
|
||
| ```bash | ||
| git clone <repo-url> | ||
| cd lending-poc | ||
| cp .env.example .env | ||
| ``` | ||
|
|
||
| Generate a real `ENCRYPTION_KEY` — the app uses it to encrypt sensitive | ||
| database fields, and the placeholder value in `.env.example` is not a valid | ||
| key: | ||
|
|
||
| ```bash | ||
| python3 -c "import base64, os; print(base64.b64encode(os.urandom(32)).decode())" | ||
| ``` | ||
|
|
||
| Paste the result into `ENCRYPTION_KEY=` in `.env`. | ||
|
|
||
| Leave `COMPOSE_PROFILES=cpu` as-is unless you have a working GPU setup — | ||
| see [Using a GPU](#using-a-gpu) below. | ||
|
|
||
| ## 2. Start the stack | ||
|
|
||
| ```bash | ||
| docker compose up --build -d | ||
| ``` | ||
|
|
||
| This builds and starts every service: `db`, `app`, `ollama`, `field_mapping`, | ||
| `translation`, `surya-inference` + `ocr`, `gateway`, and `frontend`. | ||
|
|
||
| The first run takes a while — Ollama and Surya both download models on | ||
| first use. Watch progress with: | ||
|
|
||
| ```bash | ||
| docker compose logs -f | ||
| ``` | ||
|
|
||
| ## 3. Run database migrations | ||
|
|
||
| The `app` container doesn't run migrations automatically on startup: | ||
|
|
||
| ```bash | ||
| docker compose exec app alembic -c db/alembic.ini upgrade head | ||
| ``` | ||
|
|
||
| ## 4. Pull the Ollama model | ||
|
|
||
| Needed by the `translation` and `field_mapping` services: | ||
|
|
||
| ```bash | ||
| docker compose exec ollama ollama pull gemma4:e4b-it-qat | ||
| ``` | ||
|
|
||
| (Substitute whatever `OLLAMA_MODEL` is set to in `.env` if you changed it. | ||
| If you're running the GPU profile, use `ollama-gpu` instead of `ollama` in | ||
| the command above.) | ||
|
|
||
| ## 5. Verify it's running | ||
|
|
||
| | Service | URL | Notes | | ||
| |---|---|---| | ||
| | Frontend | http://localhost:5173 | Main UI | | ||
| | Gateway | http://localhost:8080 | Fronts OCR / translation / field-mapping | | ||
| | App (backend API) | http://localhost:8000 | Docs at `/docs`; health at `/health` | | ||
| | Postgres | localhost:55439 | pgvector-enabled | | ||
| | OCR | http://localhost:8010 | Not normally called directly | | ||
| | Translation | http://localhost:8001 | Not normally called directly | | ||
| | Field mapping | http://localhost:8002 | Not normally called directly | | ||
| | Surya inference | http://localhost:8500 | OCR's inference backend | | ||
|
|
||
| There are effectively two subsystems sharing this compose file: the | ||
| `app` + `db` lending backend, and a separate OCR/translation/field-mapping | ||
| pipeline fronted by `gateway`. The frontend talks to the gateway for | ||
| document processing and to the app for everything else. | ||
|
|
||
| ## Using a GPU | ||
|
|
||
| `surya-inference`/`ocr` and `ollama` each come in a CPU and a GPU variant, | ||
| selected by `COMPOSE_PROFILES` in `.env`: | ||
|
|
||
| - `COMPOSE_PROFILES=cpu` (default) — always works, no GPU required. | ||
| - `COMPOSE_PROFILES=gpu` — requires an NVIDIA GPU on the host plus the | ||
| NVIDIA Container Toolkit (and, on Windows, WSL2 GPU passthrough). | ||
|
|
||
| To switch: | ||
|
|
||
| ```bash | ||
| # in .env | ||
| COMPOSE_PROFILES=gpu | ||
| ``` | ||
|
|
||
| ```bash | ||
| docker compose up --build -d | ||
| ``` | ||
|
|
||
| Both GPU containers detect GPU access at startup and fall back to CPU | ||
| automatically if it isn't actually usable — but `docker compose up` will | ||
| fail to create the containers at all if the toolkit isn't installed, | ||
| since the GPU device reservation can't be satisfied. | ||
|
|
||
| Ollama's own image auto-detects CUDA at runtime with no separate build, so | ||
| switching the profile is enough for it; `surya-inference`/`ocr` are built | ||
| from CUDA base images specifically for the `gpu` profile (see | ||
| [docker-compose.yml](docker-compose.yml) and | ||
| [document_processing/ocr/README.md](document_processing/ocr/README.md) | ||
| for details). | ||
|
|
||
| ## Stopping and cleanup | ||
|
|
||
| ```bash | ||
| docker compose down | ||
| ``` | ||
|
|
||
| Add `-v` to also delete the named volumes (`pgdata`, `ollama_models`, | ||
| `surya_models`) — this wipes the database and downloaded models, so only | ||
| do this if you want a clean slate: | ||
|
|
||
| ```bash | ||
| docker compose down -v | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extra="ignore" lets pydantic-settings skip env vars not declared on Settings (e.g. POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST_PORT, COMPOSE_PROFILES — used by docker-compose, not read by the app). Without it, pydantic's default extra="forbid" would raise ValidationError on startup since those keys aren't fields on the model.
Tradeoff: a typo in a key we do care about (e.g. DATABASE_URL) would also be silently ignored rather than erroring.