Feature: Per-preprocessor endpoint override (#973)

.gitignore

@@ -1,21 +1,7 @@
 .DS_Store
 __pycache__/
 *.swp
-config/maps.env
-config/azure-api.env
-config/chart.env
-config/apis-and-selection.env
-config/vision_api_key.json
-config/ollama.env
-config/healthcheck.env
-config/image-summary.env
-config/image_daily_cron.env
-config/imagelogs.env
-config/make_request.env
-config/pp_daily_test.env
-config/sendimagereq.env
-config/slack-webhook-preprocessors.env
-config/imageup.env
-config/gemini.env
 *.sh
 docker-compose.override.yml
+# Ignore all .env files in config/
+config/**/*.env

DEPLOYMENT.md

@@ -103,11 +103,11 @@ Docker Compose uses environment files to configure how services run. In IMAGE, y
     b) Component-specific envs: These configure runtime services (preprocessors, handlers, or standalone services) which are loaded by containers at runtime. They often hold API keys, model names, or URLs that you don't want to check into git. Filenames are specified in docker-compose.
     For example, ensure the following files exist in the config/ folder and are populated with appropriate credentials:
 
-    `apis-and-selection.env, azure-api.env, llm.env, maps.env`
+    `cloud-ocr.env, azure-api.env, llm.env, maps.env`
 
     If a service is not in use, simply create an empty file with the same name to avoid startup errors.
     Here is a command to create them all:
-    `touch config/{maps.env,express-common.env,llm.env,azure-api.env}`
+    `touch config/{maps.env,llm.env,azure-api.env}`
 
     TIP: once completed, these files include credentials that should not be committed in Git!
 
@@ -199,9 +199,11 @@ networks:
 ```
 
 # Connecting to, or running your own, visual LLM
-Since multiple IMAGE preprocessors us a visual LLM (currently qwen), you need to either specify a cloud endpoint, or run your own LLM locally on your server.
+Since multiple IMAGE preprocessors use a visual LLM (currently gemma), you need to either specify a cloud endpoint, or run your own LLM locally on your server.
+
 Our production reference server uses vLLM as the runtime for LLM models.
 Like Traefik, it runs with its own docker-compose in its own directory on our server (e.g., `/var/docker/ollama`).
+
 It’s not a preprocessor itself; instead, several preprocessors in IMAGE-server call out to whatever LLM endpoint you configure (Ollama/vLLM locally, or a remote API if you set one). For example, the following preprocessors connect to the LLM via env_file `/var/docker/image/config/llm.env`: content-categoriser, graphic-caption, text-followup, and multistage-diagram-segmentation.
 Key features:
 - Accepts multimodal graphic and text prompots
@@ -217,8 +219,30 @@ You must specifiy services such as:
 - vllm: runs `vllm/vllm-openai:latest`, reserves a GPU, serves an OpenAI-style API at port 8000.
 - open-webui: optional WebUI that connects to Ollama or vLLM and is exposed via Traefik with a hostname like `ollama.unicorn.cim.mcgill.ca`.
 
+## Using a different model per preprocessor
+By default, `config/llm.env` sets the LLM endpoint and model used by every preprocessor that calls out to a vision LLM. If you want a specific preprocessor to use a different endpoint or model than this default, you can add a preprocessor-specific override.
+
+In `docker-compose.yml`, preprocessors that support this list two `env_file` entries:
+```yaml
+env_file:
+  - path: ./config/llm.env
+    required: true
+  - path: ./config/<preprocessor-name>.env
+    required: false
+```
+The global `config/llm.env` is always loaded first; the preprocessor-specific file is loaded second **only if it exists**, and any variable it sets overrides the global value for that preprocessor alone. If the file doesn't exist, the preprocessor simply falls back to the global configuration. No extra setup is required unless you actually want an override.
+
+To set one up, create a `.env` file within the existing `config/` directory using the naming convention `<preprocessor-name>.env`. Set `LLM_API_KEY`, `LLM_URL`, and `LLM_MODEL`. Add this path to the `docker-compose.yml` file's `env_file` entry for that service (if it does not exist) and set the `required` flag to `False`.
+
+**If you override any one of these three, set all three**: they describe a single endpoint together, and partially overriding them will send incorrect credentials to the new endpoint and fail the request.
+
+Preprocessors currently supporting this override pattern (check 
+`docker-compose.yml` for the current list, as this may grow): 
+`content-categoriser`, `graphic-caption`, `text-followup`, `object-detection-llm`, `multistage-diagram-segmentation`.
+
+The same layered `env_file` override functionality can be used for other service configurations as well.
 
-GPU Notes:
+## GPU Notes:
 Some containers that require GPU (and don't use a cloud endpoint or the LLM) include espnet-tts, text-followup, semantic-segmentation,object-detection, action-recognition, and so on. You can see which ones need GPU directly in docker-compose.yml since they include a `deploy.resources.reservations.devices` stanza with `driver: nvidia`. 
 
 TIP: if you run into `Cannot start service ...: could not select device driver "nvidia"`, you can use this checklist to guide you:

config/Readme.md

@@ -11,28 +11,23 @@ This file contains the API key used to call Google Places API. [Here](https://de
 * [autour preprocessor](https://github.com/Shared-Reality-Lab/IMAGE-server/tree/main/preprocessors/autour)
 * [openstreetmap preprocessor](https://github.com/Shared-Reality-Lab/IMAGE-server/tree/main/preprocessors/openstreetmap)
 
-Following is the sample format of maps.env file:
-```
-GOOGLE_PLACES_KEY = [INSERT KEY STRING]
-```
+Check [`maps.env.example`](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/config/maps.env.example) for format specifications.
 
-### apis-and-selection.env
+### cloud-ocr.env
 Entries in this env file are required by the following preprocessors:
 * [ocr-clouds-preprocessor](https://github.com/Shared-Reality-Lab/IMAGE-server/tree/main/preprocessors/ocr)
 
-Following is the sample format of apis-and-selection.env file:
+Check [`cloud-ocr.env.example`](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/config/cloud-ocr.env.example) for format specifications.
+
+#### Note:
+`CLOUD_SERVICE` is set within the [docker-compose](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/docker-compose.yml) file.
 
-```
-AZURE_API_KEY = [INSERT KEY STRING]
-FREEOCR_API_KEY = [INSERT KEY STRING]
-GOOGLE_APPLICATION_CREDENTIALS = [INSERT KEY FILE PATH AS STRING]
-CLOUD_SERVICE = [INSERT OPTION STRING (see options below)]
-```
 * `CLOUD_SERVICE` determines the desired cloud service to be used. Its possible values are:
     * `AZURE_OCR` (for [Microsoft Azure OCR API](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2/operations/56f91f2e778daf14a499f20d))
     * `AZURE_READ` (for [Microsoft Azure Read API](https://learn.microsoft.com/en-us/azure/cognitive-services/computer-vision/how-to/call-read-api))
     * `GOOGLE_VISION` (for [Google Cloud Vision API](https://cloud.google.com/vision/docs/ocr))
     * `FREE_OCR` (for [Free OCR API](https://ocr.space/OCRAPI))
+
 * `AZURE_API_KEY` is found in your [Azure portal](https://portal.azure.com)
 * `FREEOCR_API_KEY` can be obtained at [OCR API portal](https://ocr.space/ocrapi)
 * `GOOGLE_APPLICATION_CREDENTIALS` contains path to credentials file. Refer [documentation](https://cloud.google.com/docs/authentication/application-default-credentials#GAC) for details.
@@ -42,7 +37,11 @@ CLOUD_SERVICE = [INSERT OPTION STRING (see options below)]
 This env file is required by the following preprocessors:
 * [graphic tagger](https://github.com/Shared-Reality-Lab/IMAGE-server/tree/main/preprocessors/graphic-tagger)
 
-Following is the sample format of azure-api.env file:
-```
-AZURE_API_KEY = [INSERT KEY STRING]
-```
+Check [`azure-api.env.example`](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/config/azure-api.env.example) for format specifications.
+
+### llm.env
+Entries in this env file are required by preprocessors that use an LLM via the shared LLM client ([`utils/llm/client.py`](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/utils/llm/client.py)).
+
+Check [`llm.env.example`](https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/config/llm.env.example) for format specifications.
+
+**Using a different model per service:** `llm.env` sets the default LLM endpoint and model for *all* preprocessors that use the shared LLM client. If a specific preprocessor needs to use a different endpoint or model than this default, do not edit this file. Instead, create a separate `.env` file for that preprocessor override using the naming convention: `<preprocessor-name>.env` within the existing `config/` directory. This applies to any service — preprocessor, handler, or otherwise — that may support its own override in the future, not just LLM-based preprocessors.

config/azure-api.env.example

@@ -0,0 +1,2 @@
+# Following is the sample format of azure-api.env file:
+AZURE_API_KEY = [INSERT KEY STRING]

config/cloud-ocr.env.example

@@ -0,0 +1,4 @@
+# Following is the sample format of cloud-ocr.env file:
+AZURE_API_KEY = [INSERT KEY STRING]
+FREEOCR_API_KEY = [INSERT KEY STRING]
+GOOGLE_APPLICATION_CREDENTIALS = [INSERT KEY FILE PATH AS STRING]

config/llm.env.example

@@ -0,0 +1,12 @@
+# Sample format of llm.env file:
+LLM_API_KEY = [INSERT KEY STRING]
+LLM_URL = [INSERT OPENAI-COMPATIBLE ENDPOINT URL]
+LLM_MODEL = [INSERT MODEL NAME]
+
+# Note:
+
+# LLM_URL can point at a local model server (e.g. Ollama, vLLM) 
+# or a cloud provider that exposes an OpenAI-compatible API 
+# (e.g. OpenRouter).
+
+# LLM_MODEL must be a model the chosen endpoint supports.

config/maps.env.example

@@ -0,0 +1,2 @@
+# Following is the sample format of maps.env file:
+GOOGLE_PLACES_KEY = [INSERT KEY STRING]