Feature: Per-preprocessor endpoint override (#973)

[MansiDhanania]

Related to #973

Summary

Extended and documented the per-preprocessor LLM configuration override pattern. This includes a shared default endpoint for all LLM-related preprocessors, with the ability for any individual preprocessor to override the endpoint/model independently.

Initial Tests and Results

@Miliya-Ai's tests showed that the specifications in config/llm.env work with any OpenAI-compatible cloud endpoint called via client.py. The endpoint is used for all preprocessors requiring this service.

Changes made

In order to allow for different preprocessors, orchestrators or handlers to call a different endpoint, the following changes were incorporated:

The docker-compose.yml file was extended to include the paths to both a global and local .env file:

env_file:
  - path: ./config/llm.env
    required: true
  - path: ./config/<name>.env
    required: false

The global llm.env is always loaded first. A preprocessor-specific file, if present, loads second and overrides only the variables it sets for that one preprocessor. If the override file doesn't exist, the preprocessor falls back to the shared default.

This addresses the issue as there now exists:

• A shared default endpoint/model for preprocessors using the main LLM
• A service-level override when needed
• Setup documented in DEPLOYMENT.md and config/Readme.md.

How I tested this

• Tested on unicorn: created preprocessors/object-detection-llm/config/object-detection-llm.env calling Qwen3.8 via Mallory, while leaving config/llm.env pointed at the default Gemma 4 Pegasus endpoint. Recreated only the object-detection-llm container via override and queried an image.
• Confirmed via logs that:
• object-detection-llm picked up the Qwen override.
• Other preprocessors (e.g., content-categoriser) continued to call Gemma.

---

Required Information

• I referenced the issue addressed in this PR.
• I described the changes made and how these address the issue.
• I described how I tested these changes.

Coding/Commit Requirements

• I followed applicable coding standards where appropriate (e.g., PEP8)
• I have not committed any models or other large files.

New Component Checklist (mandatory for new microservices)

• I added an entry to docker-compose.yml and build.yml.
• I created A CI workflow under .github/workflows.
• I have created a README.md file that describes what the component does and what it depends on (other microservices, ML models, etc.).

OR

• I have not added a new component in this PR.

[jeffbl]

Looks good, and I love that we have a more disciplined system. Couple things below. In addition, suggest:

1. in README, remove the example keys and such, and just make example files with endpoints we have tested already filled in, so people can see exactly what we have working (without our keys, of course!)
2. the text between DEPLOYMENT.md and the README.md seems repetitive. Could be postponed, but this should probably be tightened up in the future.

[aidanbruneel]

@MansiDhanania I would like responses to these questions/concerns before approving.

[aidanbruneel]

Some further questions/concerns and also the PR description still shows the override at ./preprocessors/<name>/config/<name>.env, while the final implementation uses ./config/<preprocessor-name>.env. It also links to preprocessors/object-detection-llm/config/Readme.md, which doesn't appear to exist on this branch. Could you update the description to match the final implementation?

[aidanbruneel]

This still seems inconsistent with the configuration above. cloud-ocr.env now contains the provider credentials, while CLOUD_SERVICE is selected directly in docker-compose.yml. Could we update this sentence accordingly?

[aidanbruneel]

I see what you mean from your reply. I think this sentence is still a little ambiguous in this LLM-specific section, though. maps.env.example and cloud-ocr.env.example document shared/base configurations rather than examples of the <preprocessor-name>.env override described above. Could we say that the same layered env_file pattern can be used for other service configurations, while pointing LLM overrides specifically to llm.env.example?

[aidanbruneel]

Looks good to me :)