feat(elt-pipelines): Port opralogweb to elt-pipelines

[ambolt314]

Summary

Move OPRALOGWEB staging tables from warehouses to elt-pipelines

Ref #458 .

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 61063c26-7bca-487a-8cf8-326647f240b9

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[martyngigg]

I should have spotted this the last time I looked but we need to keep the old model files in warehouses around too. The production pipelines are still using them so we can't delete them until we've fully switched over. For this and the other pull requests we'll just need copies rather than moves of the files.

[martyngigg]

Thanks for putting these files back.

As these are the old pipelines their content needs to remain unchanged. They still use the old ingest approach that changes the column names to lower case. Essentially this pull request and the future ones should not contain any changes in the warehouses directory.

The simplest way to get everything in the warehouses directory back to what it looks like on main is to run these commands

git fetch
git checkout origin/main warehouses

from the root of the repository and then commit and push the results.

[martyngigg]

Looks good. The old pipelines still work as needed for production.

The full elt run facility_ops accelerator.opralogweb works locally when the environment variables are set.