[rhaiis] Add CPU support

[maryamtahhan]

Summary

This PR adds full CPU accelerator support to the RHAIIS orchestration layer,
covering engine configuration, manifest generation, CLI ergonomics, and
concurrent-load testing — plus fixes for regressions introduced during the rebase.

feat: CPU accelerator support (1aed41c)

Engine / config

• runtime_config.py: get_engine_args returns rhaiis.vllm_args_cpu for CPU; get_serving_image resolves rhaiis.images.cpu / rhaiis.images.cpu-vanilla by flavor; merge_env_vars layers accelerator_env_vars.cpu + accelerator_env_vars.cpu-{flavor} (LD_PRELOAD only on rhaiis)
• config.d/rhaiis.yaml: adds rhaiis.images.cpu / cpu-vanilla, vllm_args_cpu (trust-remote-code, no-enable-log-requests, tensor-parallel-size: 1), accelerator_env_vars.cpu (VLLM_CPU_KVCACHE_SPACE=40), accelerator_env_vars.cpu-rhaiis (LD_PRELOAD)
• config.d/models.yaml: adds tinyllama-cpu, qwen3-0-6b-cpu, llama-3-2-1b-cpu, llama-3-2-3b-cpu, granite-3-2-2b-cpu, llama31-8b-w8a8-cpu with per-model max-model-len and vllm_args
• config.d/workloads.yaml: adds cpu-smoke, cpu-chat-baseline, cpu-rag-baseline, cpu-code-baseline, cpu-summarization-baseline, cpu-chat-realistic, cpu-code-realistic

CLI

• cli.py test: adds --accelerator cpu, --cpu-flavor {rhaiis,vanilla} flags; _apply_cli_overrides wires accelerator/flavor/engine into config; dry-run prints CPU flavor
• manifests.py: passes accelerator through to _build_resources; CPU branch in _build_resources sets Guaranteed QoS (limits == requests, no GPU keys)
• test_phase.py: passes accelerator to manifest builders

Docs / tooling

• CPU_TESTING.md: full guide — cluster setup (login → diagnose → namespace → secrets → PVC → artifact dir), preset-based smoke test and concurrent-load commands, model/workload/image tables, troubleshooting
• scripts/diagnose_cpu_cluster.sh: checks node resources, AVX2/AVX-512/AMX, NUMA topology, CPU Manager policy, KServe CRDs

feat: CPU concurrent load matrix (5280090)

• cpu_concurrent_load_phase.py: run/do_test sweep models × cpu_requests × workloads, calling _run_test with deploy_cfg_overrides={cpu_request: ...} per cell; honours continue_on_error
• cli.py concurrent-load: new subcommand with --models, --cpu-requests, --workloads, --namespace, --cpu-flavor, --image-pull-secret, --service-account-name, --continue-on-error, --dry-run
• Defaults: tinyllama-cpu, [8, 16, 32] CPU requests, cpu-chat-baseline

fix: CPU accelerator regressions (388cdee)

• HF+PVC storageUri (GPU regression): restored if storage_source == "hf" and storage_pvc / elif storage_source != "hf" — the branch had inadvertently flipped the condition to pvc, causing GPU jobs to lose the model cache PVC
• --image-pull-secret raises on missing key: was calling set_config("rhaiis.deploy.image_pull_secret", ...) (singular, non-existent); now writes [secret] to rhaiis.deploy.image_pull_secrets (the actual list key) in both test and concurrent-load; dry-run updated to match
• CPU Guaranteed QoS: _build_resources CPU branch now returns early with matching limits and requests for cpu/memory; without limits the pod could burst the whole node
• --cpu-flavor clobbering presets: concurrent-load --cpu-flavor default changed from "vanilla" to None; flavor is only set_config'd when explicitly passed; cpu_concurrent_load_phase drops the cpu_flavor param entirely so preset values survive
• Whitespace trim: comma-separated --models, --cpu-requests, --workloads now strip whitespace on split
• --serving-image key mapping: rhaiis flavor → rhaiis.images.cpu, vanilla → rhaiis.images.cpu-vanilla (was writing non-existent rhaiis.images.cpu-rhaiis)
• max-model-len precedence: removed max-model-len from vllm_args_cpu defaults so per-model values (tinyllama-cpu: 2048) are not overridden by the engine default
• CPU presets restored: adds cpu, cpu-vanilla, cpu-smoke, vanilla-cpu-smoke, cpu-chat-baseline, vanilla-cpu-chat-baseline to presets.d/presets.yaml with memory_request: 64Gi
• Fournos hardware resolver: ci.py resolve_hardware_request returns {} immediately for accelerator == "cpu" before any GPU/TP lookup
• Offline validation: test_cpu_config.py checks image selection, LD_PRELOAD isolation, tinyllama max-model-len stays 2048 after merge, and CPU _build_resources Guaranteed QoS

docs: CPU_TESTING.md setup flow (a999cf0)

• Added model-pvc PVC creation as step 5 of cluster setup (required — KServe mounts it at /mnt/models for the HF download cache)
• Updated smoke test and concurrent-load examples to use presets (vanilla-cpu-smoke, cpu-smoke, cpu-vanilla, cpu) rather than explicit flags
• Added first-run download note and StorageClass tip

Test plan

• Offline: PYTHONPATH=$PWD python projects/rhaiis/orchestration/test_cpu_config.py
• Dry-run (vanilla): python -m projects.rhaiis.orchestration.cli test --preset vanilla-cpu-smoke --namespace forge-rhaiis --dry-run
• Dry-run (rhaiis): python -m projects.rhaiis.orchestration.cli test --preset cpu-smoke --namespace forge-rhaiis --image-pull-secret rhaiis-pull-secret --dry-run
• Dry-run (concurrent-load): python -m projects.rhaiis.orchestration.cli concurrent-load --preset cpu --image-pull-secret rhaiis-pull-secret --dry-run
• Live smoke: python -m projects.rhaiis.orchestration.cli test --preset vanilla-cpu-smoke --namespace forge-rhaiis
• GPU run unaffected: confirm storageUri condition is correct for storage_source=hf

🤖 Generated with Claude Code

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign sjmonson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @maryamtahhan. Thanks for your PR.

I'm waiting for a openshift-psap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 397c5cd0-1cfb-4123-a00e-de0071c3b3d3

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}