[rhaiis] Add named workload aliases, prefix-cache toggles, and composite benchmark presets - #183
Conversation
…ite benchmark presets Add human-readable workload aliases (balanced, variable, summarization, long-context, ultra-long-context, multi-turn, heavy-heterogeneous) that mirror profile1-7 for better readability in presets and CLI usage. Add prefix-cache-on/off feature toggle presets to enable A/B comparison of vLLM prefix caching behavior. Add composite benchmark presets for common test scenarios: - benchmark-standard (balanced + variable) - benchmark-long-context (long-context + ultra-long-context) - benchmark-full-sweep (all workloads) - benchmark-multi-turn with prefix-on/off variants - benchmark-heterogeneous
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Summary
balanced,variable,summarization,long-context,ultra-long-context,multi-turn,heavy-heterogeneous) that mirror profile1–7 for better readability in presets and CLI usage.benchmark-standard— balanced + variablebenchmark-long-context— long-context + ultra-long-contextbenchmark-full-sweep— all workloadsbenchmark-multi-turnwithprefix-on/prefix-offvariantsbenchmark-heterogeneousMotivation
The numbered profile names (profile1–7) are kept for model-furnace compatibility, but they're hard to remember. Named aliases let users write
benchmark-multi-turn-prefix-on + llama-70b + nvidiainstead of juggling profile numbers and manual vllm arg overrides.Test plan
--enable-prefix-cachingto vLLM