Skip to content

env_params: support full dot-path keys from TestDefinition (cmd_args.foo.bar, bench_cmd_args.*) #950

Description

@rutayan-nv

Summary

Today env_params keys must name a top-level cmd_args field. They cannot address nested attributes (cmd_args.foo.bar) or other CLI argument bags some workloads expose (e.g. vllm's bench_cmd_args). This makes the env_params path semantics inconsistent with the rest of the parsing surface.

Proposed by @podkidyshev in review of #901 (thread). Explicitly out of scope for #901; tracked here as a follow-up.

Current behavior

TestDefinition.validate_env_params rejects any key that is not a top-level cmd_args field:

unknown = sorted(k for k in self.env_params if k not in cmd_args_fields)
if unknown:
    raise ValueError(f"env_params keys {unknown} are not cmd_args fields on ...")

and is_env_sampled / is_dse_job match on the bare key, so nested and non-cmd_args paths are unreachable.

Proposed behavior

Let an env_params key be a full dot-path rooted at TestDefinition, so it can target any nested attribute consistently:

num_nodes = 2

[cmd_args]
  foo = 1

  [cmd_args.args]
  bar = [1, 2, 3]

[env_params]
"cmd_args.args.bar" = { weights = [0.1, 0.1, 0.8] }

This makes env_params paths consistent with how other features address nested config, and unblocks workloads whose sweepable knobs live outside the root cmd_args (e.g. bench_cmd_args).

Scope notes

  • Path resolution must be shared with is_env_sampled / is_dse_job / param_space / apply_params_set so all of them agree on what a key addresses.
  • Keep the existing leaf/list/weights validation (a key must resolve to a leaf candidate list).
  • Backward compatibility: bare top-level keys should keep working (or a clear migration).

cc @podkidyshev (offered to take this).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions