Skip to content

scan-path-standards.py rejects Agent Skills spec frontmatter fields (metadata, license) and misparses multi-line description #117

Description

@matera-ipd

Title: scan-path-standards.py rejects Agent Skills spec frontmatter fields (metadata, license) and misparses multi-line description: >

Describe the bug

skills/bmad-workflow-builder/scripts/scan-path-standards.py (and the copy under skills/bmad-agent-builder/scripts/) hard-codes VALID_FRONTMATTER_KEYS = {'name', 'description'} and reports every other top-level key as a high finding with the action "Remove … from frontmatter". The Agent Skills specification (https://agentskills.io/specification) allows additional optional frontmatter fields — license, allowed-tools, metadata (arbitrary key/value map), compatibility — so a SKILL.md that is fully spec-compliant fails the scan.

The check also parses the frontmatter line by line and treats any line containing : as a key: value pair. Two consequences:

  1. Nested keys under metadata: (e.g. author:, risk:) are each reported as an invalid top-level key.
  2. A folded/literal block description: > is misparsed: any continuation line that contains a colon is reported as an invalid key whose "name" is the description text itself.

quick_validate.py in the same skill has a related limitation: --allow-keys makes the key set configurable (good), but split_frontmatter rejects any line that is not key: value, so a multi-line description: > fails with frontmatter line is not key: value even when metadata is allowed.

Because the Analyze intent runs these scanners as a pre-pass and feeds the findings to the lenses, a module whose skills use metadata (in our case, a required metadata.xxx.risk classification consumed by our own validator) gets a frontmatter finding on every skill, and the Edit intent is nudged to "fix" spec-compliant frontmatter.

Steps to reproduce

  1. Create a minimal skill:
mkdir -p /tmp/example-skill && cat > /tmp/example-skill/SKILL.md <<'EOF'
---
name: example-skill
description: >
  Validates things. Use when the user says "validate: this" or asks
  for a check.
license: Apache-2.0
metadata:
  author: acme
  risk: read
---

# example-skill
EOF
  1. Run the path-standards scanner:
python3 skills/bmad-workflow-builder/scripts/scan-path-standards.py /tmp/example-skill

Output (bmad-builder v2.2.2, scanner version: 3.0.0) — status: fail, five high findings in category frontmatter:

line 4 Invalid frontmatter key: Validates things. Use when the user says "validate
line 6 Invalid frontmatter key: license
line 7 Invalid frontmatter key: metadata
line 8 Invalid frontmatter key: author
line 9 Invalid frontmatter key: risk
  1. Run the quick validator, with and without the extra keys allowed:
python3 skills/bmad-workflow-builder/scripts/quick_validate.py /tmp/example-skill/SKILL.md
python3 skills/bmad-workflow-builder/scripts/quick_validate.py /tmp/example-skill/SKILL.md --allow-keys name,description,license,metadata

Both return {"ok": false, "errors": [{"check": "frontmatter", "message": "frontmatter line is not key: value -> 'for a check.'"}]}.

Expected behavior

  • Top-level keys defined by the Agent Skills spec (name, description, license, allowed-tools, metadata, compatibility) are accepted; keys nested under metadata are never reported.
  • A folded (>) or literal (|) block scalar for description is parsed as a single value, so continuation lines are not treated as keys.
  • status: pass for the skill above.

Suggested implementation, in order of preference:

  1. Parse the frontmatter as YAML (PyYAML if available, with the current line-based parser as a fallback), and validate top-level keys against the spec set.
  2. If a real YAML parser is out of scope, at minimum: (a) track indentation so that indented lines under a top-level key are skipped, and (b) treat a value of > or | as the start of a block scalar and skip its indented continuation lines.
  3. Make the allowed set configurable from the builder's customize.toml (the same way quick_validate.py already exposes --allow-keys), so organizations that add their own metadata sub-fields do not have to override the scanner.

Environment (if relevant)

  • Model(s) used: n/a (deterministic script)
  • Agentic IDE used: Claude Code
  • BMad version: core 6.10.0; bmad-builder v2.2.2 (sha 4a1422274a2acb0fb0ec0511753da6263948f072, channel stable)
  • Project language: Python 3.12

Screenshots or links

  • Agent Skills specification, frontmatter fields: https://agentskills.io/specification
  • skills/bmad-workflow-builder/scripts/scan-path-standards.py, check_frontmatter() / VALID_FRONTMATTER_KEYS
  • skills/bmad-workflow-builder/scripts/quick_validate.py, split_frontmatter()

PR

Happy to open a PR with the YAML-aware parser and tests if the maintainers agree with the direction. Please indicate which of the three options above you prefer.

Additional context

We maintain a BMad module (21 skills, 4 agents) whose skills carry a required metadata.xxx.risk field consumed by our own inventory validator. We are adopting the builder skills (Analyze/Edit) as the authoring path for the module, and today every skill gets a frontmatter finding that we have to exclude from our non-regression gate and declare as an accepted convention in a build_standards override. Aligning the scanner with the spec would remove that noise for any module that uses metadata, not just ours.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions