[P2] Audit: Sitemap needs validation for legacy URLs, parameter URLs, and stale lastmod values

[baixiangcpp]

Imported from the byteflow.tools deep audit on 2026-06-22.

Problem: sitemap.xml exists, but it needs a full validation pass. The audit identified legacy taxonomy URLs and possible parameterized All Tools states that should not be present. lastmod values also need verification so crawlers receive useful freshness signals.

Affected URL:

• https://byteflow.tools/sitemap.xml

Reproduction steps:

1. Fetch sitemap.xml.
2. Parse every URL.
3. Check for legacy category URLs such as /en/web-api, /en/design-media, and /en/convert-encode.
4. Check for query-parameter URLs.
5. Check for 404, redirect, noindex, or canonical-to-other-page entries.
6. Validate lastmod against actual content update history.

Expected behavior: sitemap should include only canonical, indexable, 200-status URLs with meaningful lastmod values.

Actual behavior: sitemap existence is confirmed, but its content needs systematic validation.

Impact: crawl budget waste, stale signals, and indexation of unintended pages.

Proposed fix:

• Add a sitemap audit script to CI.
• Fail builds if sitemap contains redirected, noindex, 404, parameterized, or legacy URLs.
• Generate lastmod from content/tool manifest modification dates where possible.
• Split sitemap by locale or page type if size grows.

Acceptance criteria:

• Sitemap contains only canonical indexable 200 URLs.
• Legacy and parameter URLs are excluded.
• lastmod values are generated from reliable source data.
• CI validates sitemap before deploy.

[baixiangcpp]

Closed as completed by the canonical BF audit backlog work. This older non-BF audit issue duplicates work now merged through PRs #209-#219, with the BF issues closed and guard/test coverage in place. The remaining non-duplicate follow-up is #7, which stays open because the upstream Next/PostCSS advisory is not fixed yet.

[baixiangcpp]

Reopened after closure audit. This maps to the still-open BF-047 sitemap audit gap: legacy/noindex/parameter exclusions exist, but full per-sitemap-URL status/canonical/robots/hreflang validation and release diff reporting are not complete yet.