Tests: mark visual tests optional

[antirotor]

Changelog Description

Make visual test optional except on GH Action.

Additional info

Running test locally will produce inconsistent errors about pixel deviation tolerances. Running them on Github Actions is more consistent and produce no errors as of now. Until we can figure out why the testing differences are so big on individual machines, we'll mark those tests optional.

BREAKING CHANGE: UI visual tests are not running by default, you need to add optional tests using --optional argument.

Testing notes:

Running tests using ./tools/manage.sh run-tests should deselect failing visual tests.
Running tests using ./tools/manage.sh run-tests --optional should a run visual tests.
CI Action on Github should run them too.

[Copilot]

Pull request overview

This PR makes UI visual regression tests opt-in for local runs (to avoid machine-specific pixel diff failures) while still running them in GitHub Actions for consistency.

Changes:

• Introduces a new optional pytest marker and excludes optional tests by default via pyproject.toml.
• Marks visual UI tests as optional, and updates manage.sh / manage.ps1 to include them when --optional is provided.
• Updates CI workflow to run tests with --optional, and ignores generated test_images outputs.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/manage.sh	Adds --optional switch handling for test runs (marker expression changes).
tools/manage.ps1	Adds --optional switch handling for test runs on Windows (marker expression changes).
tests/client/ayon_core/ui/test_visual.py	Marks visual regression tests as optional.
pyproject.toml	Registers optional marker and excludes it by default via pytest addopts.
.gitignore	Ignores test_images output directory created on visual test failures.
.github/workflows/pr_unittests.yaml	Runs CI tests with --optional to keep visual tests enabled on GitHub Actions.

Suppressed comments (2)

tools/manage.sh:180

• run-tests currently checks for --optional but doesn't shift it out, and it also doesn't forward any remaining arguments to pytest. This makes it impossible to pass through useful pytest flags (e.g. --force-regen, -k, -vv) without causing pytest to error once argument forwarding is added later. Consider shifting --optional when present and forwarding the remaining args to pytest.

  local test_marker="not server and not optional"
  if [ "$1" = "--optional" ]; then
    test_marker="not server"
  fi
  uv sync --extra test
  uv run pytest ./tests -m "$test_marker"

tools/manage.ps1:243

• Help output only documents run-tests --optional, but run-tests without --optional remains the default behavior. Consider listing both forms so users can discover how to run the default (non-optional) suite.

    Write-Info -Text "  run-tests --optional          ", "Run ayon-core tests including optional tests" -Color White, Cyan

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.