[pull] devel from ansible:devel - #572
Open
pull[bot] wants to merge 2153 commits into
Open
Conversation
github-actions
Bot
added
component:api
component:ui
community
component:awx_collection
labels
github-actions
Bot
added
the
dependencies
* Fix ARM64 build failure by upgrading dev container Node.js to 18 Node.js 16.13.1 fails to extract on ARM64 in Docker BuildKit's overlay filesystem during multi-arch builds. Upgrade to Node 18 which is already used by the UI builder stage and has proper ARM64 support. * Fix collectstatic failure by setting AWX_MODE=default AWX_MODE=defaults is an intentionally "invalid" environment name that: 1. Loads only defaults.py - the base settings file without any environment-specific overrides (development_defaults.py, production_defaults.py, etc.) 2. Bypasses production checks - since "production" not in "defaults", it skips the assertion that requires /etc/tower/settings.py to exist 3. Bypasses development mode - since is_development_mode would be false This is perfect for collectstatic during container build because: - No database connection needed - No secret key needed (hence SKIP_SECRET_KEY_CHECK) - No PostgreSQL version check (hence SKIP_PG_VERSION_CHECK) - Just need minimal Django settings to collect static files
…aarch64 (#16225) Use dnf module for Node.js 18 instead of n version manager The n version manager fails to extract Node.js archives due to very long file paths in include/node/openssl/archs/ directories when running in Docker BuildKit's overlay filesystem. This causes CI build failures with tar "Cannot open: Invalid argument" errors. Switch to installing Node.js 18 directly from CentOS Stream 9's module stream which avoids the archive extraction issue entirely.
Switch to git-based installation of kubernetes python client from github.com/kubernetes-client/python at commit df31d90d6c910d6b5c883b98011c93421cac067d (release-34.0 branch). This also allows removing the urllib3<2.4.0 upper bound constraint that was previously required by kubernetes 34.1.0 from PyPI.
Introduces new Makefile targets to update and upgrade requirements files using pip-compile, both directly and via docker-runner. These additions streamline dependency management for development and CI workflows.
Refactored code to use Python's built-in datetime.timezone and zoneinfo instead of pytz for timezone handling. This modernizes the codebase and removes the dependency on pytz, aligning with current best practices for timezone-aware datetime objects.
* docs: update readthedocs.io URLs to docs.ansible.com equivalents 🤖 Generated with Claude Code https://claude.ai/code Co-Authored-By: Claude <noreply@anthropic.com> * Update Bullhorn newsletter link in communication docs --------- Co-authored-by: Claude <noreply@anthropic.com>
…le organizations (#16170) fixed module organizations description for option notification_templates_approvals Co-authored-by: Pascal Kontschan <pascal.kontschan.extern@atruvia.de>
Assited-by: Claude
…16214) * Slightly alter history to avoid having a Django 5 related migration * Revert prior field states to be slightly more clear
- Move kubernetes from git-based install to PyPI (v35.0.0 now available) - Remove urllib3 cap comment since kubernetes 35.0.0 no longer restricts it - Update README.md upgrade blocker documentation
Remove transitive dependencies no longer needed by kubernetes 35.0.0 Removes google-auth and rsa which were transitive dependencies of the older kubernetes client but are no longer required in v35.0.0. Adds cachetools as a direct dependency since it's used by awx/conf/settings.py for TTLCache (was previously a transitive dep of google-auth).
) * Enhance OpenAPI schema with AI descriptions and fix method names Add x-ai-description extensions to API endpoints for better AI agent comprehension. Fix view method names to ensure proper drf-spectacular schema generation. * Enhance OpenAPI schema with AI descriptions and fix method names Add x-ai-description extensions to API endpoints for better AI agent comprehension. Fix view method names to ensure proper drf-spectacular schema generation.
… memory alloc (#16563) Django's cascade collector materializes all JobHostSummary IDs into a single UPDATE ... IN (...) to SET_NULL on Host.last_job_host_summary. With many jobs x hosts this exceeds PostgreSQL's max memory allocation size for a single query (1GB). Pre-delete JHS rows and clear Host FK references in chunks of 1000 job IDs using raw SQL before Django's .delete() runs, so the cascade collector finds nothing to collect. Assisted-by: Claude Code via Google Vertex AI
…ons (#16558) * AAP-84057: Set PostgreSQL statement_timeout on web worker DB connections When uwsgi's harakiri kills a worker, the PostgreSQL backend continues running the query indefinitely. These abandoned queries accumulate and create resource contention for all other queries. Add a connection_created signal handler that sets statement_timeout on new DB connections. Under uwsgi, the timeout is auto-derived from the harakiri value (minus 5s margin so PostgreSQL cancels the query before uwsgi kills the worker). Outside uwsgi (task workers, migrations), no timeout is applied. A manual DATABASE_STATEMENT_TIMEOUT setting is available as a fallback for non-uwsgi deployments. * Remove timeout value caching because it brings no significant gains * Use proportional margin between statement_timeout and harakiri timeout * Fix zero-harakiri test to patch fake uwsgi module instead of None * Refactor statement_timeout from signal handler to connection string Move statement_timeout configuration from a connection_created signal handler (extra SQL round-trip per connection) to a dynaconf merge function that sets it via the libpq OPTIONS connection string parameter. This mirrors the existing merge_application_name() pattern and eliminates the SET statement on every new connection. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ream pagination (#16557) * AAP-83773 — Use unfiltered count for activity stream pagination Forward-port of tower#7606 (stable-2.6). The RBAC-filtered COUNT(*) on activity_stream takes ~36 min per call on large tables (713K rows) due to the pk__in subquery shape introduced by the AAP-81860 LEFT JOIN fix. Replace with an unfiltered table count for pagination -- an approximate over-count is harmless for UI page navigation while the actual page results remain RBAC-filtered. * AAP-83773 — Add unit tests for ActivityStreamPagination Cover ActivityStreamPaginator and ActivityStreamPagination to satisfy SonarCloud coverage gate on new lines in awx/api/pagination.py. * AAP-83773 — Save/restore django_paginator_class in base Pagination Address review feedback from lallen92: use a save/restore pattern in Pagination.paginate_queryset() so subclasses only need to set the class attribute. Remove the now-redundant paginate_queryset override from ActivityStreamPagination. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Liam Allen <lallen@redhat.com>
* fix: prefer scm_revision as the cache_id if available * add tests * fix: make sure galaxy requirements are always updated when a new revision is pulled * make sure existing project cache id is used when available --------- Co-authored-by: Liam Allen <lallen@redhat.com>
dict() with keyword arguments is idiomatic for Ansible module argument_spec definitions. Suppress the "use literal instead of constructor" rule (python:S7498) for awx_collection/plugins/modules/. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com>
awxkit has its own tox.ini and when `tox -e linters` runs, awxkit/.tox/ can get populated with a py3 virtualenv. Since flake8 is configured to scan the `awxkit` directory, it recurses into awxkit/.tox/py3/lib/python3.12/site-packages/ and reports hundreds of false positives (F405, E265, E266) from third-party packages like PyYAML. Adding .tox to the flake8 exclude list prevents this — matching the existing exclusion of `env` for virtualenvs. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
#16555) * AAP-81173 — Replace 4-way OR with UNION in unified job list RBAC query The UnifiedJobAccess.filtered_queryset() method used a 4-way OR to determine which unified jobs a user can see. Under load, the OR forces PostgreSQL to evaluate all four branches in a single plan, preventing branch-specific index optimization and causing 35 hours of DB time per 30-minute Scale Lab window. Split each RBAC branch (template read_role, inventory update, ad-hoc command, org auditor) into separate querysets combined with UNION, giving the planner an independent optimal plan per branch. The UNION result is wrapped in pk__in= for compatibility with BaseAccess.get_queryset() prefetch_related and the workflowapproval filter. This follows the same pattern proven in AAP-83319 (team list UNION fix). * AAP-81173 — Add UnifiedJobPagination to prevent COUNT regression The pk__in UNION pattern used for RBAC filtering forces the large outer table as the driving table for COUNT(*), requiring PostgreSQL to materialize all subquery result sets. On large deployments this produces catastrophic query times (see AAP-83773 for the identical issue on activity_stream). Override the paginator count to use an unfiltered UnifiedJob.objects.count() — the over-count is acceptable for pagination UI. Also fix test_unified_job_list_rando_sees_nothing to assert on results length instead of count, since count is now unfiltered. * AAP-81173 — Add unit tests for UnifiedJobPagination coverage Cover the UnifiedJobPaginator.count cached property and the count_disabled branch in UnifiedJobPagination.paginate_queryset to satisfy SonarCloud's 90% new-code coverage gate. * AAP-81173 — Address review feedback: save/restore paginator class, format consistency - Fix Pagination.paginate_queryset() to save/restore django_paginator_class instead of hardcoding DjangoPaginator in the finally block. This lets subclasses set the class attribute without needing to override the method. - Remove UnifiedJobPagination.paginate_queryset() override — now only needs to set django_paginator_class = UnifiedJobPaginator as a class attribute. - Format by_org_auditor consistently with the other three UNION branches. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* test: add regression tests for old RBAC sync after bulk claims Tests that save_user_claims (which uses bulk_create, skipping signals) correctly syncs old Role.members when run through AwxJWTAuthentication. Covers add, remove, and multi-org/team scenarios. * test: call process_permissions instead of internal _sync_old_rbac Tests now exercise the public AwxJWTAuthentication.process_permissions() API with the JWT layer mocked, rather than calling the private _sync_old_rbac method directly. The test_bulk_claims_skips_old_rbac_signals test asserts that save_user_claims does NOT populate old Role.members via signals. On current DAB, save_user_claims uses the serial give_permission path which fires signals normally, so old Role.members IS populated. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* [AAP-82668] Skip old RBAC sync on cascade-deleted assignments When a non-RBAC parent (e.g. Organization) is deleted, Django cascades to RoleUserAssignment/RoleTeamAssignment and fires post_delete for each. The sync_assignments_to_old_rbac_delete handler would then do 3-4 FK/GFK queries per assignment to sync removals to the old Role model — entirely redundant since the old Role M2M tables cascade-delete from the same content object. Use Django's post_delete `origin` kwarg (4.1+) to detect this: when origin is a Model instance whose app_label is not dab_rbac, the delete is a cascade from a non-RBAC parent and the sync is skipped. Measured on 150 teams / 321 users / 48K assignments: Baseline: 78.8s With fix: 20.4s (via service-index endpoint) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…job RBAC query (#16577) * AAP-87084 — Replace UNION with OR + pre-computed role set in unified job RBAC query The UNION ALL approach introduced by AAP-81173 forces PostgreSQL to materialize all accessible job IDs across four RBAC branches before any filtering or LIMIT can apply. This causes a 10x per-call regression on the unified jobs list (36ms → 366ms) and a 3x regression on the dashboard date-bucketed aggregation (88.9ms → 261.5ms). Pre-compute the user's role IDs once as a Python list and pass them as literal parameters to OR-based RoleEvaluation filters. This eliminates the 4x redundant roleuserassignment subquery scans and restores single-pass filtering with early LIMIT exit. Resolves: AAP-87084, AAP-87087 * Assert RBAC query list is non-empty before checking for UNION Ensures the test does not pass vacuously if no RBAC query is captured. * Add test for singleton permission shortcut branches Exercises the UJT, inventory, and org auditor singleton permission paths in filtered_queryset() to increase coverage on new code. * Add test for team-granted permissions in unified job list Verifies that a user who can view a job template only through a team assignment (not direct) sees the corresponding unified jobs. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
… index (#16586) Adds `create_host_summary_index` management command that creates the composite index (host_id, id DESC) on main_jobhostsummary using CREATE INDEX CONCURRENTLY. The command is idempotent: it checks for existing valid/invalid indexes, drops invalid indexes from failed prior attempts, and skips if the index already exists. Disables statement_timeout for the session to avoid killing the index build on large tables where ALTER ROLE timeout is set. Forward-port of tower#7893 (stable-2.6). Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…hs (#16585) Use SeparateDatabaseAndState with CREATE INDEX IF NOT EXISTS so the migration succeeds when the index was already created manually (via awx-manage create_host_summary_index or raw SQL workaround). Forward-port of tower#7895 (stable-2.7). Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
fix: do not delete and of the jwt managed roles during cleanup
…16579) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…Docker digest to 33add02 (#16524) Signed-off-by: red-hat-konflux-kflux-prd-rh02 <190377777+red-hat-konflux-kflux-prd-rh02[bot]@users.noreply.github.com> Co-authored-by: red-hat-konflux-kflux-prd-rh02[bot] <190377777+red-hat-konflux-kflux-prd-rh02[bot]@users.noreply.github.com>
When mintmaker bumps tekton resources it is failing on running the pipeline-migration-tool. This runs as a post upgrade task and fails when trying to run against tekton bundles. Disabling this to allow CI checks to pass and merge updated pipeline bundles. Signed-off-by: Ryan Williams <3375653+ryankwilliams@users.noreply.github.com>
…#16582) * Fixed migration from old to new rbac for sys auditor * Added first migration * Added testing for migrations and fix * Removed attempts to remove platform auditor assignments * Fixed migration test failures
…filter (#16599) * AAP-87084 — Revert UnifiedJob queryset to DAB APIs, scope actor role filter The previous patch (7242fe8) inlined all DAB RBAC internals into UnifiedJobAccess.filtered_queryset to pre-compute role IDs and avoid repeated subqueries. While it fixed the UNION regression from c84575c, it created 70 lines of duplicated logic inconsistent with the UnifiedJobTemplate path and fetched ALL user roles regardless of relevance. This commit: - Reverts filtered_queryset to the clean post-f8fa690 form that delegates to DAB's access_ids_qs / accessible_pk_qs APIs - Adds _scoped_actor_role_filter() as a temporary shim that pre-filters RoleUserAssignment by content_type_id, auto-including Org and Team CTs for inherited permissions and NULL for global roles - Wires the scoped filter into UnifiedJobTemplate.accessible_pk_qs - Adds tests for custom global roles, mixed permissions, and unrelated role exclusion The shim duplicates logic planned for DAB's _actor_role_filter content_type_ids parameter; once that merges, replace and delete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused ContentType and RoleUserAssignment imports to fix flake8 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * AAP-87084 — Remove _scoped_actor_role_filter shim, use DAB _actor_role_filter directly The scoped content_type_ids parameter now lives in DAB's _actor_role_filter. The AWX shim and its hardcoded Org/Team CT inclusions are no longer needed. The unified job accessible_pk_qs uses the unscoped _actor_role_filter for correctness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…Docker digest to 94fb1b4 (#16600) Signed-off-by: red-hat-konflux-kflux-prd-rh02 <190377777+red-hat-konflux-kflux-prd-rh02[bot]@users.noreply.github.com>
* Fix path traversal in awxkit YAML * coderabbitai feedback
…16587) Use shared lock for project copy when no sync is needed When a job template runs and no project sync is required (the common case for slice jobs), multiple jobs were serializing behind an exclusive lock even though they were only reading the shared project directory to copy it. Introduce read/write lock semantics using fcntl LOCK_SH/LOCK_EX: - acquire_lock gains an exclusive= parameter (default True, preserving existing behavior for all callers that don't pass it) - sync_and_copy always acquires LOCK_SH first, then refreshes DB state and calls get_sync_needs() under the lock — preserving the invariant that the sync check always happens under a lock (no TOCTOU window) - If sync is needed, LOCK_SH is released and LOCK_EX is acquired before proceeding; sync_and_copy_without_lock re-checks internally - project.refresh_from_db() is guarded by project.pk to handle unsaved model instances used in unit tests This allows slice jobs backed by the same project to copy the project tree concurrently instead of sequentially. Measured improvement on a 200MB/54k-file project with 8 slice jobs: 140.5s total lock wait → 0s, wall time 42.6s → 16.9s (2.5x). Assisted-by: Claude Code via Google Vertex AI <cmeyers+claude@redhat.com>
Migrate Python linting and formatting toolchain from Black (formatter) and Flake8 (linter) to Ruff, which provides both in a single tool. - pyproject.toml: remove [tool.black], add [tool.ruff] config sections - tox.ini: replace black+flake8 deps/commands with ruff in linters env - awxkit/tox.ini: replace flake8 with ruff in lint env - Makefile: replace `black` target with `format` + `lint`, keep `black` as legacy alias, update `check` and `api-lint` - pre-commit.sh: use `ruff format --check`, honor AWX_IGNORE_RUFF - requirements_dev.txt: swap black/flake8 for ruff - Dockerfile.j2: install ruff instead of black - CONTRIBUTING.md: update tool references - Fix noqa comments to use # separator instead of ; (ruff requirement) - Add noqa: F841 for false positive in awxkit/cli/__init__.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pure formatting commit — no logic changes. Run `ruff format` across awx/ and awxkit/ (awx_collection excluded).
The EE sidecar container creates /etc/receptor/receptor.conf on pod startup. If the task container's dispatcherd reaches the heartbeat before the config exists, the instance rejoins the cluster at full capacity with no receptor — causing fatal job failures with no recovery path. Three layers of defense: 1. dispatcherd management command: poll for the receptor config file before entering the main loop (10s timeout, exits with error to let k8s restart the container). 2. Heartbeat: if get_receptor_ctl() raises FileNotFoundError (config or socket missing), mark the instance offline so no jobs are dispatched to it. Previously this was silently ignored and the instance kept full capacity. 3. should_update_config(): catch FileNotFoundError and return True so write_receptor_config() can create the file on first boot instead of crashing. Prior art: #12698 (closed without merge) took a broader approach with custom exception types. See PR comment for overlap analysis. Closes: AAP-89010 Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…am (#16584) * Record DAB RBAC role assignment events in the activity stream * The steam message was showing incorrectly, it was inverted based on the legacy setup * object_relationship_type was missing, this is set n the legacy handler and emit_activity_stream_change emits it. * Skip cascade deletes from non-assignment origins, same reasoning as sync_assignments_to_old_rbac_delete
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )