Skip to content

Fix relational JDBC location-overlap index to match the optimized sibling-check query - #5331

Closed
zhang-arvin wants to merge 1 commit into
apache:mainfrom
zhang-arvin:fix/jdbc-location-overlap-index-catalog-id
Closed

Fix relational JDBC location-overlap index to match the optimized sibling-check query#5331
zhang-arvin wants to merge 1 commit into
apache:mainfrom
zhang-arvin:fix/jdbc-location-overlap-index-catalog-id

Conversation

@zhang-arvin

Copy link
Copy Markdown

The optimized sibling check reads location_without_scheme via QueryGenerator.generateOverlapQuery, which filters realm_id and catalog_id. On Postgres and CockroachDB the idx_locations index led with parent_id instead of catalog_id, so the predicate could only probe the realm_id prefix and every CREATE TABLE / CREATE NAMESPACE with OPTIMIZED_SIBLING_CHECK enabled fell back to a scan instead of the intended indexed lookup. H2 already used the correct columns.

Schema v5 has already shipped in 1.7.0, so rather than editing a released script this adds schema v6 for all backends with the corrected index and bumps the latest schema version. Fresh installs bootstrap at v6 and get the right index automatically. Existing deployments need a one-time manual index recreation, documented in the changelog upgrade notes, because Polaris has no automated schema migrations.

Changes

  • DatabaseType.java: Bump latest schema version from 5 to 6 for all backends
  • schema-v6.sql (postgres/cockroachdb): Fix idx_locations from (realm_id, parent_id, location_without_scheme) to (realm_id, catalog_id, location_without_scheme)
  • schema-v6.sql (h2): No-op version bump to keep H2 in sync (already correct)
  • CHANGELOG.md: Document the fix and upgrade notes for existing deployments
  • SchemaLocationIndexParityTest.java: New test to ensure idx_locations column list stays consistent across backends

Fixes #5301

…ling-check query

The optimized sibling check reads location_without_scheme via
QueryGenerator.generateOverlapQuery, which filters realm_id and
catalog_id. On Postgres and CockroachDB the idx_locations index led
with parent_id instead of catalog_id, so the predicate could only
probe the realm_id prefix and every CREATE TABLE / CREATE NAMESPACE
with OPTIMIZED_SIBLING_CHECK enabled fell back to a scan instead of
the intended indexed lookup. H2 already used the correct columns.

Schema v5 has already shipped in 1.7.0, so rather than editing a
released script this adds schema v6 for all backends with the
corrected index and bumps the latest schema version. Fresh installs
bootstrap at v6 and get the right index automatically. Existing
deployments need a one-time manual index recreation, documented in
the changelog upgrade notes, because Polaris has no automated schema
migrations.
@vigneshio

Copy link
Copy Markdown
Contributor

Thanks @zhang-arvin. This is the same change as #5301, which is already under review. please close this as a duplicate.

if you'd like to help, reviewing #5301 would be the right place..

@zhang-arvin

Copy link
Copy Markdown
Author

Closing as duplicate of #5301. Thanks for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants