Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
### New Features
- Added AWS KMS decryption-key access through the `decryptionKeys` catalog storage configuration property.
- Added Kafka PolarisEventListener for publishing events to Kafka.
- Added a Table Metrics Reports REST API (beta) for querying persisted Iceberg scan and commit metrics reports. The OpenAPI spec and query SPI ship in the `polaris-extensions-metrics-reports*` extension modules; the HTTP endpoint returns HTTP 501 unless a durable query backend (e.g. the JDBC extension) is installed. Reads are gated by the new `TABLE_READ_METRICS` privilege.
- Added GCS principal attribution for vended credentials (the GCP counterpart of AWS STS session tags). Set `GCS_PRINCIPAL_ATTRIBUTION_ENABLED=true` to activate; the feature flags `GCS_PRINCIPAL_ATTRIBUTION_WIF_AUDIENCE`, `GCS_PRINCIPAL_ATTRIBUTION_TOKEN_ISSUER`, and `GCS_PRINCIPAL_ATTRIBUTION_SIGNING_KEY_FILE` are then required (a missing value is a fatal configuration error). Also requires a `gcpServiceAccount` on the catalog StorageConfiguration. When enabled, credential vending chains a catalog-signed JWT through a Workload Identity Federation token exchange and service-account impersonation, so the Polaris principal appears in GCS Data Access audit logs (`serviceAccountDelegationInfo.principalSubject`) for any client. `GCS_PRINCIPAL_ATTRIBUTION_SIGNING_KEY_ID` sets the JWT `kid` for JWKS key rotation. Attribution is keyed per-principal in the credential cache; when disabled (default), GCP vending behaviour is unchanged.
- Added the `DEFAULT_UNIQUE_TABLE_LOCATION_ENABLED` feature flag (off by default). When enabled, a managed location generated for a table or view created without an explicit location is given a unique, unpredictable suffix, so that no two tables share a path prefix.
- Added the `ALLOW_CLIENT_SPECIFIED_TABLE_LOCATION` feature flag (on by default). When set to false, a caller-specified location (the `location` field, a `SetLocation` update, or the `write.data.path` / `write.metadata.path` properties) on a create-table (including a staged create-table request), create-view, update-table, replace-view, or commit-transaction request is rejected, forcing Polaris to manage all locations. Federated catalogs, committing an already staged create, and `register table` / `register view` are unaffected.
Expand Down
2 changes: 2 additions & 0 deletions bom/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ dependencies {
api(project(":polaris-api-management-model"))
api(project(":polaris-api-management-service"))
api(project(":polaris-api-openlineage-service"))
api(project(":polaris-extensions-metrics-reports-api"))

api(project(":polaris-container-spec-helper"))
api(project(":polaris-floci-aws-testcontainer"))
Expand Down Expand Up @@ -102,6 +103,7 @@ dependencies {
api(project(":polaris-extensions-federation-bigquery"))
api(project(":polaris-extensions-federation-hadoop"))
api(project(":polaris-extensions-federation-hive"))
api(project(":polaris-extensions-metrics-reports-spi"))
api(project(":polaris-extensions-metrics-reports"))
api(project(":polaris-extensions-metrics-reports-jdbc"))
api(project(":polaris-extensions-openlineage"))
Expand Down
18 changes: 17 additions & 1 deletion extensions/auth/ranger/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,20 @@ polaris.authorization.ranger.authz.audit.destination.solr.urls=http://solr-servi

```

3. Run or restart Polaris to see that all accesses are authorized by Ranger policies, with access audit records available in Apache Ranger console.
3. Register the Polaris service type with Ranger Admin using the service definition shipped at
`src/main/resources/polaris-ranger-servicedef.json` (packaged as `polaris-ranger-servicedef.json`
on the classpath of `polaris-extensions-auth-ranger`), for example:
```
curl -u <ranger-admin-user>:<ranger-admin-password> -X POST \
-H "Content-Type: application/json" \
-d @polaris-ranger-servicedef.json \
http://ranger-admin:6080/service/public/v2/api/servicedef
```
This is the same `serviceDef` exercised by `RangerPolarisAuthorizerTest`
(`RangerServiceDefConsistencyTest` fails the build if the two drift apart), so any access type
available to grant through Ranger policies is guaranteed to be understood by the authorizer.

4. Create a Ranger service instance of type `polaris` (matching the `service-name` configured
above), then define policies against it.

5. Run or restart Polaris to see that all accesses are authorized by Ranger policies, with access audit records available in Apache Ranger console.
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@
"table-drop",
"table-data-read",
"table-data-write",
"table-metrics-read",
"table-properties-read",
"table-properties-write",
"table-properties-set",
Expand Down Expand Up @@ -338,6 +339,7 @@
"table-create",
"table-drop",
"table-list",
"table-metrics-read",
"table-properties-read",
"table-properties-write",
"table-properties-set",
Expand Down Expand Up @@ -423,6 +425,7 @@
"table-statistics-remove"
]
},
{ "itemId": 70, "name": "table-metrics-read", "label": "Table Metrics Read", "category": "READ", "impliedGrants": [ "table-list" ] },

{ "itemId": 56, "name": "view-create", "label": "View Create", "category": "CREATE", "impliedGrants": [ "view-list" ] },
{ "itemId": 57, "name": "view-drop", "label": "View Drop", "category": "DELETE" },
Expand Down Expand Up @@ -586,7 +589,13 @@
{ "type": "table-statistics-remove" },
{ "type": "table-structure-manage" }
],
"users": [ "admin1" ] }
"users": [ "admin1" ] },
{
"accesses": [
{ "type": "table-data-read" },
{ "type": "table-data-write" }
],
"users": [ "dataonly1" ] }
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ enum ResolvedPathRooting {
private static final String TABLE_WRITE_PROPERTIES = "table-properties-write";
private static final String TABLE_READ_DATA = "table-data-read";
private static final String TABLE_WRITE_DATA = "table-data-write";
private static final String TABLE_READ_METRICS = "table-metrics-read";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LIST_TABLE_METRICS now requests table-metrics-read, but Polaris pins Ranger 2.9.0 and neither its Polaris service definition nor Ranger master defines that access type. Ranger validates policy access types against the service definition, while this PR adds it only to embedded test fixtures and does not add it to the table resource’s accessTypeRestrictions. Operators therefore cannot grant the new permission through the shipped definition. Could we provide an operator-consumable service-definition update and test LIST_TABLE_METRICS against that exact artifact?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 32d7bf7351d (fix(metrics): ship operator-consumable Ranger service def, close data-read/metrics-read gap):

  • Extracted the serviceDef into extensions/auth/ranger/src/main/resources/polaris-ranger-servicedef.json, the artifact operators register with Ranger Admin (README updated with the registration steps).
  • Added table-metrics-read to the table resource's accessTypeRestrictions, which was indeed missing.
  • Added RangerServiceDefConsistencyTest, which fails the build if that shipped artifact ever drifts from the serviceDef in authz_tests/dev_polaris.json — the same fixture RangerPolarisAuthorizerTest exercises for LIST_TABLE_METRICS and the rest of the authz suite. So the operator-facing service definition and the tested one are now provably the same artifact.

private static final String TABLE_ATTACH_POLICY = "table-policy-attach";
private static final String TABLE_DETACH_POLICY = "table-policy-detach";
private static final String TABLE_ASSIGN_UUID = "table-uuid-assign";
Expand Down Expand Up @@ -254,6 +255,10 @@ enum ResolvedPathRooting {
PolarisAuthorizableOperation.REPORT_READ_METRICS,
new RangerPolarisOperationSemantics(
toSet(TABLE_READ_DATA), null, ResolvedPathRooting.ROOT));
RBAC_SEMANTICS_BY_OPERATION.put(
PolarisAuthorizableOperation.LIST_TABLE_METRICS,
new RangerPolarisOperationSemantics(
toSet(TABLE_READ_METRICS), null, ResolvedPathRooting.ROOT));
RBAC_SEMANTICS_BY_OPERATION.put(
PolarisAuthorizableOperation.REPORT_WRITE_METRICS,
new RangerPolarisOperationSemantics(
Expand Down
Loading