Skip to content

Improve Table Materialization to Minimize Downtime #265

Description

@gjvanvuuren

As addressed in dbt-fabric issue 189, the fabric adapter's table materialization was changed to:

  1. load the data into a temp table and then
  2. to swap the temp table with the destination table.

This prevents the destination table from being offline, possibly for longer periods, while the data loads.

The dbt-synapse adapter (v1.8.1) still drops the target table and creates a new table during table materialization. This is creating errors, because the target table disappears for minutes, causing other queries and reports to fail.

Would it be possible to merge the above update from dbt-fabric into dbt-synapse?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions