Add native support to the adapter to send workloads to Synapse Spark Pools, in addition to already supported dedicated SQL Pools.
We've heard numerous requests from dbt Cloud users and interested teams that they would like to use dbt with Synpase, not only with dedicated SQL pools, but also Spark Pools, for them to choose the right option depending on the need. It may be cost optimisation, or architectural decisions - but flexibility is greatly desired.
Similar work has been done for dbt-fabric already (See microsoft/dbt-fabricspark). Yet, ideally this could be unified under a single adapter to provide a simpler experience to developers.
Add native support to the adapter to send workloads to Synapse Spark Pools, in addition to already supported dedicated SQL Pools.
We've heard numerous requests from dbt Cloud users and interested teams that they would like to use dbt with Synpase, not only with dedicated SQL pools, but also Spark Pools, for them to choose the right option depending on the need. It may be cost optimisation, or architectural decisions - but flexibility is greatly desired.
Similar work has been done for dbt-fabric already (See microsoft/dbt-fabricspark). Yet, ideally this could be unified under a single adapter to provide a simpler experience to developers.