Skip to content

docs(warehouse-native): document Fully & Hybrid modes and Data Sources (FT-2068) - #280

Merged
joalves merged 4 commits into
masterfrom
feat/FT-2068/warehouse-native-docs
Jul 20, 2026
Merged

docs(warehouse-native): document Fully & Hybrid modes and Data Sources (FT-2068)#280
joalves merged 4 commits into
masterfrom
feat/FT-2068/warehouse-native-docs

Conversation

@joalves

@joalves joalves commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Documents the two Warehouse Native operating modes now supported in ABsmartly — Hybrid and Fully — and rounds out the Data Sources configuration docs. JIRA: FT-2068.

The two modes are framed around one rule that drives everything: exposures come only from the default data source.

  • Hybrid Warehouse Native — ABsmartly's managed ClickHouse stays the default data source (it handles assignment + exposures), and you attach external warehouses for goals & metrics. Exposures are joined into the external warehouse through the Parquet exposures-import bridge.
  • Fully Warehouse Native — your own warehouse is the default data source, so exposures, goals, and attributes all live in your warehouse and nothing user-level leaves your infrastructure. Because exposures come from the default source, Fully is what unlocks exposure-side robot/scraper filtering and external attribute enrichment — the main reasons to choose it over Hybrid.

Changes

  • New page warehouse-native/modes.mdx — Hybrid vs Fully, the default-data-source concept, a comparison table, and when-to-choose guidance.
  • overview.mdx — introduces the two modes and links to the new page.
  • get-started.mdx — adds the default-data-source note to Step 1, scopes table mapping to default vs external sources in Step 2, and adds a conditional Step 4 documenting the Hybrid exposures-import (Parquet) bridge.
  • New screenshot static/img/warehouse-native/datasources-list.png — the Data Sources list showing the managed ClickHouse marked Default alongside an external BigQuery source (illustrates the Hybrid setup).
  • .gitignore — ignore /.worktrees and local dev/screenshot scratch.

Screenshot — Data Sources list (new)

The managed ClickHouse holds the Default badge (protected actions greyed out); an external BigQuery data source sits alongside it — the Hybrid pattern.

Data Sources list with the managed ClickHouse marked Default alongside an external BigQuery data source

Verification

  • CodeRabbit review (high effort): 3 valid findings applied; 1 false positive skipped (it claimed Snowflake object-store URLs need gcs://, but the app's DataSourceForm.tsx uses gs:// — docs match the code). Clean re-review afterward (0 findings).
  • Docusaurus dev server compiles all three pages; internal cross-page anchors resolve and the new screenshot loads. (The repo's pre-existing cytoscape/mermaid build error is unrelated to these pages.)

Notes for reviewers

  • "Fully Warehouse Native" / "Hybrid Warehouse Native" are product names for the default-vs-external data-source architecture, not literal in-app toggle labels — the docs present them as such.
  • No Application- or Unit-type-level data-source attachment is documented because none exists; association is at the Goal level. The experiment-level dataSourceId override (no confirmed UI) is intentionally left out.

Summary by CodeRabbit

  • Documentation
    • Added guidance explaining Hybrid vs Fully Warehouse Native modes, including responsibilities and how exposures, goals, and metrics are handled.
    • Expanded the “Get Started” guide with default data source details, improved table-mapping notes, and added Hybrid-only steps for configuring Exposures import.
    • Introduced a new modes page with a comparison and mode-selection guidance, and updated the Warehouse Native overview and “next steps” wording.
    • Updated .gitignore to ignore additional local development output directories (/worktrees, /playwright-shots, /docs/superpowers).

…s (FT-2068)

Add a Warehouse Native Modes page explaining Hybrid (managed ClickHouse
stays the default data source) vs Fully (customer warehouse is the default),
centered on the rule that exposures come only from the default data source —
so exposure-side robot/scraper filtering and external attribute enrichment
require Fully mode.

Update the overview and get-started pages to introduce the two modes, explain
the default data source concept, scope table mapping to default vs external
data sources, and add a conditional Step 4 for the Hybrid exposures-import
(Parquet) bridge. Add a Data Sources list screenshot showing the Default badge.
@netlify

netlify Bot commented Jul 18, 2026

Copy link
Copy Markdown

Deploy Preview for absmartly-docs ready!

Name Link
🔨 Latest commit f3cb2c4
🔍 Latest deploy log https://app.netlify.com/projects/absmartly-docs/deploys/6a5ba83c41815d00074f6d9c
😎 Deploy Preview https://deploy-preview-280--absmartly-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1d06ab67-183c-469b-b30c-55dcd77c16c0

📥 Commits

Reviewing files that changed from the base of the PR and between 24beec4 and f3cb2c4.

📒 Files selected for processing (2)
  • docs/web-console-docs/warehouse-native/get-started.mdx
  • docs/web-console-docs/warehouse-native/modes.mdx

Walkthrough

The PR adds Warehouse Native mode documentation, updates overview and setup guidance, documents default data source behaviour and Hybrid exposure imports, and adds ignore rules for local worktrees and scratch outputs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: bmsilva, calthejuggler, marcio-absmartly

Poem

A rabbit hops through modes so bright,
Parquet carries exposures light.
Setup steps bloom, clear and neat,
While scratch paths hide beneath my feet.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main documentation updates around Hybrid/Fully Warehouse Native modes and Data Sources.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/FT-2068/warehouse-native-docs

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/web-console-docs/warehouse-native/get-started.mdx`:
- Around line 200-201: Update the “Object store URL” table row to include the
accepted object-store URL scheme examples for ClickHouse and Databricks,
alongside the existing BigQuery, Redshift, and Snowflake entries, so all
supported warehouses are covered.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ddd1b333-fab5-4937-b105-29a97c5c4259

📥 Commits

Reviewing files that changed from the base of the PR and between b487493 and 5c8f2b0.

⛔ Files ignored due to path filters (1)
  • static/img/warehouse-native/datasources-list.png is excluded by !**/*.png
📒 Files selected for processing (4)
  • .gitignore
  • docs/web-console-docs/warehouse-native/get-started.mdx
  • docs/web-console-docs/warehouse-native/modes.mdx
  • docs/web-console-docs/warehouse-native/overview.mdx

Comment on lines +200 to +201
| **Object store URL** | The bucket/prefix where exposures are exported. The accepted scheme depends on your warehouse (for example, `gs://` for BigQuery, `s3://` for Redshift, and `s3://`, `gs://`, or `azure://` for Snowflake). |
| **Credentials** | The access needed to write and read those objects — an access key/secret for BigQuery and ClickHouse, a stage name for Snowflake, an IAM role ARN for Redshift. Databricks needs no extra credentials. |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Object store URL schemes omit ClickHouse and Databricks.

The Credentials row covers all five supported warehouses, but the Object store URL row only lists schemes for BigQuery, Redshift, and Snowflake. A user configuring an external ClickHouse or Databricks data source for Hybrid exposures import won't know which URL scheme to enter. Add scheme examples for the remaining two warehouses for parity.

📝 Suggested addition
 | **Object store URL** | The bucket/prefix where exposures are exported. The accepted scheme depends on your warehouse (for example, `gs://` for BigQuery, `s3://` for Redshift, `s3://`, `gs://`, or `azure://` for Snowflake, `s3://` or `gs://` for ClickHouse, and `s3://`, `gs://`, or `azure://` for Databricks). |
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
| **Object store URL** | The bucket/prefix where exposures are exported. The accepted scheme depends on your warehouse (for example, `gs://` for BigQuery, `s3://` for Redshift, and `s3://`, `gs://`, or `azure://` for Snowflake). |
| **Credentials** | The access needed to write and read those objects — an access key/secret for BigQuery and ClickHouse, a stage name for Snowflake, an IAM role ARN for Redshift. Databricks needs no extra credentials. |
| **Object store URL** | The bucket/prefix where exposures are exported. The accepted scheme depends on your warehouse (for example, `gs://` for BigQuery, `s3://` for Redshift, `s3://`, `gs://`, or `azure://` for Snowflake, `s3://` or `gs://` for ClickHouse, and `s3://`, `gs://`, or `azure://` for Databricks). |
| **Credentials** | The access needed to write and read those objects — an access key/secret for BigQuery and ClickHouse, a stage name for Snowflake, an IAM role ARN for Redshift. Databricks needs no extra credentials. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/web-console-docs/warehouse-native/get-started.mdx` around lines 200 -
201, Update the “Object store URL” table row to include the accepted
object-store URL scheme examples for ClickHouse and Databricks, alongside the
existing BigQuery, Redshift, and Snowflake entries, so all supported warehouses
are covered.

joalves added 3 commits July 18, 2026 16:59
…2068)

In Hybrid mode, goals are assigned a data source per goal — they can come
from an external warehouse OR from ABsmartly's managed ClickHouse, and
customers can mix both. Previous wording implied all goals/metrics are
computed against the external warehouse.
A goal *can* live in an external warehouse in Hybrid mode — it isn't required.
Exposures wording is unchanged (they always come from the default data source).
…posures import (FT-2068)

Describe the Exposures import object store in user-facing terms (a storage
location you provide that your warehouse reads from) rather than exposing the
internal Parquet export/import mechanism.
@joalves
joalves merged commit 1c5c1ee into master Jul 20, 2026
6 checks passed
@joalves
joalves deleted the feat/FT-2068/warehouse-native-docs branch July 20, 2026 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant