Feature Flag Migration Playbook#95
Conversation
- Updated docs/what-is-mixpanel.mdx - Created images/FeatureFlagSetup.png - Created images/ExperimentSetup.png - Created images/ExperimentSetupMetrics.png - Created guides/strategic-playbooks/feature-flag-migration-playbook.mdx - Updated docs.json Mintlify-Source: dashboard-editor
kurbycchua
left a comment
There was a problem hiding this comment.
Overall comment is we need to make sure we fact check these as we do not want a customer ending up in messing up their experiment / FF data in mixpanel. Migrating the exposure of $experiment_started I find it the easy part, the challenging part is the Experiment report and the FF report of which underlying drives the analysis and the gates
- Updated docs/what-is-mixpanel.mdx - Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
| "guides/strategic-playbooks/onboarding-playbook/beyond-onboarding" | ||
| ] | ||
| }, | ||
| "guides/strategic-playbooks/feature-flag-migration-playbook", |
There was a problem hiding this comment.
@amymadden-bit more of a nitpick on this... depending on how we want to position this the actual content has a title of "Migrating Your Feature Flags and Experiments to Mixpanel" but the menu says "Feature Flag Migration Playbook" Also since the content is also covering feature flag that are NOT related to experiments - that's another angle to consider.
There was a problem hiding this comment.
@kurbycchua fair point, it's a bit long but I updated to "Feature Flag & Experiments Migration Playbook" (I think.... I did it in mintlify so hopefully it pulled over)
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
- Updated guides/strategic-playbooks/feature-flag-migration-playbook.mdx Mintlify-Source: dashboard-editor
|
|
||
| ## Step 1. Migrate past experiment exposure events | ||
|
|
||
| An exposure event records that a user was enrolled in an experiment, and was bucketed into an experiment variant. To keep your historical data, export your existing exposure events from your current vendor and ingest them to Mixpanel as `$experiment_started` events. |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit nitpick: This statement "An exposure event records that a user was enrolled in an experiment, and was bucketed into an experiment variant." might imply to readers that exposure events (ie feature gate, dynamic config) not related to experiments need not be migrated.
| sidebarTitle: "Feature Flag & Experiments Migration Playbook" | ||
| --- | ||
|
|
||
| This guide walks through migrating your feature flags and experiments to Mixpanel from another platform. It's for teams currently running flags and experiments on a third-party vendor who want to consolidate that work in Mixpanel instead. |
There was a problem hiding this comment.
@amymadden-bit does this read better: This guide walks through migrating your feature flags and experiments hosted in another platform into Mixpanel.
As Customer Education we should envision to review para / sentences to help easy reading, an idea for a claude skill perhaps?
|
|
||
| ### Export exposure events from your vendor | ||
|
|
||
| Most vendors have a dedicated exposure event you can export. You can usually export them directly from the UI or by using the vendor API. |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit We should state that Mixpanel minimally requires these 4 information for an exposure event?
|
|
||
| #### E**quivalent events** | ||
|
|
||
| Each vendor's exposure event corresponds to a different Mixpanel event depending on whether it came from a feature gate, an experiment, or a dynamic config. Use this table to find the equivalent for your setup: |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit so are we saying here "corresponds to a different Mixpanel event" that there are different exposure events on Mixpanel side? Can be confusing perhaps?
Should we just say "Each vendor has its own definition of exposure event and they can be mapped to Mixpanel's exposure event depending on the Mixpanel Feature Flag type (i.e., feature gate, an experiment, or a dynamic config)
| By the end of this, your flags and experiments will be set up in Mixpanel, your historical results will carry over, and your application code will be evaluating flags through Mixpanel rather than your old provider. | ||
|
|
||
| <Note> | ||
| This guide is a reusable template. The steps below are vendor-agnostic. To migrate from a provider (Statsig, LaunchDarkly, Optimizely, Firebase Remote Config, VWO, GrowthBook, or any OpenFeature setup), substitute that vendor's export API in each "Export" step — the Mixpanel side stays the same. |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit Don't think we covered Firebase Remote Config, VWO although it is mentioned here
|
|
||
| Each vendor's exposure event corresponds to a different Mixpanel event depending on whether it came from a feature gate, an experiment, or a dynamic config. Use this table to find the equivalent for your setup: | ||
|
|
||
| | Mixpanel Feature Flag Type | Statsig Equivalent | LaunchDarkly Equivalent | Optimizely Equivalent | GrowthBook Equivalent | |
There was a problem hiding this comment.
@xinlintanmp In all honesty this table with the exception of Statsig is quite confusing to me, but since unless I try to validate where this is coming from I have no way of knowing their correctness.
Example what are we saying in the following
For LaunchDarkly
- feature event within Experiment=true (streaming); evaluation_events rows with experiment_iteration_id (warehouse)
- feature event (boolean flag eval)
- feature event (multivariate/JSON flag eval)
I use Gemini in an attempt to try to understand what these are saying, linking here for reference:
-Statsig: https://docs.statsig.com/integrations/event_webhook#event-format
-LaunchDarkly: https://launchdarkly.com/docs/integrations/data-export/warehouse-schema-reference#evaluation-events and https://launchdarkly.com/docs/integrations/data-export/schema-reference#feature-events
|
|
||
| Transform each exported exposure row into a Mixpanel `$experiment_started` event and send it through the **Mixpanel Import API**. | ||
|
|
||
| The two required properties are **Experiment name** and **Variant name**. Keep the original exposure timestamp so historical analysis is accurate, and include a unique `$insert_id` so re-runs don't create duplicates. |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit this can be confusing: Under "
Export exposure events from your vendor" we list 4 bullets and here we say "The two required properties are Experiment name and Variant name" which means distinct_id, timestamp, $insert_id can be optional? We need to make sure we are careful with how to word things in a playbook
| "properties": { | ||
| "distinct_id": "alice@example.com", // enrollment ID | ||
| "time": 1709275888, // Timestamp | ||
| "$insert_id": "exp_4821_alice_1709275888", |
There was a problem hiding this comment.
@xinlintanmp maybe a comment like //prevent re-run duplicates
|
|
||
| ### Export configurations from your vendor | ||
|
|
||
| Use your vendor's management API to export each configuration. Ensure you list and fetch definitions for feature gates, experiments, and dynamic configurations. |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit Are we sure that all vendors can provide "management API to export each configuration."
| "status": "enabled", | ||
| "data_group_id": null, | ||
| "serving_method": "client", | ||
| "experiment_id": 12345, |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit We didn't really explain these? My guess is they should be tied to the id returned when creating the experiment? This is why I think it's important to have an engineer review these examples?
- experiment_id
- is_experiment_active
| } | ||
| ``` | ||
|
|
||
| ## Step 3. Continue setup in the Mixpanel UI |
There was a problem hiding this comment.
@xinlintanmp @amymadden-bit I personally feel this section is confusing, we suddenly jump to the UI without linking how those API calls from the previous section are related to this section. Example
- Do I search for the experiment that I created using the API and then do the UI configuration - unclear if I am someone reading this that has no idea yet how MIxpanel UI for experiments + FF works
- This "Once you have imported the $experiment_started events into Mixpanel, you need to create a corresponding Mixpanel Experiment report" Didn't I create an Experiment and FF already using API does this mean I am creating another duplicate?
kurbycchua
left a comment
There was a problem hiding this comment.
See additional comments, I have resolved those that were updated. This is a more detailed read from me and I literally took a longer time trying to push my shoes into someone who is reading this for the first time and trying to figure out how to migrate. It's a good start but I feel it needs more meet to be a playbook
Summary
This update introduces a new guide for migrating feature flags and experiments from other platforms to Mixpanel. It outlines a five-part process to ensure a smooth transition, covering exposure events, configurations, and verification.
Changes
guides/strategic-playbooks/feature-flag-migration-playbook.mdx.images/FeatureFlagSetup.png,images/ExperimentSetup.png, andimages/ExperimentSetupMetrics.png.docs.jsonnavigation structure.Mintlify
10 threads from 1 user in Mintlify