From 77daa1d2fcec07467b31c0a18fcec5669df12963 Mon Sep 17 00:00:00 2001 From: Cal Courtney Date: Tue, 14 Jul 2026 13:16:38 +0100 Subject: [PATCH 1/3] docs(events): document the exposure flags bitmask --- .../Events/downloading-events.mdx | 7 ++ .../Events/exposure-events.mdx | 80 ++++++++++++++++++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/web-console-docs/Events/downloading-events.mdx b/docs/web-console-docs/Events/downloading-events.mdx index fdf79eec..df929464 100644 --- a/docs/web-console-docs/Events/downloading-events.mdx +++ b/docs/web-console-docs/Events/downloading-events.mdx @@ -9,6 +9,13 @@ import Image from "../../../src/components/Image" Raw event data can be downloaded directly from an experiment's overview page or from the [Events Page](the-events-page). +:::note Check the `flags` column before you analyse exposures +An export contains every exposure we ingested, not just the real participants. +The [`flags` field](exposure-events#the-flags-field) is what separates the two. +Skip it and your participant counts come out too high, and the same visitor can +end up on more than one variant. +::: + ## From the Experiment's overview Downloading events data directly from an experiment makes it easy to download all experiment related data in a single file directly into your computer. diff --git a/docs/web-console-docs/Events/exposure-events.mdx b/docs/web-console-docs/Events/exposure-events.mdx index a819a2b7..37720709 100644 --- a/docs/web-console-docs/Events/exposure-events.mdx +++ b/docs/web-console-docs/Events/exposure-events.mdx @@ -103,12 +103,84 @@ Below is an overview of some of the data you will find in the exposure event's r | **experiment_id**| The experiment's id. | | **experiment_name**| The experiment's name. | | **variant**| The variant's assignment'. 0 for base. 1 for variant 1. etc. | +| **flags**| A bitmask describing how this exposure was assigned. See [The flags field](#the-flags-field) below. | + +## The flags field + +The `flags` field matters most when you export events and crunch the numbers +yourself. Inside the UI it's already applied, so you rarely have to think about +it. In a raw export you get every exposure we ingested, and `flags` is how you +tell a real participant apart from a bot, an ineligible visitor or an overridden +assignment. + +`flags` is a **bitmask**. Each bit answers one yes/no question about the +exposure, all packed into a single integer, so one event can have several bits +set at once. That means you test it with a bitwise `AND` rather than comparing +the whole number. + +### The bits + +| Bit value | Name | Meaning | +|-----------|------|---------| +| `1` | eligible | The unit was eligible to participate. When this is **0**, traffic allocation was below 100% and this unit fell outside it. | +| `2` | assigned | The assignment logic ran for this exposure. When this is **0**, no variant was actually assigned. | +| `4` | overridden | The assignment was overridden by the SDK. | +| `8` | full on | The experiment was full on (100% on a single variant) when the event was ingested. | +| `16` | custom | The exposure used a custom assignment. | +| `32` | audience mismatch | The unit did not match the experiment's intended audience. | +| `64` | crawler | The unit was detected as a bot or crawler. | +| `128` | ignored | The unit was flagged to be ignored via its unit attributes. Present in exported data but not shown in the UI. | +| `256` | rule override | The variant was determined by a rule override rather than the normal assignment. | + +### Reading a value + +Take `flags = 3`. In binary that's `0000 0011`, so bits `1` and `2` are set: + +```javascript +const eligible = (flags & 1) !== 0; // true +const assigned = (flags & 2) !== 0; // true +const crawler = (flags & 64) !== 0; // false +``` + +That's a clean exposure: the visitor was eligible and got assigned a variant. +Compare it with `flags = 66` (`0100 0010`), where bit `2` (assigned) and bit +`64` (crawler) are set but bit `1` (eligible) isn't. That row is a bot, so it +shouldn't count as a participant. + +### Matching the UI's participant count + +A row counts as an effective participant when it's eligible and assigned and +none of the disqualifying bits are set. You can check all of that in one masked +test, `(flags & 207) = 3`: the mask keeps bits `1 + 2 + 4 + 8 + 64 + 128`, and +`= 3` means eligible (`1`) and assigned (`2`) are the only ones left standing. + +```sql +-- matches the UI's participant count +SELECT count(DISTINCT unit_uid) +FROM exposures +WHERE (flags & 207) = 3 +``` + +The `207` mask throws out overridden (`4`), full on (`8`), crawler (`64`) and +ignored (`128`) rows. Rule overrides get excluded too, just by a different +route: the SDK never marks them as assigned (bit `2`), so they fail the `= 3` +test on their own. Custom (`16`) and audience mismatch (`32`) aren't in the +mask, so on their own they won't drop a row from the count. ## Events info & warning Exposure events can be labelled with extra information or warning -| Label | Type | Description | -|-------|--------------|-------------| -| **An experiment with this name was not running at the time of ingestion** | warning | Indicates that the underlying experiment was not running when this event was triggered . | -| **The experiment was full on at the time of ingestion**| info | The experiment related to this event is full on. It is good practice to clean up full on experiments. | \ No newline at end of file +| Label | Type | Trigger | +|-------|------|---------| +| **An experiment with this name was not running at the time of ingestion** | warning | `experiment_id` empty (not flag-driven) | +| **The unit of this exposure event was not eligible to participate in the experiment at the time of ingestion because traffic allocation for the experiment was < 100** | warning | eligible bit (`1`) not set | +| **The assignment logic did not run for this exposure event** | warning | assigned bit (`2`) not set | +| **The assignment of this exposure was overridden by the SDK** | warning | overridden bit (`4`) set | +| **The experiment was full on at the time of ingestion** | info | full on bit (`8`) set | +| **This exposure has a custom assignment** | info | custom bit (`16`) set | +| **The unit of this exposure event did not match the intended audience** | info | audience mismatch bit (`32`) set | +| **The unit of this exposure event was detected to be a bot/crawler** | info | crawler bit (`64`) set | +| **The variant of this exposure was determined by a rule override** | info | rule override bit (`256`) set | + +The full on label is worth acting on: it's good practice to clean up full on experiments once you're done with them. \ No newline at end of file From fed4c627476a43bf0b8b005770f378912a68367a Mon Sep 17 00:00:00 2001 From: Cal Courtney Date: Tue, 14 Jul 2026 13:26:40 +0100 Subject: [PATCH 2/3] docs(events): scope participant query and export wording --- docs/web-console-docs/Events/downloading-events.mdx | 8 ++++---- docs/web-console-docs/Events/exposure-events.mdx | 8 +++++++- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/web-console-docs/Events/downloading-events.mdx b/docs/web-console-docs/Events/downloading-events.mdx index df929464..a03429dd 100644 --- a/docs/web-console-docs/Events/downloading-events.mdx +++ b/docs/web-console-docs/Events/downloading-events.mdx @@ -10,10 +10,10 @@ import Image from "../../../src/components/Image" Raw event data can be downloaded directly from an experiment's overview page or from the [Events Page](the-events-page). :::note Check the `flags` column before you analyse exposures -An export contains every exposure we ingested, not just the real participants. -The [`flags` field](exposure-events#the-flags-field) is what separates the two. -Skip it and your participant counts come out too high, and the same visitor can -end up on more than one variant. +An export contains every matching exposure within your export scope, not just the +real participants. The [`flags` field](exposure-events#the-flags-field) is what +separates the two. Skip it and your participant counts come out too high, and the +same visitor can end up on more than one variant. ::: ## From the Experiment's overview diff --git a/docs/web-console-docs/Events/exposure-events.mdx b/docs/web-console-docs/Events/exposure-events.mdx index 37720709..d5587cef 100644 --- a/docs/web-console-docs/Events/exposure-events.mdx +++ b/docs/web-console-docs/Events/exposure-events.mdx @@ -155,12 +155,18 @@ test, `(flags & 207) = 3`: the mask keeps bits `1 + 2 + 4 + 8 + 64 + 128`, and `= 3` means eligible (`1`) and assigned (`2`) are the only ones left standing. ```sql --- matches the UI's participant count +-- matches the UI's participant count for one experiment SELECT count(DISTINCT unit_uid) FROM exposures WHERE (flags & 207) = 3 + AND experiment_id = + AND unit_type = '' ``` +Scope the count to a single experiment and unit type, as shown above. A +`unit_uid` is only unique within its `unit_type`, and an export can span many +experiments, so counting across the whole file would merge separate visitors. + The `207` mask throws out overridden (`4`), full on (`8`), crawler (`64`) and ignored (`128`) rows. Rule overrides get excluded too, just by a different route: the SDK never marks them as assigned (bit `2`), so they fail the `= 3` From f52e5abbc990d5d5678cd30a1a4423818d29c1df Mon Sep 17 00:00:00 2001 From: Cal Courtney Date: Tue, 14 Jul 2026 13:44:46 +0100 Subject: [PATCH 3/3] docs(events): distinguish filtered experiment exports from raw exports --- .../Events/downloading-events.mdx | 24 +++++++++++++++---- .../Events/exposure-events.mdx | 9 ++++--- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/docs/web-console-docs/Events/downloading-events.mdx b/docs/web-console-docs/Events/downloading-events.mdx index a03429dd..e2d438f1 100644 --- a/docs/web-console-docs/Events/downloading-events.mdx +++ b/docs/web-console-docs/Events/downloading-events.mdx @@ -9,11 +9,13 @@ import Image from "../../../src/components/Image" Raw event data can be downloaded directly from an experiment's overview page or from the [Events Page](the-events-page). -:::note Check the `flags` column before you analyse exposures -An export contains every matching exposure within your export scope, not just the -real participants. The [`flags` field](exposure-events#the-flags-field) is what -separates the two. Skip it and your participant counts come out too high, and the -same visitor can end up on more than one variant. +:::note The `flags` column depends on which export you use +Experiment-overview exports are already filtered to effective participants, so +you can use them as-is. Events-page exports are raw and still include +non-participants, so you'll need the [`flags` field](exposure-events#the-flags-field) +to filter them. Count a raw export without it and your participant numbers come +out too high, and one visitor can show up on more than one variant. Each section +below spells out which case applies. ::: ## From the Experiment's overview @@ -33,10 +35,22 @@ To download an experiment's raw event data The link to download the file is only valid for 30 days. A new request can be made if the link has expired. ::: +An experiment export is already filtered to effective exposures. Ineligible +visitors, bots, and overridden and rule-override assignments are stripped out for +you, so every row you get is a participant ABsmartly counts. There's no `flags` +mask to apply. + ## From the Event's page Downloading events from the Events page makes it possible to select which events will be downloaded and does not restrict events from a single experiment. +:::warning This export is raw +An events-page export isn't filtered the way an experiment export is, so +non-participants are still in there. Apply the +[`flags` field](exposure-events#the-flags-field) yourself before you count +participants. +::: + ### Exporting Events To set up a new export, click `Export` in the top-right hand corner. diff --git a/docs/web-console-docs/Events/exposure-events.mdx b/docs/web-console-docs/Events/exposure-events.mdx index d5587cef..b6b0b1e0 100644 --- a/docs/web-console-docs/Events/exposure-events.mdx +++ b/docs/web-console-docs/Events/exposure-events.mdx @@ -109,9 +109,12 @@ Below is an overview of some of the data you will find in the exposure event's r The `flags` field matters most when you export events and crunch the numbers yourself. Inside the UI it's already applied, so you rarely have to think about -it. In a raw export you get every exposure we ingested, and `flags` is how you -tell a real participant apart from a bot, an ineligible visitor or an overridden -assignment. +it. A raw [events-page export](downloading-events#from-the-events-page) gives you +every exposure we ingested, and `flags` is how you tell a real participant apart +from a bot, an ineligible visitor or an overridden assignment. An +[experiment export](downloading-events#from-the-experiments-overview) skips this +step, since it's already filtered down to effective exposures, but any query you +run against a raw export or your own warehouse needs it. `flags` is a **bitmask**. Each bit answers one yes/no question about the exposure, all packed into a single integer, so one event can have several bits