A/B experiments (cadences and pitches)

Experiments compare two alternatives that are more than a single message: two outreach cadences, two sales pitches or two landing page versions. Each participant is assigned to a variant in a stable way, the CRM measures the conversion rate and only names a winner once there is enough sample.

The A/B test inside a campaign compares messages and measures opens and replies. Experiments exist for what does not fit there: comparing two outreach cadences, two sales pitches, two call scripts or two versions of a page. The metric here is conversion, a yes or no that you record when the goal is met.

Creating an experiment

1. In the menu, under Analytics, open Experiments.
2. In the New experiment block, fill in the Name.
3. Pick the type: Pitch, Cadence or Generic. The type is an organizing label, it does not change the math.
4. Fill in the Variant A label and the Variant B label, which are the names that show up in the results.
5. Click Create. The experiment starts out Running.

The screen creates experiments with two variants. Each participant is assigned by a fixed calculation over their record, so the same lead always falls into the same variant and is never counted twice. If you have already picked a winner, every new participant goes straight into it.

Reading the results

Click the experiment in the list on the left and the Results table appears on the right with one row per variant: Participants, Conversions, Rate and Lift. Lift compares the variant against the first variant, which acts as the control, which is why the control itself shows a dash.

• Enough sample: the winner is reliable. This sentence only appears when every variant has at least 20 participants and the best one beats the control.
• Not enough sample yet for a reliable winner. While that sentence is on the screen, the CRM commits to nothing, even if one rate is visibly higher.
• The Apply winner button only appears when there is a computed winner and the experiment has not been decided yet. On clicking it, the experiment moves to Decided.

Testing landing pages

To test two versions of a page outside the CRM, create the experiment here and have each page notify the CRM: a view event when the page loads and a conversion event when the form is submitted. The call uses the public experiment tracking address with your workspace's lead capture token. The results land in the same table, with the same winner rule.

POST /api/public/experiment-track
{
  "token": "<your workspace lead capture token>",
  "experimentId": "<experiment id>",
  "variantKey": "a",
  "subjectRecordId": "<visitor identifier>",
  "event": "view"
}

Limits and permissions

• The name accepts up to 200 characters. The screen creates experiments with two variants.
• The minimum sample is 20 participants per variant and it is fixed: there is no field to change it.
• You need access to the campaigns module to open the screen. On per-user plans, the A/B testing feature has to be included in the edition you contracted.
• Conversion is not guessed by the CRM: somebody has to record that the participant converted. That is what keeps the number auditable instead of invented.

Common questions

• How is this different from the campaign's A/B test? The campaign one compares message copy and measures opens and replies automatically. An Experiment compares anything and measures a conversion that you declare.
• Can a participant switch variants? No. The assignment is stored and kept, even if the experiment changes later.
• What happens after I apply the winner? The experiment becomes Decided and every new participant starts getting the winning variant.
• Why is the list empty? Either no experiment has been created yet, or your role has no access to the campaigns module.

Open this article inside the system →