Skip to main content
A/B testing lets you compare different message versions to find what resonates with your audience. Test copy, offers, CTAs, and more. Variants are sometimes referred to as blasts in the API. The UI displays them as “variants” for clarity.

​
How It Works

  1. Create multiple variants (blasts) in your campaign
  2. Assign a percentage of the audience to each
  3. Contacts are randomly assigned to variants
  4. Track performance by variant
  5. Use the winner for future campaigns

​
Setting Up an A/B Test

​
Step 1: Create Campaign

  1. Go to Schedules > New Schedule
  2. Select Blast Mode
  3. Configure sending list and audience

​
Step 2: Add Variants

Click Add Variant for each version you want to test: Each variant is a ScheduleBlast with its own inline message text (using {{linkN}} placeholders), optional variant_name for reporting, and optional mms_image_url for MMS.

​
Step 3: Set Distribution

Adjust percentages to control how audience is split:
  • 50/50: Equal split between two variants
  • 80/20: Mostly variant A, test B on smaller group
  • 33/33/34: Three-way split
Percentages must add up to 100%. The system validates that all variant (blast) percentages sum to exactly 100% before the campaign can be scheduled.

​
Step 4: Schedule and Send

Set your schedule as usual. When the campaign runs:
  • Each contact is randomly assigned to a variant
  • They receive that variant’s message
  • Assignment is tracked for reporting
Each variant’s message must contain at least one {{linkN}} placeholder (e.g., {{link1}}). Messages without a link placeholder will fail validation.

​
What to Test

​
Message Copy

Test different approaches to the same offer:

​
Urgency vs. Value

​
CTA Phrasing

​
Personalization

​
Different Offers

Test which offer drives more conversions:

​
Analyzing Results

After the campaign sends, compare variants:

​
Metrics to Compare

​
Viewing Results

  1. Go to Schedules
  2. Click on your campaign
  3. View Performance by Variant
You’ll see:
Variant B wins on clicks and revenue.

​
Statistical Significance

Automatic statistical significance calculation is not yet available. Before declaring a winner, review results manually:

​
Sample Size

More contacts = more reliable results:

​
Margin of Victory

A 1% difference isn’t meaningful. Look for:
  • Click rate: 2%+ difference
  • Revenue: 10%+ difference
  • Conversions: 5%+ difference
If results are close, the variants perform similarly. Pick the one that’s easier to scale or matches your brand better.

​
Testing Strategy

​
Test One Thing at a Time

❌ Don’t do this:
You won’t know what caused the difference. ✅ Do this:
Now you know if the copy matters.

​
Iterate Based on Learnings

  1. Week 1: Test copy A vs B → B wins
  2. Week 2: Test offer X vs Y (using winning copy) → Y wins
  3. Week 3: Test CTA 1 vs 2 (using B + Y) → 2 wins
  4. Ongoing: Use B + Y + 2 as your champion

​
Keep a Control

Always have a “control” variant you can compare against: This tells you if changes are actually improvements.

​
Multi-Variant Testing

Testing more than 2 variants: More variants require larger audiences for meaningful results. With 4 variants, you need 4x the sample size.

​
Best Practices

Test things that might move the needle significantly: offers, core value prop, urgency. Save minor tweaks for later.
For recurring campaigns, run tests across multiple sends before concluding.
Keep a record of what you tested and results. Build institutional knowledge.
Don’t just test—implement winners. The goal is improvement, not just data.

​
Next Steps

Automated Selection

Let AI pick the best creative

Message Execution

How messages are delivered