> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracklysms.com/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B Testing

> Test message variants to optimize performance

A/B testing lets you compare different message versions to find what resonates with your audience. Test copy, offers, CTAs, and more.

Variants are sometimes referred to as **blasts** in the API. The UI displays them as "variants" for clarity.

## How It Works

1. Create multiple **variants** (blasts) in your campaign
2. Assign a percentage of the audience to each
3. Contacts are randomly assigned to variants
4. Track performance by variant
5. Use the winner for future campaigns

## Setting Up an A/B Test

### Step 1: Create Campaign

1. Go to **Schedules > New Schedule**
2. Select **Blast Mode**
3. Configure sending list and audience

### Step 2: Add Variants

Click **Add Variant** for each version you want to test:

| Variant | Message                           | Offer      | %   |
| ------- | --------------------------------- | ---------- | --- |
| A       | "Flash sale today! {{link1}}"     | offer\_123 | 50% |
| B       | "Limited time: 50% off {{link1}}" | offer\_123 | 50% |

Each variant is a `ScheduleBlast` with its own inline `message` text (using `{{linkN}}` placeholders), optional `variant_name` for reporting, and optional `mms_image_url` for MMS.

### Step 3: Set Distribution

Adjust percentages to control how audience is split:

* **50/50**: Equal split between two variants
* **80/20**: Mostly variant A, test B on smaller group
* **33/33/34**: Three-way split

Percentages must add up to 100%. The system validates that all variant (blast) percentages sum to exactly 100% before the campaign can be scheduled.

### Step 4: Schedule and Send

Set your schedule as usual. When the campaign runs:

* Each contact is randomly assigned to a variant
* They receive that variant's message
* Assignment is tracked for reporting

<Warning>
  Each variant's message must contain at least one `{{linkN}}` placeholder (e.g., `{{link1}}`). Messages without a link placeholder will fail validation.
</Warning>

## What to Test

### Message Copy

Test different approaches to the same offer:

| Variant | Message                                                   |
| ------- | --------------------------------------------------------- |
| A       | "Flash sale! 50% off everything today only"               |
| B       | "Your exclusive discount: Half off for the next 24 hours" |

### Urgency vs. Value

| Variant | Approach                         |
| ------- | -------------------------------- |
| A       | Urgency: "Only 3 hours left!"    |
| B       | Value: "Save \$50 on your order" |

### CTA Phrasing

| Variant | CTA                     |
| ------- | ----------------------- |
| A       | "Shop now →"            |
| B       | "Claim your discount →" |

### Personalization

| Variant | Message                                          |
| ------- | ------------------------------------------------ |
| A       | "Hey! Check out our sale"                        |
| B       | "Hi {{first_name}}, exclusive sale just for you" |

### Different Offers

Test which offer drives more conversions:

| Variant | Offer               |
| ------- | ------------------- |
| A       | offer\_summer\_sale |
| B       | offer\_clearance    |

## Analyzing Results

After the campaign sends, compare variants:

### Metrics to Compare

| Metric               | What It Tells You                |
| -------------------- | -------------------------------- |
| **Click Rate**       | Which message drives more clicks |
| **Conversion Rate**  | Which drives more sales          |
| **Revenue per Send** | Which generates more revenue     |
| **Opt-out Rate**     | Which causes more unsubscribes   |

### Viewing Results

1. Go to **Schedules**
2. Click on your campaign
3. View **Performance by Variant**

You'll see:

```
Variant A: 5,000 sent | 12% clicked | $2,340 revenue
Variant B: 5,000 sent | 15% clicked | $2,890 revenue
```

Variant B wins on clicks and revenue.

## Statistical Significance

Automatic statistical significance calculation is not yet available. Before declaring a winner, review results manually:

### Sample Size

More contacts = more reliable results:

| Audience Size | Confidence                    |
| ------------- | ----------------------------- |
| \< 1,000      | Low - results may be random   |
| 1,000 - 5,000 | Medium - directional insights |
| 5,000+        | High - reliable conclusions   |

### Margin of Victory

A 1% difference isn't meaningful. Look for:

* Click rate: 2%+ difference
* Revenue: 10%+ difference
* Conversions: 5%+ difference

<Tip>
  If results are close, the variants perform similarly. Pick the one that's easier to scale or matches your brand better.
</Tip>

## Testing Strategy

### Test One Thing at a Time

❌ Don't do this:

```
Variant A: Different copy + different offer + different time
Variant B: Original everything
```

You won't know what caused the difference.

✅ Do this:

```
Variant A: Different copy, same offer
Variant B: Original copy, same offer
```

Now you know if the copy matters.

### Iterate Based on Learnings

1. **Week 1**: Test copy A vs B → B wins
2. **Week 2**: Test offer X vs Y (using winning copy) → Y wins
3. **Week 3**: Test CTA 1 vs 2 (using B + Y) → 2 wins
4. **Ongoing**: Use B + Y + 2 as your champion

### Keep a Control

Always have a "control" variant you can compare against:

| Variant | Purpose                     |
| ------- | --------------------------- |
| Control | Your current best performer |
| Test    | New variant you're testing  |

This tells you if changes are actually improvements.

## Multi-Variant Testing

Testing more than 2 variants:

| Variant | %   | Use Case         |
| ------- | --- | ---------------- |
| A       | 25% | Control          |
| B       | 25% | New copy         |
| C       | 25% | New offer        |
| D       | 25% | New copy + offer |

More variants require larger audiences for meaningful results. With 4 variants, you need 4x the sample size.

## Best Practices

<AccordionGroup>
  <Accordion title="Start with high-impact tests">
    Test things that might move the needle significantly: offers, core value prop, urgency. Save minor tweaks for later.
  </Accordion>

  <Accordion title="Run tests for sufficient time">
    For recurring campaigns, run tests across multiple sends before concluding.
  </Accordion>

  <Accordion title="Document your tests">
    Keep a record of what you tested and results. Build institutional knowledge.
  </Accordion>

  <Accordion title="Act on results">
    Don't just test—implement winners. The goal is improvement, not just data.
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="Automated Selection" icon="robot" href="/guides/creatives/ml-selection">
    Let AI pick the best creative
  </Card>

  <Card title="Message Execution" icon="paper-plane" href="/guides/campaigns/execution">
    How messages are delivered
  </Card>
</CardGroup>