Skip to content

Planning for Black Friday? See what a referral program could add to your sales.

Calculate my ROI

A/B testing

A/B test your referral rewards instead of guessing

Run your current reward against one variation, split new participants roughly 50/50, and let conversions decide. When you end the test, BLOOP applies the winner to the live campaign.

What it is

A/B testing, explained

A reward too small never spreads, and one too generous quietly eats margin, and from the outside they look the same. An A/B test on a campaign compares two reward setups side by side. Control (A) is your current rewards, and Variation (B) starts as a copy you edit to test one idea.

You can change the reward value, type, the give-and-get split, the minimum requirement or the expiry on the referrer and referee side. Reward settings are locked while a test runs, so the result is not spoiled, and there is one active test per campaign.

For your customers

How it works on your store

  1. 01

    A new referrer joins

    BLOOP assigns them to Control or Variation, roughly 50/50. They keep that assignment for the whole test.

  2. 02

    They see their side's reward

    Each referrer gets the reward setup for their group. Variation B can also carry its own display text, so wording can differ as well as value.

  3. 03

    Friends buy

    BLOOP records invites, store visits, coupon claims, orders and revenue for each side.

  4. 04

    You read the results

    BLOOP shows the chance that Variation B beats Control A, a 95% range for the gap and the risk of switching. The read-out starts after 100 customers per variant and 7 days.

  5. 05

    You end the test and pick a winner

    The winner's settings are applied to the campaign automatically.

For you

Set it up in BLOOP

Steps follow the BLOOP user guide. Most stores finish setup in about 30 minutes.

  1. 01. Open A/B Testing on the campaign

    Go to Referral program, open the campaign, then Overview, and choose View details in the A/B Testing section.

    Read the guide →
  2. 02. Create and name the test

    Click Create test and choose a test length of 2, 4 or 8 weeks. BLOOP estimates the smallest difference your recent traffic can detect. The audience split is fixed at 50/50.

    Read the guide →
  3. 03. Edit the Variation column

    In the variation settings, use Edit reward on the Variation column and change exactly one thing.

    Read the guide →
  4. 04. Start the test and leave it alone

    Click Start test, then wait until both sides have real volume. Settings are read-only meanwhile.

    Read the guide →
  5. 05. Read results and end the test

    Choose a metric, then end the test and pick Control A or Variation B. BLOOP emails you when the result is ready, and a test that runs 14 days past its end date ends automatically and keeps Control A. An ended test cannot be restarted.

    Read the guide →

Options you control

Make it fit your store

What you can test

Reward value, reward type, the split between referrer and referee, minimum requirement and expiry.

Ten result metrics

Customers with a referral order and customers who shared (the two main metrics, counted per customer), referral revenue, invites, store visits, coupon claims, referral orders, orders per invite, reward cost and revenue after rewards.

Cost is shown, not judged

Reward cost sits under its own heading and never picks the winner, so you can see when a variant wins on referrals but gives away too much.

Wording as well as value

Variation B keeps its own reward display text, so you can test how an offer reads.

Exclusions

Referrers who were already active before the test started are excluded entirely.

Guides

Go deeper

FAQ

A/B testing: common questions

How long should a test run?

Until both variants have enough conversions to trust the gap. That depends on volume, not the calendar, and early leads often swap back and forth.

Which metric should I judge on?

Start with customers with a referral order and customers who shared, which are counted per customer. Then check reward cost and revenue after rewards, because a variant can win on referrals and still lose money if it gives away more per order.

Can I test while using FlexiTiers?

No. Tiers and A/B testing are mutually exclusive, so disable one to use the other.

Is it included in the free plan?

Yes. Advanced A/B testing is included in the Free plan, which covers up to $400 per month in referral revenue.

More referral program features

Turn Your Customers Into Your Best Marketers

Boost revenue, cut CAC, and build loyalty with BLOOP’s all-in-one referral & affiliate solution for Shopify.