Everybody Runs Klaviyo. Reuse Tested It.
At a Glance
| Result | Reuse with Polar |
|---|---|
| Orders lift, new subscribers | +19%In the days after joining the list, before a first purchase |
| Orders lift, existing customers | ~0%No measurable difference after three months of email |
| Klaviyo accounts in one test | 3, pooled into a single readChile, Peru and Mexico |
| What Reuse changed | Fewer calendar campaigns, more time on top flows, lead capture as the growth lever |
We send hundreds of thousands of mails every month with a lot of platform attributed sales but the true causal value is low. Instead, the new customers’ flow that are only some thousands of mails have a strong causal effect.
About Reuse
Reuse is a Latin American marketplace for refurbished electronics: phones, laptops, tablets and consoles, refurbished and resold across Chile, Peru and Mexico. It runs a full Klaviyo programme in each country, campaigns and flows, the way every DTC brand does.
Each country runs its own Shopify store and its own Klaviyo account, which is why the test was built to pool all three into a single read.
The channel nobody audits
Every Shopify brand runs Klaviyo, and nearly every one reports that it drives 20% to 40% of revenue. That is what the benchmarks say a healthy programme should do, and the dashboard confirms it.
The number is last-click attribution. It credits email with any sale it touched on the way, which includes a great many people who were going to buy anyway. It measures what email was present for, not what it caused.
And the channel gets none of the scrutiny paid media gets, because it has no media cost. No auction, no CPM, no daily spend to watch. The real cost is people: a retention manager or an agency, plus the design and copy hours behind every flow and every campaign. That cost scales with how hard you push, and it is time not spent elsewhere.
So email sits in an odd position at most brands: one of the largest revenue claims in the stack, and the least tested.
Why Reuse had reason to doubt it
Read the email benchmarks closely and they all carry the same qualifier: email's share of revenue scales with how often customers come back. The 35% to 45% figures belong to consumables. Apparel, where people buy a few times a year, sits closer to 30%. The benchmark is an average across businesses whose customers return within weeks.
A person who buys a refurbished laptop does not need another one for years.
For a clothing brand, a weekly campaign makes sense: the next purchase is a matter of catching someone at the right moment. For Reuse, that same campaign lands on someone with no need. There was a reasonable case, before any data, that the programme cost more than it returned. Nobody had checked, because nobody checks email.
The causal inference approach
Reuse brought the question to Polar. Most of the incrementality tests we run are geo holdouts on Google, Meta and TikTok: switch a channel off in some regions, keep it on in others, and reconstruct what would have happened. That is the right instrument when you cannot identify who saw the ad.
Email is the opposite. The brand knows exactly who is on the list, so the right design is a proper A/B test at the customer level. Our commitment is to causal rigour, not to one method. When the question calls for a different design, we build it, and we bring the statistics that go with it: here, a randomised holdout pooled across three Klaviyo accounts, with variance-reduction methods on the analysis so that a real effect has the best possible chance of showing itself.
The only way to learn what a channel causes is to take it away from someone and watch.
Geo holdout
Switch a channel off in some regions, keep it on in others, and reconstruct what would have happened. Right when you cannot identify who saw the ad (Google, Meta, TikTok).
Customer-level holdout
Switch email off for specific people. Possible because the brand knows exactly who is on the list, so it becomes a proper A/B test.
Every profile
Existing and newly added, across Chile, Peru and Mexico: three Klaviyo accounts pooled into one test.
Random split
One customer in ten picked at random and removed from every campaign and every flow.
Three months
Everyone else got business as usual.
Measure in Shopify
Orders and revenue matched back to each profile, not Klaviyo's attributed number.
On existing customers: nothing
For everyone who was already on the list when the test began, three months of email made no measurable difference to what they bought. Orders, conversion rate and revenue all landed within noise of the group that received nothing.
That is the audience the campaign calendar exists for, and it is nearly the whole list. The effect on their buying is either zero or too small for a three-month test to see. Either way it is nowhere near what the attributed number implies.
The dashboard says the opposite, and that is worth understanding
Look at the same customers through Klaviyo's engagement metrics and email looks like a success: site visits up 36%, add-to-carts up 39%, with purchases flat. No funnel behaves that way.
| Existing customers | Lift from being emailed | p |
|---|---|---|
| Active on Site (Klaviyo) | +36% | <0.001 |
| Added to Cart (Klaviyo) | +39% | <0.001 |
| Orders (Shopify) | −2% | 0.7 |
| Conversion rate (Shopify) | 0% | >0.9 |
| Revenue (Shopify) | +4% | 0.6 |
The reason is mechanical. Klaviyo only records a visit for a profile it has identified, and clicking an email is the main way it identifies anyone. A visit from an email click carries the visitor's identity with it. The same visit from Google or Instagram usually does not. The emailed group was not more active. It was more visible. Orders carry an email address whichever way the customer arrived, which is why orders show no gap.
Treat engagement metrics as proof the email was delivered and opened. Not as proof it changed anything.
On new subscribers: a fifth more orders
The people who joined the list during the test told a different story.
Most profiles are created at checkout or signup, and the purchase that creates a profile cannot have been caused by an email that had not yet been sent. Set that aside and look at what happens next, in the days after someone arrives and before they have bought, and email lifted orders by 19%.
Profile created
At checkout or signup. When that moment is a purchase, email cannot have caused it: nothing had been sent yet.
Set asideStill deciding
The first emails do the work of reassurance:
- What condition the device is in
- What the warranty covers
- Why buy refurbished rather than new
- Often a first-purchase incentive
orders vs the holdout, counting only purchases after arrival
New subscriber
In the market for a phone or a laptop right now. Email activates that demand.
Existing customer
Bought a laptop last quarter. No need for another one for years.
The factual conclusion is narrow: Klaviyo's efforts work on this segment. Why they work is a hypothesis, and a reasonable one: someone who has just landed on a refurbished-electronics store has not decided yet, and each of those first emails is a nudge toward a first order.
Reuse's own reading sharpens the point. In their words, email activates demand that already exists rather than creating it. One consequence worth being honest about: the holdout was removed from campaigns and flows together, so the test cannot say whether it was the flows alone or the flows plus the calendar campaigns that moved new subscribers. A campaign landing on someone who is in the market may well help. What the test does settle is where the demand is.
What Reuse changed
Not "switch off Klaviyo". The effort was pointed at the wrong half of the list, and the fix is three concrete changes.
Fewer calendar campaigns, to fewer people. The existing-customer campaign calendar takes most of the sends and most of the production hours, and against a real holdout it returns nothing detectable. Reuse is cutting both the number of calendar emails and the size of the audience they go to.
More time on the top flows. Welcome, abandoned cart, retargeting. The new-subscriber journey is a fraction of the volume and carries the whole measurable effect, so that is where the update and testing hours moved. Klaviyo's own A/B tests are worth running there now, because the baseline value of the flows they sit inside has been established.
Treating lead capture as the real lever. If email activates demand that already exists, the constraint is how many in-market people reach the list in the first place. Reuse has started tracking lead-acquisition events in Polar, pop-up signups and click-to-WhatsApp among them, to see which channels bring in new leads and put weight behind those.
Same team, same tool, no extra media spend. Pointed at the part of the programme the test says moves purchases, and at the front door that feeds it.
How sure are we
| Segment | Metric | Lift | 90% CI | p |
|---|---|---|---|---|
| Existing customers | Orders | −2% | [−10%, +7%] | 0.7 |
| Existing customers | Conversion rate | 0% | [−8%, +8%] | >0.9 |
| Existing customers | Revenue | +4% | [−7%, +15%] | 0.6 |
| New subscribers, after arrival | Orders | +19% | [+7%, +31%] | 0.01 |
| New subscribers, after arrival | Profile-level conversion rate | +17% | [+6%, +28%] | 0.01 |
| New subscribers, after arrival | Revenue | +23% | [+7%, +39%] | 0.02 |
Lift is emailed vs holdout, relative to the holdout mean. "After arrival" excludes the purchase that created the profile, which email could not have caused. All estimates are regression-adjusted on each profile's pre-cutoff behaviour (Lin 2013 / multi-covariate CUPED), which we apply by default in A/B designs to maximise the chance of observing an effect if one exists. Two-sided Welch tests; 90% intervals. Figures rounded.
The new-subscriber result is solid. Orders, conversion and revenue all move the same direction at similar size, the three countries agree with each other, and the purchases email could not have caused show no gap between the groups, which confirms the split was clean.
The existing-customer result is a null, with the caveat every null carries: a small positive effect could be hiding in the noise. It is not a large one, and it is not close to the attributed claim.
If you run a Klaviyo programme
The average klaviyo claimed benchmark is not your number. If your customers return every few years, a benchmark built on brands whose customers return every few weeks is describing someone else's business.
Engagement lift is not evidence. Any measurement where one group is easier to observe than the other will manufacture a lift, and in email that asymmetry is built in.
The value is probably not where the volume is. Here the smallest part of the programme carried all of the measurable effect.
The fastest growing Shopify brands use Polar Analytics
