Reading results
What every number on Results means: key metrics for any two groups, monthly estimates, chance to be best, rolling out a group, Claude summaries and charts.
On this page
Open a running test and you land on Results. It updates as visitors and orders come in.
The page reads top-down: the answer (Where it stands and the totals), then the filters (what decides the winner, the period and segments), then one table comparing every version with the original, then More detail (folded): key metrics for any two groups, the per-metric charts, the funnel, breakdowns and every number.
Where it stands
The top card shows the test's stage and, once there is one, the call:
- Keep running or Results moved, still settling: not enough evidence yet. The card explains what it's waiting for.
- Settled, confirming: the result has stopped moving and Crovio is making sure it holds.
- Variant A is likely better, Control is likely better or No meaningful difference: the call. It's safe to end the test.
It also shows the best variant, its lift on the primary goal and its chance it's better than the original, and a progress bar for each stage. How Crovio decides a winner.
Key metrics
To compare two groups, open More detail and use the Key metrics card. It compares two groups at a time on the metrics you picked on the Goals tab, in that order with the primary goal first (click Edit to change them). Besides conversion, revenue and order value, you can add profit per visitor and per order (price tests), orders per visitor, units per order, abandoned cart and checkout rates, checkout completion, free-shipping orders, shipping paid per order, discount per order, discounted orders and upsell attach rate (checkout tests). Pick the groups under Compare; the swap button flips them. It starts on control against the group in the lead. For abandon rates, lower is better; for discount and shipping metrics, neither direction is good on its own, so they aren't coloured green or red.
Tested products only
The Goals tab lets you count only orders with a tested product (which orders count): always on price and offer tests (the default for price tests), and on template and content tests that run on a list of exact product or collection pages. Results then show a Tested products only chip, and every number on the page (conversion, revenue, profit, the verdict) counts only orders that hold a tested product or its test copy. Click the chip's × to count all orders again; nothing is lost either way.
Extras per test type
- Price tests: a Profit table (profit per visitor and per order, from each product's cost per item in Shopify) and a Products table with each tested product's orders, units, revenue and profit per group. Copies count as the product they copy. Products without a cost per item are flagged, since their sales count in full as profit.
- Offer tests: Discounts and order size: average order value, units per order, discount per order and the share of discounted orders.
- Checkout tests: Checkout and upsells: checkout rate, checkout completion (began checkout, then ordered), abandoned checkout rate and upsell attach rate (orders with a product added from a Crovio checkout upsell).
- Shipping tests: the Shipping table (free-shipping orders, shipping paid per order, method chosen).
- Split URL, template and theme tests: the breakdown opens on landing page, so you can see how each entry page did.
Each metric shows:
- The lift of the second group over the first, or No change when the likely range still includes zero.
- Likely between … and …: where the true difference most likely lies (the 95% range).
- An estimated monthly difference ("≈ $120 more a month"): what the lift would add (or cost) in a month at today's traffic, if all of this test's visitors saw the second group. It's an estimate, not a promise.
- A bar per group and the chance the second group beats the first.
The primary goal's card has a trophy. Average order value needs 10 orders per group before it compares.
How each version compares
This table puts every group on one metric (pick it from the menu; it starts on the primary goal):
| Column | What it means |
|---|---|
| Difference | The lift against the original, and where it likely lies (the 95% range) |
| Chance it's better | How likely this group beats the original on this metric |
| Chance it's best | How likely this group is the best of all groups at once, simulated from each group's numbers. With more than one variant, it's the one to watch |
| Roll out / Keep original | Roll out ends the test and shows that group to every visitor it targeted. On the original's row, Keep original ends the test and keeps your store as it is |
Click Roll out (or Keep original), then confirm (Show Variant A to all or End test). It's there while the test runs, is paused, or has ended without a winner. Shipping rate tests don't roll out: set the winning rates in Shopify.
Summary with Claude
Summarize with Claude opens Claude with a snapshot of the results: a plain-words read of what's ahead, how sure it is and what to do next. With the Claude connector, Claude reads the live results through Crovio.
The metrics
| Metric | What it means |
|---|---|
| Conversion rate | Share of visitors who placed an order |
| Revenue per visitor | Order revenue divided by visitors: conversion and order size in one number. Range and win chance appear once every group has 10 orders |
| Avg. order value | Revenue divided by orders. Range and win chance appear once every group has 10 orders |
| Add-to-cart rate | Share of visitors who added anything to the cart |
| Checkout rate | Share of visitors who started checkout |
Switch metric with the tabs above the table. Each variant shows its lift vs control with a 95% interval. If the interval bar crosses 0, the difference may be noise.
Chance it's better (the tests list calls it Win chance) is the probability that a group beats the original. Crovio never shows certainty: it caps at ">99%" and "<1%". The range and the win chance come from the same maths, so a 95% range entirely above zero always goes with a win chance of 97.5% or more.
Revenue metrics wait for 10 orders per group. With fewer, one large basket can swing revenue per visitor by thousands of percent, so the card shows Range and win chance need 10 orders per group instead.
Charts
- Lift: cumulative lift over time. Each line starts once its group has 50 visitors and settles as data builds up.
- Chance it's better: how sure Crovio was, day by day. Wait for it to stay above 95% (or below 5%).
- Visitors per day: new visitors entering each group.
Funnel, goals and events
- Funnel: the share of each group's visitors who added to cart, reached checkout and ordered.
- Custom metrics: the share of visitors who reached each custom metric.
- Custom events: the share who fired each custom event.
- Break down by: device, browser, country, traffic source and more. Segments and filters.
Click Export CSV for all numbers.
How the numbers are counted
- A visitor is counted once, the first time they see the test.
- Orders, revenue and goals only count visitors who saw the test.
- Conversion rate counts visitors who ordered at least once. Revenue uses order subtotals in your store's currency.
- Preview visits are never counted.
Orders and visitors left out
When results leave some orders or visitors out, a Left out of results line above the verdict says how many. Click it for one line per reason:
| Line | What it means |
|---|---|
| Orders in another currency | Paid in a currency other than your store's. They count as conversions; revenue adds them once Shopify sends their value in your store currency. |
| Orders without the buyer's group | Placed through a checkout that didn't carry the group (Buy it now, another app's cart or checkout link), so the buyer saw the original checkout. Not counted. |
| Orders with another group's prices (or version, or shipping) | The order was placed with a different group's change than the buyer's, for example a shared cart link. Not counted for either group. |
| Cancelled orders | Cancelled after they were placed. Not counted in conversions or revenue. |
| Test orders | Placed with a test payment gateway (Bogus, test mode). Never counted. |
| Visitors who saw another group later | The same visitor was shown another group's version later (cleared cookies, signed in on another device). Counted only for the group they saw first. If this keeps growing, check the test in QA. |
Two problems show as their own red banners instead, because they make the numbers unreliable:
- A group's template is missing: the live theme doesn't have that group's template (deleted, or another theme published). Its visitors see the default page and aren't counted. Add the template back or end the test.
- The traffic split looks broken: see below.
Was this page helpful?