Skip to content

Test types

Content, code, template, theme, split URL, price, offer, shipping and checkout tests, and when to use each one.

On this page

When you click New test, Crovio asks what you want to test. Each type is a card with a short description and what it's best for; filter them by Content, Pricing, Offers, Shipping or Checkout. The draft then opens in that type's editor. While a test is a draft you can change its type from the header; it can't change after launch.

TypeUse it toPlan
Content editChange text, images and sections with the visual editorAny
Code (CSS & JS)Write your own CSS and JavaScript per groupAny
Split URLSend each group to a different pageAny
TemplateShow each group a different theme template on the same pagesAny
ThemeShow each group a different theme, with no Shopify preview barAny
PriceTest real prices on chosen products, charged at checkoutAny
OfferTest volume discounts, spend thresholds, product discounts, gifts and buy X get YAny
Shipping ratesTest real checkout rates and free-shipping thresholdsCarrier-calculated shipping
Free shipping / Shipping discountTest a free or cheaper shipping offer over an amountAny
CheckoutTest blocks, upsells and payment or shipping methods in checkoutPlus for blocks inside checkout steps

Choosing between similar types

  • Template or split URL? The difference is the address. A template test keeps shoppers on the same page and swaps its theme template (variant groups get ?view= added), so it suits a new layout for product, collection or home pages. A split URL test sends each group to another page, so it suits two landing pages or a page that lives at its own URL.
  • Price or offer? A price test changes the price itself, everywhere it shows. An offer test keeps your prices and adds an automatic offer, gift or tiered deal on top.
  • Shipping rates or free shipping? A shipping rates test replaces your checkout rates, so it can raise a free-shipping threshold or test new rates. Free shipping and shipping discount tests only make shipping cheaper over an amount, and work on every plan.
  • Template or theme? A template test changes one page type inside your live theme. A theme test switches the whole store to another theme for variant groups: use it for a redesign or a new theme.

Content

Change the page with the visual editor, CSS or JavaScript: headlines, images, buttons, layout, new sections. This is the most common test. Content tests run on the pages you choose in Targeting.

Template

Show each group a different theme template on the same pages, for example a redesigned product page against your current one. Pick the page type (product, home, collection, page and more), the control template and a template for each variant. Shoppers stay on the same page (variant groups get ?view= added to the address) and never see Shopify's preview bar. Run it on every page that uses the control template, or only on chosen pages such as one product. Template tests.

Theme

Show each variant group a different theme: your live theme is control and each variant picks an unpublished theme, with a live preview of each. Variant shoppers browse the whole store on their theme. The address stays the same and Shopify's preview bar is hidden. Turn on the Crovio app embed in every tested theme (the QA checklist checks it), and keep the test themes for 60 days after the test ends so returning visitors can be sent back to your live theme. Checkout is the same for every group. Theme tests.

Split URL

Send each group to a different page URL, for example two landing pages or an old and a new product page. Simple mode pairs each original page with the page each variant sees; Advanced mode uses a URL rule to pick the pages and sends variants to one URL or the same URL with some text replaced. UTM tags are kept and search engine crawlers are never redirected. Split URL tests.

Price

Show each group its own prices for chosen products, from product page to checkout. Set a price and a compare-at price per group in a grid (or with a formula), tag where prices show in your theme once, and launch. Crovio makes a hidden copy of each product per group so checkout charges the price shoppers saw. Price tests.

Offer

Give each group its own automatic offer: a volume discount, an order discount (spend thresholds), a product discount, a free gift or buy X get Y. Any tier can add free shipping. Control gets no offer unless you choose to compare two offers. Crovio creates one automatic discount in Shopify that checks the shopper's group at checkout, and shows each group's offer on your store. Offer tests.

Shipping

Test shipping across your whole store. There are three kinds of shipping test, picked when you create the test:

  • Shipping rates: each group sees its own real rates at checkout, for example $6.95 with free shipping over $100 against your current $8 with free shipping over $70. Crovio serves the rates through Shopify's carrier-calculated shipping and hides the Shopify rates you replace at checkout, only for shoppers whose group gets new rates. Your Shopify rates themselves are never changed. It can raise a free-shipping threshold as well as lower it. Needs carrier-calculated shipping (Advanced or Plus plan, or Shopify's add-on). Shipping rate tests.
  • Free shipping: free shipping over an amount, per group.
  • Shipping discount: a percentage off shipping over an amount, per group.

Free shipping and shipping discount tests leave your Shopify rates as they are. Crovio applies each group's offer at checkout with an automatic discount called Crovio shipping tests. Set the minimum cart subtotal in your store currency (it's converted for other currencies) and the label shoppers see. These offers can only make shipping cheaper, so to test a higher threshold use a shipping rates test. For a subscription, the offer covers the first order only: renewals ship at your normal rates in every group, so keep that in mind when you compare subscription revenue. Visitors join on their first page view.

Checkout

Test what shoppers see and choose inside checkout. For each group you can:

  • Add checkout blocks: upsells, a free gift, a free-shipping bar, trust badges, payment icons, a review, a call out, FAQs, a timer, an image or a logo, and next-order codes or links on the thank you page
  • Hide, rename or reorder payment methods and shipping methods
  • Show or hide the Crovio checkout blocks you already have

Each block has a Location ID (like crovio-1). Place a Crovio block with that ID once in Shopify's checkout editor and every later test can reuse it. Set up a checkout test.

Testing the cart drawer

There's no separate type for it: in any test's groups, switch on Show the Crovio cart drawer in this group. A/B test the cart drawer.

Testing widgets

The same goes for widgets: in a content, template or split URL test, pick the widgets each group shows. A/B test a widget.

Was this page helpful?