Measuring whether a wallet option adds completed orders

Randomly assign eligible checkouts to a version with the wallet option or a version without it. Compare total confirmed completed orders across everyone assigned, including shoppers who never select the wallet. Wallet payment count measures use of the option; some wallet buyers might otherwise have paid by card.

Define who enters the experiment

Choose the wallet, placement, integration and purchase routes in scope. Availability depends on the actual setup and shopper environment. For example, Stripe's Express Checkout Element presents methods according to activation, supported browser and currency, domain registration and relevant customer setup. Stripe says a customer won’t see a Google Pay button if they don’t have Google Pay set up. Those rules describe that Stripe product, not every wallet integration.

Choose a population that can be identified in both groups before the new option changes behaviour. If the store can reliably determine who could be shown the wallet in either group, it may predefine that eligible population. If it cannot, use all assigned checkouts for the primary comparison and report actual wallet visibility separately. Do not select control shoppers using a treatment-only button event.

Preserve the alternative payment route

Add the wallet in treatment while keeping an ordinary supported payment route. Hold the product, amount, delivery choices, promotions and account rules steady. If the wallet appears before the usual checkout entry, assign shoppers before that express entry point; otherwise, early wallet users could disappear from the experiment denominator.

Before launch, check the offered wallet and fallback paths in the provider's permitted setup, including approval, cancellation and an ineligible environment. Compare the displayed total and order record. These checks assess integration behaviour, not added sales. Link retries and method switches to the same checkout.

Measure addition and substitution

Use distinct confirmed completed orders per eligible assigned checkout as the primary rate. Show assigned checkout and completed-order counts for both groups and keep pending outcomes visible until the agreed observation window closes. Reconcile order and provider outcomes where a payment can finish after the browser leaves.

Inspect wallet visibility, selection, attempts, confirmed wallet orders, card orders, failures and fallback as diagnostics. More wallet orders alongside fewer card orders can indicate substitution. Evidence for added orders comes from the difference in total completed orders between assigned groups, subject to uncertainty and data-quality checks. Comparing self-selected wallet users with card users cannot answer that question.

Check allocation, missing outcomes, incorrect totals, duplicate orders and failed fallback before deciding. If the total-order effect remains uncertain, report that uncertainty and the wallet's observed reach. Apply any result to the tested placement and population; a different integration or placement may have different reach.

Steps to Measure Wallet Impact on Checkout Completion