
Fraud Controls
Part of Payment authentication and fraud controls
Testing fraud rules before applying them to every order
Testing fraud rules before applying them to every order: practical criteria and a clear evaluation process for marketing teams.
undefined
A useful test has two outputs: an integration check and an estimate of production reach. Stripe Radar searches historical live-mode payments matching a proposed rule before it is added or updated, while Adyen Protect offers custom-rule backtesting against historic data (Stripe docs; Adyen docs). Neither result proves that future fraud will fall.
Pros and cons of testing fraud rules in production-like environments
- ProsRealistic signal behaviour, accurate response handling, early detection of edge cases
- ConsNo guarantee of future fraud reduction; may affect real customers; requires careful monitoring
Specify and test the proposed fraud rule
Write the rule's condition, action and purpose. Is it meant to request authentication, send a payment for review or block it? Identify the payment methods, regions and order types within scope. For Adyen custom rules, choose Allow, Block, Review or Check for 3DS, and set whether the rule runs before or after authorisation (Adyen docs).
Define what should happen when the underlying signal is missing or unreliable. Set an owner and a rollback trigger before enabling the rule.
Build test cases for an expected match and non-match, including a legitimate-looking order near the threshold, a repeated attempt and a missing signal. Record the expected action for each case, then check the shopper response, order state and fulfilment queue in the provider test environment. This tests payment handling, not the real-world fraud outcome.
With Stripe, create a sandbox payment in the Dashboard or use a test API key; 4242 4242 4242 4242 simulates a successful card payment (Stripe docs). Stripe's test values also cover declines, fraud, invalid data and 3D Secure authentication; use the documented value for each case and check the Radar rule's response (Stripe docs).
Adyen's documentation also describes testing configured RevenueProtect rules before go-live. Check whether the rule triggers (Adyen docs).
Testing fraud rules before applying them to every order
- Define the rule's condition, action and purposeSpecify whether the rule should request authentication, send for review or block a payment. Identify scope: payment methods, regions, order types.
- Set fallback behaviour and ownershipDefine what happens if signals are missing or unreliable. Assign an owner and set rollback trigger.
- Build test casesCreate scenarios including expected match, non-match, threshold near-lookalike, repeated attempt, and missing signal. Record expected outcomes.
- Execute in test environmentUse Stripe’s sandbox (e.g., test card 4242 4242 4242 4242) or Adyen’s test tools. Verify shopper response, order state and fulfilment queue.
Backtest the rule against production history
Before adding or updating a rule, Stripe Radar searches historical live-mode payments matching its criteria. The test summary reports volume and counts in categories including blocked and failed payments; inspect matching payments to see which legitimate orders a block or allow rule could affect (Stripe docs).
Adyen Protect's custom-rule backtesting simulates performance using historic data before activation or a change; it is a Premium feature (Adyen docs). Adyen lists an online payments integration, enabled risk management and premium features as requirements, as well as a Customer Area role such as Merchant change risk settings or Risk admin (Adyen docs). Treat outputs as estimates: historical data does not label every blocked transaction, and customer behaviour can change.
PaymentBrief recommends shadow-mode before live cut-over and setting a baseline for fraud, chargeback, approval, manual-review and false-positive metrics (PaymentBrief). Where supported, start in review-only mode or a narrow segment, then compare those measures and customer contacts with the baseline before expanding. Decide whether the effect on legitimate orders is acceptable before applying the rule more broadly.
Fraud disputes may take longer to appear, so an immediate drop in approved orders should not be called a fraud success. Keep issuer declines separate from merchant risk blocks.
Expand fraud rules deliberately
After reviewing early results and later fraud outcomes, adjust the condition or action before applying it to more traffic. Record the version and activation time so later changes can be traced.
If legitimate customers are caught unexpectedly, pause or roll back while the cause is investigated. Avoid stacking several new rules at once, as it becomes difficult to tell which one caused a change.
The release decision should say what the rule is expected to catch, who may be turned away, and what evidence would justify changing it.
Further reading: Custom rules (docs.adyen.com).



