Analyzing A/B test results with Primer's Split utility

Primer's Split utility in Workflows empowers merchants to conduct A/B tests, allowing for the random distribution of traffic based on predefined split percentages. To effectively analyze the results of these A/B tests and optimize payments accordingly, you can leverage Primer's Observability Dashboards. Here's a step-by-step guide to make the most of your split test results:

Understanding the Split utility

1. 1
   Create A/B test conditions: Before using the Split utility, it's essential to define your A/B test conditions within your workflow. Ensure that you have clearly set up the conditions and any trigger conditions that guide the split.
2. 2
   Utilize Split utility: Employ the Split utility to distribute traffic as per your predefined split percentage. This utility is a powerful tool for conducting controlled experiments.

Analyzing Split test results

1. 1
   Assess workflow conditions: Begin by closely examining the conditions within your workflow before the Split utility. Understand the different payment setups you are testing and the variables that might impact the results. Don’t forget to look at any trigger conditions that have been set up as part of a workflow, and custom metadata included as part of the authorized action on the payments app.
2. 2
   Choose Relevant Dashboards: Depending on the nature of your A/B test, select the appropriate dashboards within Observability. For instance:
   • If you're testing different 3D Secure options, consider using the "3D Secure Dashboard", for example, when comparing Primer’s 3D Secure and Adaptive 3D Secure for specific BINs or issuing banks.
   • If you’re comparing different processor authorization rate performance for a sub-set of payments, opt for the "Payments Dashboard" which includes a detailed analysis of authorization rates.
3. 3
   Incorporate conditions as filters: To effectively analyze your split test results, apply the conditions you've set in your workflow as filters within the chosen Dashboard. This step ensures that you isolate and compare the outcomes of different payment setups configured using the Split utility only.
4. 4
   Review and Compare: Examine the results presented in the dashboard, specifically the chart that compares the two options you're comparing. You can switch between Dashboards if you need to analyze multiple charts, for example, assessing authorization rate and 3D Secure authentication rate together.

By following these steps, merchants can effectively assess A/B test results, gain valuable insights, and make informed decisions to optimize payment processes. Primer's Split Utility and Observability features work hand in hand to enable data-driven experimentation and continuous improvement in your payment strategies.