Under the hood: UpliftAI
UpliftAI, our AI-enabled smart routing solution, is a game-changer for merchants. Our pilot has shown it can deliver a 5% authorization rate uplift for merchants with nearly zero effort.
But how does it work? And what steps did the team take to build it?
I spoke with our Product Lead for UpliftAI, Tim Cooper, and our Machine Learning Engineer, Art Koci, to go under the hood and learn more.
Tell us about the specific architecture of the AI model used to build UpliftAI
UpliftAI primarily operates as a supervised binary classification model. It analyzes incoming observations, categorizing them into one of two outcomes. Specifically, UpliftAI determines whether a payment processor will successfully process a payment or not.
That oversimplifies somewhat, however. UpliftAI is anything but a standard machine-learning model; it’s unique to Primer and the use case we’re trying to solve. We’ve built a ton of custom functionality designed to ensure UpliftAI is incredibly accurate in predicting whether or not a payment processor will authorize a payment.
What data informs UpliftAI, and do specific data points take precedence?
First, it’s worth highlighting Primer's position in the payment lifecycle. As a Unified Payments Infrastructure that connects a business's disparate payment services, we see payments flow end-to-end and across all processors. This is a unique vantage point and one that sets UpliftAI apart.
How we trained the model is also what sets the solution apart. We conducted numerous experiments to pinpoint the data fields with the most influence on authorization rates. We even built new custom data fields that utilize payment data points to see if these would make an impact.
Through this process, we identified the most influential data points. It’s this subset that the model uses today, ensuring it’s incredibly accurate in its predictions and also lightweight —it takes just 25 milliseconds on average for UpliftAI to make its recommendations.
How do you benchmark performance?
Our model undergoes ongoing monitoring using industry-standard machine learning metrics to assess its base performance. Additionally, we compare the predictions made by UpliftAI with successful fallbacks for transactions that don’t utilize UpliftAI. This process aids in validating performance, especially when retraining existing models and deploying new ones.
How do we optimize the model?
We employ two primary approaches for model optimization:
Automated retraining: An automated system constantly updates the model with the latest data to prevent performance decay caused by data shifts—either internal or external. The aim is to keep UpliftAI aligned with current payment trends.
Manual updates: We run experiments utilizing new machine learning algorithms, data features, functionality, or other Primer solutions. These optimizations typically take longer but tend to deliver a significant boost in the performance of UpliftAI when successful. Manual updates are also necessary in rare situations where merchants or Primer alter the data structure fed into the UpliftAI.
Additionally, we maintain a control sample of payments, establishing a training baseline for ongoing learning and monitoring performance improvements resulting from model updates.
AI models can often be seen as "black boxes." How have you tackled the challenge of making the AI model's decision-making process more transparent and understandable to users?
Most, if not all, AI and machine learning models suffer from the black box issue to some degree. In the case of UpliftAI, transparency is delivered through our use of structured and consistent tabular data. This allows us to analyze how the model influences merchant authorization rate improvements.
This visibility is important because we want to provide merchants insights into when UpliftAI was used and the respective confidence levels for all the available processors.
Addressing bias and ensuring fairness in AI models is crucial. What steps have been taken to identify and mitigate biases?
Our goal is to increase the authorization rate for our merchants, so any adverse effects that may arise from bias will eventually hurt our merchant’s customers and the performance of UpliftAI. The steps we’ve taken to mitigate the risk of bias include omitting certain data sets from the decision-making process of UpliftAI. For example, we only use generic customer data and anonymize it so it can’t be linked to a particular customer.
How can merchants be assured their data is secure?
We maintain strict data protection policies across Primer, enforced by our infrastructure teams and computational resources. These protocols are consistent across all Primer products, including UpliftAI.
What’s next for UpliftAI and AI more broadly at Primer?
Our goal is to eliminate unjustified payment declines. Solutions like UpliftAI will make this happen.
We’re already exploring expanding the utility of UpliftAI in various ways, such as adding new payment processors and working on an AI model built to improve checkout conversion.
In the near term, we’re focusing on making iterations on the model using the production data we have from the pilot. We’re already looking at how we can improve the model performance based on the results we’re seeing to make an even more significant impact on merchant authorization rates.
