Home
Chase Teichmann Request an intro
Rules score each return, unclear cases go to an LLM, and flagged returns wait for staff.Illustration: a magnifying glass over a conveyor of return slips. Three slips marked with the same item share a HOLD sticker, and a later slip passes with a green check.HOLDPATTERN
Rules score each return, unclear cases go to an LLM, and flagged returns wait for staff.
Rules score each return, unclear cases go to an LLM, and flagged returns wait for staff. Return started to Fraud indicators; Fraud indicators to LLM evaluator; LLM evaluator to Staff guidance; Staff guidance to Fraud?; Fraud? to Usual flow (No); Fraud? to Flagged & blocked (Yes) Return started Fraud indicators LLM evaluator OpenAI · A/B-tested prompts Staff guidance Fraud? Usual flow Automation as set up Flagged & blocked From automation No Yes
Rules score each return, unclear cases go to an LLM, and flagged returns wait for staff.

AI

Returns fraud evaluator

I built an AI fraud evaluator that holds suspicious returns for a person to review, instead of letting automation approve them.

Company
ReturnLogic
Role
Built the evaluator
Period
2025
Stack
OpenAI · TypeScript · Node.js · PostgreSQL · AngularJS · Sequelize · Jest · Docker · ECS · Fargate
Add to cart Message me on LinkedIn

“Add to cart” takes you to the store with this project in your cart.

The risk

Retailers on ReturnLogic could authorize and process returns automatically, which is also the path a fraudulent return takes. Fraud leaves patterns in a shopper’s return history: the same item sent back again and again, the same return type or reason every time, a burst of returns in a short window.

How a return is judged

  1. ScoreRulesRules score the return’s fraud indicators. Clear-cut returns are settled here.
  2. JudgeOpenAIWhen the rules can’t decide, OpenAI decides whether to flag the return and writes its reasoning.
  3. HoldThe automationA flagged return is held back from automatic authorization and processing. It isn’t denied.
  4. DecideStaffStaff see the flag and the reasoning on the return and make the final call.

What testing caught

A provider abstraction let us A/B test prompts and swap providers without a rewrite. The test picked the prompt we shipped and exposed three guardrail gaps, which we closed.

  • Flagging on thin evidenceThe model flagged returns the rule scores didn’t support.
  • Inconsistent verdictsThe same return history could get different answers.
  • Shopper text steering the modelReturn reasons and notes that shoppers write could sway the verdict.

Cost

Each evaluation got cheaper three ways: a shorter prompt, varied reasoning effort on one primary model, and calling the model only when the rules can’t decide.