Open-box AI Tagging
Would you trust an AI's decision more if it showed you why, and let you overrule it?
What you're looking at
An AI has reviewed six marketplace listings and flagged the ones it thinks are counterfeit. You play the brand-protection analyst who decides whether the AI is right. The same AI comes in two versions: a black box that only gives its verdict, and an open box that shows the evidence behind it.
How to try it
- Switch to Black box. Read each listing and the AI's verdict, then click Agree or Overrule. Notice how hard listing 2 is to judge with no reasons.
- Switch to Open box. Each listing now shows the evidence behind its risk score and how much each piece counted. Colors mark the kind of evidence and stay the same in every listing. Bars right of the line raise the risk, bars left of it lower it. Click a color in the legend to see only that kind across all six listings.
- Look at listing 2 again. It was flagged on price alone, but the seller is the brand's own outlet with an eight-year history. Overrule it.
- Check the line under the queue. It counts your calls and tells you whether you caught the AI's mistake. Undo any call to try again.
Review queue
A simulated model flags counterfeit risk on marketplace listings. Brands and sellers are fictional.
Review each listing: agree with the model, or overrule it.
Hypothesis
If an AI shows the evidence behind each decision and makes overruling it effortless, experts will trust it enough to stop rechecking its work. Accuracy alone does not earn that trust. People who are accountable for a decision need to see why it was made, especially when the AI is wrong.
What would prove it:
- Analysts spend less time per listing in the open box view, because they verify the evidence instead of researching from scratch.
- They catch the AI's mistakes, like listing 2, more often.
- They agree with the AI's correct calls more often, instead of second-guessing everything.
The idea
At OpSec Security, analysts protecting brands from counterfeiters were handed machine learning that tagged listings for them. The tags were often right, but analysts couldn't see why, so they rechecked everything and the time savings evaporated. The fix wasn't a better model. It was an open box: show the evidence behind every tag, and let analysts overrule it.
Try both views above. In black box mode, the second listing just says "Needs review," so an analyst has to research it from scratch. In open box mode, you can see the model flagged it on price alone, from a seller with an eight-year history, and that it's a genuine clearance sale you can overrule in a second.
Design principles
- Show the evidence, ranked. Each signal and how much it pushed the decision, in plain language.
- Make overruling easy. One click, no justification required, and the model learns from it.
- Be honest about confidence. A middle band for "needs review" instead of forcing every call into yes or no.
- Measure agreement, not just accuracy. How often experts accept the model is the real signal of trust.
In the shipped Smart Tagging feature this approach reported efficiency gains of up to 62%. The full story is in the OpSec case study.
How it works
There is no real model here. Each listing carries a few weighted signals, the way a classifier's features would, and the score is their sum. That's enough to show the interaction design, which is the point: the same decision feels very different depending on what the interface lets you see.