Ten questions that separate an AI vendor from a reseller
A lot of "AI agencies" are one person with a no-code subscription and a markup. These questions surface that in about ten minutes, and they work even if you are not technical.
The barrier to calling yourself an AI automation company is currently a domain name and a landing page. Some of these firms do good work. Many are reselling a no-code platform at a 400% markup, and when it breaks they cannot fix it because they never understood it.
You do not need to be technical to tell the difference. You need to ask questions where the honest answer and the evasive answer sound obviously different.
The ten questions
1. Who owns the code when we are done?
The answer should be "you do, and it is in your GitHub organization." If the answer involves their platform, their account, or their portal, you are renting. Renting is a legitimate model — but know that you are doing it, and price the switching cost accordingly.
2. What happens if we stop working with you?
You want to hear that another competent developer could pick it up. If leaving means the thing stops working, the vendor has a hostage, not a client.
3. Show me something you built that is running in production right now.
Not a demo. Not a video. Something with real users. Ask what broke and how they found out — anyone who has actually operated software has a good answer to that and enjoys telling it.
4. What is the accuracy, and how did you measure it?
For anything involving extraction or classification, there is a number and it is not 100%. A vendor who says "highly accurate" either has not measured it or does not want to say. Ask what they measured it against.
5. What happens when it gets something wrong?
The right answer describes a specific mechanism: a review queue, a confidence threshold, a reconciliation check, an alert. "It rarely does" is not an answer to this question.
6. What is the monthly cost, broken into hosting and AI usage?
They should be able to itemize this. A single bundled monthly number with a healthy margin baked in is worth knowing about before you sign, not after.
7. Who writes the code — you, or someone else?
Offshore or subcontracted development is not disqualifying. Being cagey about it is. You want to know who is actually going to understand your business.
8. What would you tell us not to automate?
This one is the most revealing question on the list. Anyone with real experience has strong opinions about what is a bad idea. A vendor who thinks everything is a great fit for their product is selling, not advising.
9. How do you handle our data?
Where is it stored, who can see it, is it used for training, what happens on termination. If they have not thought about this, they have not worked with anyone who cares — which tells you about their client base.
10. What does the handoff look like?
Documentation, a runbook, training for your staff, and a defined support arrangement. "We are always here" is not a support arrangement.
Two red flags that are not questions
- Urgency. If someone is pressuring you to sign before you have talked to anyone else, the pressure exists for their reasons rather than yours.
- Volume of jargon. Genuine expertise sounds simpler, not more complicated. If you cannot follow the explanation of what they are building, ask them to explain it again — a good engineer can, and will.
Written by
Matthew Hoffman, founder of SGH Logic. He writes the code and sits in the discovery meetings — more about that here.
Keep reading