How to Hire an AI Development Agency Without Getting Burned

A polished chat window tells you little about how an AI feature behaves with your data. Before hiring a team, ask to see the test cases, failed outputs and what happens when the system cannot answer. This guide gives you a practical starting point for that review.

Separate demos from production

A demo runs on happy-path inputs in front of a friendly audience. Production has to handle the question your model has no answer for, the retrieval that comes back empty, and the user who does something no one predicted. Ask what happens in those cases. If the answer is vague, the system is a demo.

Questions that reveal real experience

• How do you measure and reduce unsupported answers? Ask for test results, fallback behavior and the cases that still need human review.
• What do you do when retrieval returns nothing relevant?
• Where does a human stay in the loop, and why there?
• How do you evaluate the model before launch, not just eyeball it?
• Can you point to AI you've put in front of real users, not just a demo?

Watch for these

• AI as decoration: a chatbot bolted on that doesn't touch your actual data.
• No talk of grounding, evaluation or fallbacks, only capabilities.
• A binding build price without scope or feasibility assumptions. A planning range is useful when the uncertainty is explicit.
• No willingness to say an idea isn't feasible yet.

What good looks like

Ask the team to explain a failure they found and show what changed after it. For retrieval, inspect the source passages alongside the answer. For an agent, inspect the requested action, permission checks and approval boundary. A case-study percentage needs a metric definition, test conditions and limitations before you can compare it with your own project.

Ask for the evaluation record

Start with the actual task and the cost of a wrong result. Use representative questions from intended users, with permission to use the data. Keep a held-out set separate from examples used to tune prompts. Add difficult cases as you discover them. OpenAI's evaluation guidance describes this approach and the need to evaluate changes over time.

• Test-set origin: where the inputs came from, which user groups they cover, how sensitive data was handled and what is missing.
• Expected behavior: the source answer or allowed action, when to ask for clarification and when to refuse or escalate.
• Run details: date, model identifier, prompt and retrieval configuration, document snapshot and relevant package versions.
• Results: per-case outputs and judgments, counts and denominators for each failure category, repeated runs where behavior varies, and named reviewers.
• Operating limits: measured latency and provider costs under stated load, timeout behavior and who owns monitoring after launch.

An example test set for a support assistant

These are proposed tests for a hypothetical product-support assistant. They are not results from a Prograsec deployment. Assume it can read approved documentation and draft replies, but cannot change customer accounts. Adapt the cases to your own permissions and workflow.

A worked acceptance check: the approved instructions say to open Settings, select Exports and choose CSV. The user asks how to export a report. An answer passes this case only if those steps are correct and its citation supports them. Adding an undocumented automatic-export feature fails the case, even if the rest of the answer is fluent. This example describes a grading rule, not a measured success rate.

Read the failures before the headline score

An overall accuracy number can hide a private-data leak or an unauthorized action. Review these categories separately from ordinary answer quality. Agree launch gates before the run, report failures and record any exclusions. Passing a finite test set reduces uncertainty; it cannot guarantee correct behavior on every future input.

A weak proposal says the assistant uses retrieval, so hallucinations are solved. A useful proposal identifies what is tested, shows unsupported-answer failures and explains when a person takes over. If the team cannot share client records, ask for a redacted report or a permitted demonstration against synthetic data.

Agree the pilot and the handover

Separate a feasibility pilot from the production build when the hardest questions remain open. Set its permitted data, deliverable, price and stopping point in writing. Include a decision to continue, narrow the use case or stop after reviewing the evidence.

• Assign responsibility for approving documents, access permissions and the evaluation rubric.
• Agree the failure behavior for missing sources, provider outages and unavailable reviewers.
• List provider accounts, recurring charges and the changes that trigger a new evaluation. The AI feature cost worksheet is one way to write the model line down before that conversation.
• Require handover of the agreed code, configuration, evaluation set and operating instructions, within the permitted disclosure boundaries.