AI Agent Platform Evaluation Criteria: The Nine Tests That Decide It

What are the criteria for evaluating an AI agent platform?

AI agent platform evaluation criteria are the specific, testable conditions a platform must satisfy before it is trusted with production data, production decisions or production accountability. The emphasis on testable is what separates an evaluation from a comparison. A comparison asks which platform looks more capable. An evaluation asks what happens when you put your own data, your own permissions and your own failure cases in front of it.

Most enterprise agent platform comparisons fail for the same reason. They score the things a vendor controls — how fluent the output looks, how many connectors are listed — and skip what the enterprise will live with: whether a fixed test set can be re-run on demand, whether permissions are enforced at retrieval time rather than at application time, and whether the platform can explain what a specific agent was allowed to do last Tuesday.

The nine criteria below are ordered by how often they decide a decision, not by how interesting they are to demo. Evaluation tooling comes high because it is the criterion whose absence you discover during an incident. Deployment and tenancy come high because a platform that cannot be isolated will not pass your own information security review. Model quality comes near the end — the model is the layer most likely to be replaceable in eighteen months, while the governance layers are not.

How do you test agent quality instead of watching a demo?

A demonstration is designed to succeed. It uses a clean document, a well-formed question and a topic the vendor has tuned against. The gap between that and your environment is where every agent platform eventually disappoints, so the evaluation has to close the gap deliberately.

Four things to ask to see

• The retrieved set, not just the answer. The candidate chunks, the reranked order, and which were placed in the context window. A platform that cannot show you this cannot be debugged, only trusted.
• The tool call trace. Which tool, which arguments, what came back, what the agent did with it. This is the difference between an agent and a long prompt with search bolted on.
• The failure path. What happens when a tool times out, returns malformed data, or returns a permission error. A platform with no defined behaviour here will invent one at runtime.
• The raw evaluation result. Scores per case, not a single averaged number. An average hides the case that matters.

If the vendor resists any of the four, you have your answer before the pilot. Refusal is not proof the platform is weak, but it is proof the platform will be difficult to run under your own audit obligations later.

When should you build a platform instead of buying one?

Build when the agent surface is the product: your customers interact with your agents directly, the orchestration logic is what differentiates you, or your agents must call proprietary internal machinery no vendor will connect to. The platform is then core, and the decision should default toward building the orchestration layer yourself.

Buy when the platform carries process rather than differentiation: agents serve internal work, and the differentiator is the quality of your retrieval corpus or your business semantics. The orchestration layer you build on top is thin, and the platform's lifecycle, evaluation, security and observability machinery is worth paying for rather than rebuilding.

There is a third option that enterprises underestimate: buy the platform, build the semantics. Most enterprise agent failures are not platform failures. They are the consequence of three systems holding three definitions of the same business object, so the agent is confidently reasoning over the wrong one. That work is discussed in our article on what enterprise ontology is and why agents depend on it, and it is a consulting engagement, not a procurement one.

Whichever route you take, plan on owning three things yourself: the orchestration logic, the evaluation sets, and the definition of what a correct answer is. Those are the assets that keep their value if the platform changes, and they are the assets a vendor contract will never hand you.

How do you run a proof of concept that produces evidence?

A proof of concept should produce the same artefacts a production system would, on a smaller scope. If the pilot output cannot become the first version of your test set, your pilot is a demo with a longer timeline attached.

1. Week one — prepare. Select the process, define the target outcome, and write the evaluation set with your own subject experts before any vendor is involved. If the experts are only consulted to score results afterwards, the set will be written to match whatever the platform does.
2. Week two — configure. Connect one real system of record with real permissions applied. Ingest a bounded but real corpus, including the awkward documents: scanned pages, tables, superseded versions, conflicting metadata.
3. Week three to four — run and measure. Run the fixed set, then change one parameter at a time and record the effect. Capture the traces, not just the outputs. Promote every failure into the set as a named case, so the set grows rather than resets.
4. Week five — govern. Introduce the governance layer you intend to run in production: access control for the agent identity, audit trail, escalation path, and the human review point where a wrong answer would be expensive.
5. Week six — cost and scale. Model the per-task cost at realistic volume including supervision, then test the behaviour when the corpus doubles and when a new agent is added to the same estate.