Ready to Evaluate AI Tools? Do These 5 Things First

AI tools are becoming easier to access in business than ever. In fact, your staff may already be using them (although we hope not without agency approval and guidelines!)

Even if your agency isn’t quite ready to start adopting Artificial Intelligence, however, there are still some steps to take to ensure it is protected once the transition happens.

The good news is that you can take these steps today, before you even pick out a tool, and be ready when the eventual mandate comes down.

1.Build Your Policy Before Your Pilot

To build a functional AI policy, you should first define the answers to four questions. These will guide your staff and your workflows once you start evaluating AI-powered tools.

• What AI tools are we allowed to use, and for what purposes?

• What information can and cannot be shared with AI systems?

• What documentation is required when AI is used in a procurement decision?

• Who approves new AI tools before they’re used?

Even if your policy isn’t exhaustive, it’s still useful, giving you a guideline to follow as you evaluate and consider what you want in AI and who you want as a vendor.

Include your city attorney, IT director, and HR in drafting this policy, as well. If you only include procurement, your policy probably won’t survive the first legal question. The federal OMB memoranda M-25-21 and M-25-22 and the NIST AI Risk Management Framework are useful references, even though they do not bind state and local agencies directly.

2.Evaluate AI Tools with Procurement in Mind

Treat the evaluation of an AI tool just as you would any other significant technology purchase, asking the same questions you would ask for an ERP, an HR system, or an eProcurement tool. Then, add in the questions that are specific to AI.

• What is the error rate on tasks like yours, and how was it measured?

• Where does the data you enter go, and is it used to train the vendor’s model?

• Can the system explain how it reached an output in a form that can be disclosed in a public records response?

• What happens with the AI is wrong, and what remedies does the contract provide?

3. Design TO BE AUDITABLE from Day One

Any procurement action that uses AI, even partially, should maintain a documented history for auditing and reporting purposes. This history should include five things:

• The tool and what version was used
• The input that was given to the tool
• The output from the tool exactly as produced
• Who reviewed the output and what they changed
• The final action and its basis

Keep the AI output and the final reviewed version as separate records. Public records law varies by state, so confirm your disclosure obligations with your attorney before AI ever enters a live workflow.

4. Start with a Limited-Scope Pilot

Pick on use case in one workflow, such as drafting scope language for a single category. Before you begin, define the following:

• The duration
• The success metric
• A named review gate
• A rollout plan

A pilot program that has a defined exit will be easier for your more skeptical colleagues to support and go into with an open mind, while also producing documented evidence you can use later.

5. Protect Fairness, Equity, and Vendor Rights

Review AI-generated solicitation language for specifications that could narrow the field or put small and diverse businesses at a disadvantage. Any AI used in the evaluation process must still produce documented criteria and scores that can be disclosed to losing vendors. If an AI tool cannot explain its reasoning in that form, it shouldn’t be used in formal evaluations.