Guide

AI governance software: what it does, and when a process is enough

AI governance software does four jobs. It keeps an inventory of the AI systems and tools in use, runs risk and compliance reviews against frameworks such as the NIST AI RMF and ISO/IEC 42001, monitors AI systems once they are live, and enforces some policies automatically, such as stopping sensitive data from reaching an AI tool. It records and routes decisions but does not make them, and it cannot enforce a rule inside a system it is not connected to. A documented process, meaning a maintained register plus controls built into the systems themselves, is enough while AI uses are few and rarely make consequential decisions. Software earns its cost when the register changes faster than people can keep it accurate, or when auditors and regulators need the same evidence from many systems.

AI governance software. Software that records an organisation’s AI systems, runs risk and compliance reviews on them, monitors them in use and enforces some AI policies automatically.

Written by Tayyeb Mahmud, founder and Principal Architect. Claude Certified Architect – Professional · LinkedIn profile of Tayyeb Mahmud

Checked 29 September 2026. Laws, supervisory guidance, frameworks and vendor documentation were read on the publishers’ own pages. They change, so confirm the current text on the sources below before you rely on it. This page is not legal advice.

The four jobs AI governance software does

Vendors sell this under overlapping labels: AI governance platform, AI governance tool, AI risk management software. Underneath, the products do some mix of four jobs. A product can do one of them well and the others lightly, so sort a demo by job before you compare prices. Two products with the same label can be solving different problems.

The four jobs AI governance software does, what each does not do, and where you may already own part of it
Dimension	What it does	What it does not do	You may already own part of it in
Inventory	Keeps a register of AI systems, models, vendors and uses, each with an owner, a purpose, the data it touches and a risk tier. Some products also discover AI use from network, identity or cloud data.	Say what a use is for or whether it should exist. Discovery finds traffic and accounts; a person still has to explain each one.	A model registry for the systems you build (MLflow’s is open source [12]), your identity provider’s list of connected apps, and the admin consoles of the AI tools you already license.
Risk and compliance workflow	Runs intake questionnaires and impact assessments, maps your controls to frameworks and laws, routes approvals and stores the evidence.	Make an assessment true. It records that a review happened and what people said, not how the system behaves.	Your existing governance, risk and compliance tool, if it can hold a new register and a new questionnaire.
Monitoring	Watches systems once they are live: accuracy and drift for predictive models, and prompts, responses, cost and flagged content for generative AI.	Know what a correct answer is in your business. Thresholds and test cases have to come from the people who own the process.	Your cloud provider’s machine learning tooling, your observability stack, and the logs your own AI services already write.
Policy enforcement	Applies some rules automatically at a chokepoint: blocking sensitive data in prompts, limiting which AI tools can be reached, filtering what a model returns.	Enforce a rule inside a system it does not sit in front of. A gateway cannot know that a refund above a limit needs a manager’s sign-off.	Data loss prevention and security tools. Microsoft, for example, documents its Purview AI features as reporting AI activity and applying ready-made policies against sensitive data in prompts [11].

The AI tools you already pay for belong in the inventory too, and their admin settings are part of the answer. Our explainer on the admin controls AI work tools ship with sets out what three vendors give an administrator.

Buy if you need to, but know first which of the four jobs you are short of. If the gap is the register, a spreadsheet or a file in version control can close it this quarter. If the gap is evidence across many systems and business units, it will not.

What no platform does for you

Every product in this category records decisions. None of them makes the decisions, and none of them can stop a system doing something it was built to do. That is the failure to watch for: a platform goes in, the register fills up, and nothing about what the AI systems are allowed to do has changed.

Four things stay with you whichever product you buy.

Accountability: A named owner for each AI use and each control. Software can fill in an owner field. It cannot make that person answer for the result.
Risk appetite: Which uses you allow, which need approval and which you refuse. A platform ships default risk questions; the thresholds are your decision.
Controls inside the system: Identity with the least access the job needs, checks on every consequential action, tests on every change and a switch that turns a capability off. They live in your code and configuration, which is the only place they can stop anything.
Evidence at the source: A platform can only show evidence something produced. If your systems do not record what ran, on which inputs and who approved it, there is nothing to import.

NIST’s framework draws the same line. Its Govern function asks for mechanisms to inventory AI systems, and its Measure function asks that a system’s functionality and behaviour be monitored in production [2]. The first is record-keeping a platform can do for you. The second depends on what your own systems measure and write down.

When a documented process is enough, and when it is not

A documented process means a register someone keeps accurate, a short policy, a risk tier on each use, and controls built into the systems that matter. NIST’s playbook describes an AI system inventory as an organised database of artefacts about each system, such as documentation, incident response plans, data dictionaries, links to source code and the contacts for it [3]. Nothing in that description needs a product.

Signs that a documented process is still enough, and signs that software would earn its cost
Dimension	A documented process is enough	Software earns its cost
How fast AI use changes	One owner can keep the register accurate with a monthly review	Uses change every week across several business units, and the register is out of date before anyone reads it
Who buys AI	A central team approves every tool	AI arrives inside software each department already buys, and nobody sees it until the invoice
What the systems decide	Drafting and search, with a person reading every output	Decisions on credit, jobs, housing, insurance or health care, where a law asks for notices, reviews or records
Evidence requests	An occasional customer questionnaire	Auditors, regulators or enterprise customers asking for the same evidence, repeatedly, across many systems
Frameworks in play	One reference framework, such as the NIST AI RMF	Several at once, such as the NIST AI RMF, ISO/IEC 42001 and the EU AI Act, mapped to one set of controls
Monitoring	A few systems whose owners already watch them	Many models in production, with nobody watching drift or cost in one place

If most of your answers sit in the left column, I would spend the budget on the register and the controls first. You will buy a better platform later, because by then you will know what you need it to hold.

What US rules and frameworks ask for

None of the instruments below requires you to buy software. Several ask for things software can help you keep: an inventory, records, reviews and monitoring. Read each for what it asks of you, then decide whether a process or a product does that job better.

NIST AI RMF: AI RMF 1.0, released on 26 January 2023, is voluntary [1]. Its core has four functions, Govern, Map, Measure and Manage, and its subcategories include mechanisms to inventory AI systems (Govern 1.6), monitoring of a system’s functionality and behaviour in production (Measure 2.4) and post-deployment monitoring plans (Manage 4.1) [2]. NIST published a Generative AI Profile, NIST AI 600-1, on 26 July 2024, and says AI RMF 1.0 is being revised as part of the White House AI Action Plan [1].
ISO/IEC 42001: Published on 18 December 2023, it specifies requirements and guidance for establishing, implementing, maintaining and continually improving an AI management system [4]. A management system is how AI is run across the organisation. It is not a technical control on any one system.
Bank model risk guidance: On 17 April 2026 the Federal Reserve issued SR 26-2, revised supervisory guidance on model risk management issued with the FDIC and the OCC, superseding SR 11-7 [5, 6]. The Federal Reserve says it is most relevant to banking organisations with over $30 billion in total assets regulated by the Federal Reserve [5]. The guidance calls keeping a set of information on models in development or in use common industry practice. It places generative and agentic AI models outside its scope, while saying a bank’s own risk management and governance practices should guide the controls for them [6].
Colorado: SB26-189, signed on 14 May 2026, repeals and reenacts the consumer protections Colorado enacted in 2024 in SB24-205, with new requirements on automated decision-making technology used in consequential decisions, such as those on employment, housing, lending, insurance and health care [7]. As the legislature summarises it, developers must give deployers technical documentation starting 1 January 2027; deployers must give consumers notice at the point of interaction and a plain-language description within 30 days of an adverse decision; both must keep records that show compliance for at least three years; and the attorney general enforces it [7].
New York City: Local Law 144 of 2021 requires an automated employment decision tool to have had a bias audit within one year of its use, with information about the audit made public, and requires notices to candidates or employees. The city began enforcing it on 5 July 2023 [8].
EU AI Act: It can reach US companies. Its scope includes providers placing AI systems on the EU market wherever they are established, and providers and deployers outside the EU whose system’s output is used in the EU [9]. The European Commission lists the transparency rules as taking effect in August 2026, and says that after the AI Omnibus, which entered into force on 27 July 2026, rules for high-risk areas such as employment and education apply from 2 December 2027 [10].

This section describes these instruments as their publishers stated them when read on 29 September 2026. It is not legal advice. Whether one applies to you, and what it requires of you, is a question for your counsel.

How to evaluate an AI governance platform

Ask these in the demo, and ask to see the answer in the product rather than on a slide. The right-hand column is what a weak answer tends to look like.

Questions to ask any AI governance platform vendor, and what a weak answer looks like
Dimension	What to ask	A weak answer looks like
Where the inventory comes from	Which of our systems can it find AI use in, from which data, and what still has to be entered by hand?	A claim that it finds everything, with no list of the sources it reads
Where the evidence comes from	Does it read evidence from our systems through an API, or do people upload documents to it?	Evidence is a file someone attaches the week before the audit
What monitoring watches	Which of our runtimes and model providers does it connect to, and what does it measure on each?	A dashboard demonstrated on the vendor’s own sample data
What it can enforce	Which rules does it block rather than report, and where in the traffic does it sit?	Every rule ends as an alert for a person to read
Framework and law mappings	Which versions of which frameworks and laws are mapped, who maintains them, and when were they last updated?	A mapping that still cites Colorado’s 2024 act, repealed and reenacted in May 2026
The data it collects	If it logs prompts and responses, where are they stored, for how long, and who can read them?	No clear answer on retention or on who has access
Getting out	Can we export the register, assessments and evidence in a standard format, and has a customer done it?	Export is quoted as a services project
Who runs it	Which role on our side owns it, and how much of their week does it need once live?	The vendor cannot name the role, or says it runs itself
The vendor’s own assurance	What independent security assurance does the vendor hold, what does it cover, and can we read the report?	A trust page of logos with no report behind it

Run the trial on your own register and your own logs, not the vendor’s sample. How the product copes with your mess tells you more than any scoring matrix.

Buy, build or both: the order I would do it in

Start with the register, kept wherever it will stay accurate. At the start that can be a spreadsheet or a structured file in version control, with an owner and a review date.

Then tier the uses and build controls into the systems at the top tier: identity, checks on consequential actions, tests on every change and a switch that turns each capability off. This is the part that changes what a system does, and no licence buys it.

Buy a platform when the register or the evidence outgrows the process, using the signs above. By then you know which of the four jobs you are short of, so you can buy the product that does that job well rather than the one with the most modules.

The controls still matter after the purchase, because they produce the evidence the platform stores. Bought first, a platform tends to become a very tidy record of controls nobody has built.

Where 1AYM fits

1AYM does not sell governance software. We build the part a platform cannot supply: AI governance implementation, meaning identity and permissions the system enforces, verifier gates on consequential actions, evaluation in CI and an audit trail the gates write as they run.

1AYM works in financial services. For a bank, SR 26-2 leaves generative and agentic AI to its own governance practices, and those practices need controls someone has built. A fixed-scope statement of work can start within a day of the scope being signed, and if you have already scoped the work, it can be resourced on contract from the associates who work with 1AYM, held to the same standard. We hold no ISO or SOC certification ourselves. If a platform decision is what you are weighing, a short call is a sensible next step.

For engineers: integration points, evidence and what to test in a trial

What a technical reviewer should check before the contract is signed, and what to build whether or not you buy. Each item can be tested by reading configuration, code or logs.

Register as code: Keep the AI register as structured data in version control: owner, purpose, data classes, model and provider, and risk tier per entry. A CI check fails any deploy that calls a model endpoint with no register entry. A platform can import the file; it should not replace the check.
Discovery sources: Test which sources the platform actually reads: identity provider app grants, egress or proxy logs, cloud billing for model APIs, and SDK imports in your repositories. Each finds a different slice, and an AI feature a SaaS vendor switches on server-side may show up in none of them.
Evidence at the source: Have each verifier gate and approval step write a structured record: input reference, model and prompt version, check results, approver and timestamp. Store it append-only with a stated retention period, and push it to the platform through its API rather than exporting reports by hand.
Tracing generative AI: Trace every model call with model version, prompt version, tool calls, tokens and latency, and agree which fields the platform receives. Full prompts and responses often carry personal or confidential data, so decide whether to send them, redact them or send references.
Evaluation, not dashboards: Monitoring tells you something moved. An evaluation set of real cases with agreed answers tells you whether it got worse. Run it in CI on every change to a prompt, model, tool or rule, and send the scores to the platform.
Predictive models: For classic machine learning, track input drift and performance against labels as they arrive, per model version in the registry. That is what production monitoring of functionality and behaviour (NIST AI RMF, Measure 2.4) looks like in code.
Enforcement points: A gateway or proxy can block traffic by destination, data class or content. A business rule, such as an approval limit, can only be enforced inside the application that knows it. Map each policy clause to the layer that can actually enforce it.
Exit test: During the trial, export the register, one assessment and a month of evidence, and load them somewhere else. If that is hard in the second week, it will be harder in the second year.

Sources

Frequently asked questions

What is AI governance software?

Software that keeps an inventory of an organisation’s AI systems, runs risk and compliance reviews on them, monitors them once they are live and enforces some AI policies automatically. Vendors call it an AI governance platform, an AI governance tool or AI risk management software. It records and routes decisions. The controls that stop a system acting wrongly still live in the system itself.

Do we need AI governance software to follow the NIST AI RMF or ISO/IEC 42001?

No. The NIST AI RMF is voluntary and asks for mechanisms such as an inventory and production monitoring, not for a product. ISO/IEC 42001 specifies requirements for an AI management system, which you can run on tools you already have. Software can make either easier to evidence once the number of AI systems grows.

Does Colorado’s AI law require governance software?

Nothing in the legislature’s summary of SB26-189 requires software. As summarised there, it asks for technical documentation from developers starting 1 January 2027, notices to consumers, a plain-language description within 30 days of an adverse decision, a right to request human review, and records kept for at least three years. Software may help keep those records. This is not legal advice; ask your counsel how it applies to you.

Can a spreadsheet be our AI inventory?

At the start, yes, if one person owns it and it is reviewed on a fixed date. NIST’s playbook describes an inventory as an organised database of artefacts about each system, such as documentation, incident response plans and contacts. Move it to version control or a platform when it changes faster than its owner can keep it accurate.

What is the difference between an AI governance platform and an AI observability tool?

A governance platform is built around the register, the reviews and the evidence. An observability tool is built around live telemetry: traces, cost, latency and quality scores. Some products do both. Buy for the gap you have: if nobody knows what AI is in use, start with the register; if you know and cannot see how it behaves, start with monitoring.

Who should own AI governance software?

One named role in risk, compliance or technology owns the platform and the register. Each AI system’s owner keeps its entry and its evidence current. A platform owned by a committee tends to hold a register nobody updates.

Further

AI governance implementation · The engagement that builds the controls a platform records: identity, verifier gates, evaluation in CI and audit trails.
AI governance framework · An eight-clause AI policy template, each clause mapped to its control, owner and evidence.
Agent proposes, verifier gates · The gating pattern that produces evidence at the source.
AI agents for business · Where agents fit, and the controls to set before one touches a real system.
Engagement file D-04 · Access provisioned from the HR record, with dry runs, an audit trail and a way back from a bad run.

We build these systems for a living. See the engagement files for what that looks like in practice, or write to us if yours is the next one.

Book a 30-minute call tayyeb@1aym.com See the engagement files

Last reviewed 29 September 2026 · 1AYM