Image created by AI

Last week I wrote about a New York bill that would restrict AI systems from providing professional advice in licensed fields. A few readers asked a sharp follow-up question: does the bill apply differently depending on how the AI system was built?

The answer might surprise you, and I wil…

Read more

Image created by AI

Yesterday I wrote about the shift from measuring organizations to measuring AI systems. Today, the New York State Legislature is proving why that shift is urgent.

A bill introduced by Senator Kristen Gonzalez would restrict AI systems from providing what lawmakers call “substantive responses” in fields that require professional licenses. Medicine, law, engineering, psychology, dentistry, nursing, and other regulated professions where incorrect guidance can cause serious harm.

Read that again carefully. The bill does not say “companies must have policies about what their AI says.” It says AI systems must not provide certain types of responses. The subject of the regulation is the machine, not the organization.

This is the shift happening in real time.

What the bill actually does

The proposal draws a line between general information and professional advice. An AI chatbot can share educational content about, say, symptoms of a condition or how a legal process generally works. What it cannot do is cross into substantive guidance that resembles what a licensed professional would provide. It cannot offer what looks like a medical diagnosis, a legal strategy, an engineering recommendation, or a psychological assessment.

The bill also includes a private right of action. That means individuals can sue companies if their AI systems provide restricted guidance. This is not a regulatory slap on the wrist. This is litigation exposure for every company deploying a customer-facing AI system in a licensed domain.

Why this matters beyond New York

If you are thinking “I do not operate in New York, this does not apply to me,” think again.

New York tends to set the template. When New York moved on financial regulation, the rest of the country followed. The same pattern is already forming with AI. Colorado’s AI Act takes effect in 2026. The EU AI Act becomes fully enforceable in August 2026. The NAIC Model Bulletin on AI in insurance has been adopted by 24 states. NYC Local Law 144 already requires bias audits for automated hiring tools.

The direction is clear: regulators are moving from governing organizations that use AI to governing what AI systems actually do. And they are doing it jurisdiction by jurisdiction, which means any company deploying AI across state lines will soon face a patchwork of requirements that all ask the same fundamental question: does your AI system stay within its authorized boundaries?

The measurement problem this creates

Here is where the data scientist in me gets interested.

“Substantive response” is a fuzzy concept. Where exactly does educational information end and professional advice begin? When does a health chatbot cross from sharing general wellness content into offering what could be interpreted as a diagnosis? When does a legal information tool cross from explaining a process into recommending a strategy?

These are not binary questions. They are spectrum questions. And spectrum questions require quantitative measurement, not policy checklists.

Think about what an organization would need to demonstrate under this bill. Not that they have a policy saying “our AI does not give medical advice.” They would need to demonstrate that their AI system actually stays within bounds, consistently, across thousands of interactions, including edge cases where users push the boundaries with creative phrasing.

That is a behavioral measurement problem. You cannot solve it by reading the organization’s policy documents. You solve it by observing what the AI system actually says when real people interact with it. You measure boundary adherence: how often does the system recognize when it is approaching a restricted domain, and how reliably does it pull back?

This is exactly the kind of observable, quantifiable AI system property that I described yesterday. The policy says the system will not give medical advice. The behavior shows whether it actually does or does not. The gap between those two is where the litigation risk lives.

What this means for different types of AI deployments

The bill applies regardless of how the AI system was built, but the risk profile varies significantly.

Organizations that build their own AI from the ground up have complete control over system prompts, guardrails, and response boundaries. They can engineer precise limits. But they also own 100% of the liability.

Organizations using low-code platforms like Copilot Studio or LangFlow face a shared responsibility problem. The platform provides underlying model behavior and some guardrails, but the builder configures the use case and the domain scope. When the system drifts into professional advice territory, who is liable? The platform or the builder?

And then there are the no-code deployments, the custom GPTs, the drag-and-drop chatbot builders. This is the highest risk category, and it is not close. The people building on these platforms are often the exact professionals the bill is trying to protect: small healthcare clinics, law offices, dental practices. They deploy an AI chatbot on their website, feed it their documents, and assume the platform handles compliance. It usually does not.

The gap between how easy it is to deploy AI and how hard it is to govern what it says is widest in the no-code tier. And that gap is exactly where this bill’s private right of action will land hardest.

The deeper signal

Step back from the specifics of this one bill and look at what it represents.

For decades, professional licensing has been a human-to-human regulatory framework. A doctor is licensed. A lawyer passes the bar. An engineer gets certified. The license attaches to the person, and the person is accountable for what they say.

AI breaks that model. The chatbot giving health guidance is not a licensed professional. It is not a person. It cannot be sued, sanctioned, or stripped of credentials. So the regulatory framework has to evolve. It has to attach accountability to the system’s behavior and to the entity that deployed it.

This bill is one of the first attempts to do that explicitly. It will not be the last. And every attempt will come back to the same core question: can you prove, with data, that your AI system behaves within its authorized boundaries?

That is not a policy question. That is a measurement question. And it demands the kind of quantitative, reproducible, behavior-based measurement that this newsletter exists to explore.

More next Tuesday.

This is part of the “Before The Number” series at A.I.N.S.T.E.I.N., exploring what it takes to build quantitative AI governance measurement from first principles. If this resonated, share it with someone deploying AI in healthcare, legal, or any licensed profession.

Image created by AI

I owe you an explanation for the silence.

Two weeks ago, I published “How to Measure AI Governance” and laid out the five pillars, the metrics, the frameworks, the KPIs. I meant every word of it. And then I went quiet, because something broke in my own thinking that I could not write aro…

Read more

Image by AI

If you cannot measure it, you cannot govern it.

That principle holds true across every regulated industry, from finance to healthcare to cybersecurity, and it holds true for AI.

Yet when most organizations talk about AI governance today, they are talking about policies, principles, and frameworks. They are talking about what they believe, not what they can prove. And there is a meaningful difference between having an AI ethics policy and being able to demonstrate, with data, that your AI systems are actually governed.

This essay is a practical guide to bridging that gap. It walks through the core pillars of AI governance measurement, the specific metrics that matter, the frameworks available to structure the work, and the challenges that make this harder than it sounds. If you are a CISO, a Chief AI Officer, a compliance leader, or a founder building in this space, this is the foundation you need.

Why Measurement Matters

Without measurement, AI governance is a set of intentions. It lives in documents that get written once and reviewed quarterly at best. It gives leadership a sense of comfort without giving them a basis for action.

Measurement changes that in four concrete ways.

1. It demonstrates due diligence. When regulators, boards, or the public ask how your AI is managed, measurement gives you evidence rather than assurances.

2. It allows you to identify and mitigate risks before they cause harm, because metrics like model drift detection time and fairness deviation surface problems that narrative assessments miss entirely.

3. It prepares you for the regulatory compliance landscape that is already here, with the EU AI Act requiring specific documentation and measurement for high-risk AI systems.

4. And it builds trust with every stakeholder who needs to know that your AI decisions are fair, transparent, and accountable.

The Five Pillars of AI Governance Measurement

Effective AI governance measurement is not about tracking model accuracy or inference speed. Those are performance metrics. Governance measurement focuses on accountability, fairness, transparency, compliance, and safety. These are the five pillars, and each one requires its own set of metrics.

Accountability and Ownership

This pillar measures who is responsible for your AI systems and their outcomes. It sounds basic, but in my experience, a surprising number of organizations deploy AI systems where no single person owns the governance risk. The model was built by one team, deployed by another, and monitored by no one in particular.

The qualitative goal is straightforward: every high-risk AI system should have a named business owner who is accountable for its impact. The quantitative metric that tracks this is the percentage of deployed AI systems with a defined, documented business owner. If that number is below 100% for your high-risk systems, you have a governance gap that no policy document can close.

Transparency and Explainability

This pillar measures how well your AI system’s decisions can be understood by the humans affected by them. A lending model that denies an application needs to be able to explain why. A hiring algorithm that filters out candidates needs to produce a reason that a human can evaluate.

The quantitative metric here is the percentage of AI-driven decisions that include a human-interpretable explanation. In practice, this is one of the hardest metrics to improve because many complex models, particularly large language models, are inherently difficult to explain. But the measurement itself forces the conversation about where explainability gaps exist and how material those gaps are.

Fairness and Bias Mitigation

This pillar measures the extent to which your AI systems treat different demographic groups equitably. It is not enough to say “we care about fairness.” You need to measure the actual disparity in outcomes across protected groups and track that disparity over time.

The core metric is the measurable difference in approval rates, error rates, or outcomes between demographic groups. If your lending model approves 78% of applications from one group and 61% from another, that disparity is your fairness metric, and it needs to be monitored continuously, not just checked once before deployment.

Risk and Compliance

This pillar measures adherence to both internal policies and external regulations. With the EU AI Act, NIST AI RMF, and ISO 42001 all converging on requirements for risk classification and documentation, this pillar is becoming the most operationally urgent.

The key metrics include the percentage of high-risk AI systems that have completed an Algorithmic Impact Assessment, the percentage of inventoried systems that have undergone formalized risk classification, and the policy adherence rate across all AI projects. These numbers tell you whether your governance framework is actually being followed or whether it exists only on paper.

Safety and Security

This pillar measures your AI system’s resilience against attacks, errors, and unintended harm. It includes incident response readiness and the speed at which AI-specific failures are detected and resolved.

The metrics that matter here are the average time to detect and time to resolve AI-related incidents, including model drift, toxic output, adversarial attacks, and data pipeline failures. If your organization cannot tell you how long it takes to detect when a model has drifted from its intended behavior, your safety posture has a blind spot.

Key Performance Indicators for AI Governance

Beyond the five pillars, there are specific KPIs that give leadership a clear picture of governance health across the organization.

Program health metrics include AI inventory coverage (the percentage of all AI systems currently cataloged), risk classification completion (the percentage of inventoried systems that have been formally classified by risk level), and policy adherence rate (the percentage of AI projects fully compliant with established guidelines).

Decision and accountability metrics include decision latency for risk issues (how long it takes to make a material decision on an escalated AI risk), human override rate (how frequently automated decisions are reversed by human reviewers), and governance debt (the number of deferred governance controls that were postponed to speed up deployment).

Operational integrity metrics include model drift detection time, data lineage visibility (the percentage of models with full source-to-sink tracking), and audit readiness score (the percentage of models with current documentation and version control).

Ethical impact metrics include explanation coverage and fairness deviation, both of which I discussed in the pillars section above.

The important thing about these KPIs is that they are specific, measurable, and tied to real governance risk. They are not opinions. They are not traffic lights. They are numbers that a board can track quarter over quarter and that an auditor can verify independently.

The Frameworks That Structure This Work

Organizations do not need to build their measurement approach from scratch. Several established frameworks provide the structure.

The NIST AI Risk Management Framework provides guidelines for managing risks to improve the trustworthiness of AI systems. NIST has also recently released a preliminary draft Cyber AI Profile (NISTIR 8596) that maps AI considerations directly onto the Cybersecurity Framework 2.0, embedding AI governance into operational security infrastructure rather than treating it as a separate discipline.

ISO/IEC 42001 is an international standard specifying requirements for establishing, implementing, maintaining, and continually improving an AI management system. As an ISO 42001 Lead Auditor, I work with this framework regularly, and its strength is that it provides a certifiable standard that organizations can be audited against.

The EU AI Act is the most comprehensive regulatory framework currently in effect, requiring specific measurement and documentation for high-risk AI systems. It is not optional for organizations operating in or selling into the European market, and its requirements are driving measurement adoption globally.

These frameworks tell you what to measure and why. The challenge is translating their requirements into the specific quantitative metrics I described above, and doing so continuously rather than at a single point in time.

The Challenges That Make This Hard

If measuring AI governance were easy, every organization would already be doing it. Several factors make it genuinely difficult.

Concepts like fairness and transparency are contextually dependent. What counts as fair in a lending model may differ from what counts as fair in a hiring algorithm. There is no single universal formula, and measurement requires thoughtful interpretation alongside the numbers.

Many complex AI models, particularly large language models, are inherently difficult to explain. This makes transparency measurement challenging not because the metric is wrong but because the underlying system resists the measurement.

Standardization is still evolving. While frameworks exist, universally accepted methods for calculating specific metrics like bias are not yet settled. Different tools and approaches can produce different results for the same system.

Organizations have historically incentivized performance over responsibility. Accuracy and speed get rewarded. Governance measurement introduces a different set of priorities, and that cultural shift is often harder than the technical implementation.

And finally, data quality and lineage remain fundamental obstacles. You cannot measure governance properly if you do not understand the data your AI systems are trained on, and many organizations have complex or undocumented data flows that make this difficult.

Where This Is Heading

Every one of these challenges is real, and none of them are reasons to avoid measurement. They are reasons to invest in building the measurement infrastructure now, before regulators require it and before the gap between what your organization claims about its AI governance and what it can actually prove becomes a liability.

The organizations that solve the measurement problem first will not just be compliant. They will set the standard that others measure against. They will have the data to report to boards, the benchmarks to negotiate with partners, and the scores to prove what checklists never could.

AI governance measurement is not a nice-to-have. It is the infrastructure that makes governance real.

Image created by AI

Every critical system in human history eventually got measured. Before FICO, loan officers decided your creditworthiness with a handshake and a gut feeling. Same income, same history, approved at one branch and denied at another. Before s…

Read more