Image created with AI tools

I want to start with something I noticed, not something I believe.

In rooms with executives, I have watched the same thing happen more than once. These are people who measure with real discipline. They can tell you their cost per transaction, their productivity per team, their adoption curve, their uptime to the decimal. Measurement is not foreign to them. It is the native language of how they run things.

Then I ask one question. How do you measure whether your organization actually trusts its AI?

And the room goes quiet.

Not a guilty quiet. Not an accusation landing. Just a pause, the particular silence of a question that has never been asked in that form before. Someone will eventually offer that they run model accuracy reports, or that they have a governance committee, or that there is an audit somewhere. All true. None of it an answer to the question I asked. The question was about trust, and there was no number to reach for.

I am not going to pretend that silence proves anything. One observation is not a finding. But it made me curious, and curiosity is a better place to begin than conviction.

The thing that made me curious

Here is what puzzled me about that silence.

We spend enormous effort measuring what our AI systems do. Their accuracy, their latency, their throughput, their cost. Whole dashboards exist for it. But we spend almost nothing measuring the thing that actually determines whether any of that effort matters: the confidence with which human beings decide to rely on the system. Whether they lean on it, or quietly work around it. Whether they trust it.

That struck me as odd. Because trust is not a soft afterthought in these systems. It is the hinge everything turns on. A model can be brilliant and go unused because no one believes it. A model can be mediocre and be over-trusted into a disaster. The performance of the model and the trust placed in it are two different quantities, and only one of them shows up on the dashboard.

So the question I could not put down was simply: why? Why do we measure the machine so carefully and the reliance on it not at all? Not because anyone decided trust was unimportant. Everyone says the opposite. So why the gap between what we say matters and what we actually measure?

A lens, from an unlikely place

I found a useful way to think about that gap in a book most people read for other reasons.

John Doerr popularized a deceptively simple idea in Measure What Matters. On the surface it is a book about OKRs, about goals and key results. But the deeper idea underneath the framework is this: OKRs were never really about metrics. They were a mechanism for making invisible priorities visible. The act of measuring something is the act of declaring, publicly and to yourself, that it matters. And the reverse is quietly true as well. What you never measure, you have treated as if it does not matter, whatever you say about it in your principles.

That reframed my question with some force. If trust is what decides whether an AI system gets adopted, questioned, overridden, or abandoned, then by any reasonable account it matters as much as anything on the dashboard. So why is it still handled as an intuition, a feeling in the room, rather than as something observable? Not because it is unimportant. Perhaps because we have not known how to see it.

Trust behaves like a latent variable

Here is the shift that, for me, changed the whole problem. And it is a small shift in language that turns out to be a large shift in method.

The instinct is to say trust should be measurable and go looking for the meter. But that is the wrong sentence. The right one is this: trust behaves like a latent variable.

That phrase comes from the sciences that have spent the longest wrestling with exactly this difficulty, the difficulty of studying something real that you cannot observe directly. And once you say it that way, the problem stops being strange and becomes familiar, because it is a problem those fields solved long ago.

Think about what we already measure this way, without blinking. Intelligence is never observed directly; it is inferred from performance across many tasks. Depression is never observed directly; it is inferred from a pattern of signs. The same is true of economic confidence, of quality of life, of customer satisfaction. Not one of these is read off an instrument. Every one of them is real, consequential, and studied rigorously. And every one of them is inferred from things you can see, rather than measured head-on.

Trust belongs in that family. You will never point a sensor at trust and get a reading. But that has never been how latent things are measured. They are measured through their traces.

What would trust leave behind?

So I stopped asking how to measure trust, and started asking the question a researcher would ask instead: if trust cannot be seen directly, what visible evidence would it leave behind?

This is the move that turns a slogan into science. You do not measure the hidden thing. You ask what the hidden thing causes, and you measure that.

So suppose trust in an AI system rises. What should we expect to see? People should override the system less often when it is right, and appropriately when it is wrong. Escalations should shift in character. The rate at which recommendations are accepted should change. The time spent re-verifying the system’s output should fall. Appeals, interventions, audit exceptions, all of these should move in patterns consistent with a change in reliance. And if trust falls, the same signals should move the other way.

Notice what has happened. Without ever claiming to measure trust directly, I now have a list of things I can observe: override rate, escalation frequency, recommendation acceptance, time to verification, human intervention, appeal rate, audit exceptions. These are not trust. They are the behavioral exhaust of trust, the visible residue a hidden quantity leaves in the operational record. And measuring the consequences of a latent variable is precisely how latent variables have always been studied.

I want to be careful here, because this is where rigor matters and where it is easy to overreach. Any single one of these signals can move for reasons that have nothing to do with trust. That is exactly why you do not lean on any single one. You look at the pattern across many, the way a diagnostician reads a panel of markers rather than a single number, and you reason from the pattern to the hidden state underneath. The mathematics for doing this well, for inferring a hidden quantity from noisy signals and for tracking how it moves over time, is mature and unglamorous and mostly borrowed from fields that have done it for decades. It does not need to be on the surface of this essay. It only needs to exist, and it does.

One more thing about trust: it does not hold still

There is a wrinkle that makes this harder than the classic textbook version, and it is worth naming because it changes what “measuring” even means.

Intelligence, once measured in an adult, is fairly stable. Trust is not. Trust in an AI system moves, and it moves because the world the system operates in moves. The data shifts. The users adapt. The conditions the system was validated under quietly stop holding. So trust is not only a hidden quantity; it is a hidden quantity in motion.

And that has a consequence I find genuinely important. If trust moves, then measuring it once tells you almost nothing about where it will be next quarter. A single reading is a photograph of a moving thing. The same measured value can mean opposite realities depending on the direction it arrived from: a system whose trust is recovering and one whose trust is collapsing can pass through the identical number on the way to very different fates. Only a sequence of readings, a trajectory, can tell them apart. Which suggests that if trust is worth measuring at all, it is worth measuring continuously, not certified once and filed away.

Why this matters to the person accountable

Let me connect this back to the executive in the quiet room, because this is not an academic puzzle for them. It is a future liability.

When an AI system fails, and eventually one will, the questions that arrive are all retrospective. Was this system trustworthy? Were you watching? What state was it in, and when did you know? These are questions about a quantity over time. And they can only be answered if that quantity was being observed as it moved, because a trajectory cannot be reconstructed after the fact. You cannot go back and take a measurement you never took. The evidence either exists because someone was collecting it all along, or it does not exist at all.

That reframes the quiet in the room. The silence was not a small gap in reporting. It was the absence of the one record that, on the hardest day, everyone will reach for and no one will find.

Where this is taking me

I will be honest that this line of thinking is a question I am still inside of, not a conclusion I am handing over. But it has led me somewhere specific, and I will name the direction even though I am not claiming the destination.

The question I keep returning to is whether executive trust in AI can be treated as a measurable construct, operationalized not through surveys and self-report alone, but through the behavioral, operational, governance, and outcome signals that a system generates as it runs, tracked over time as they evolve. Not to reduce trust to a single tidy number, but to make a hidden and consequential thing visible enough to reason about, argue over, and act on before a failure rather than after.

I do not think the next real advance in AI governance will come from larger models. I think it may come from better measurement. Not measurement of what the machine can do, we are already good at that, but measurement of the thing we keep saying matters most and keep leaving off the dashboard.

So I will end where a researcher should end, with the question rather than the answer.

Share

If trust is what matters most, what evidence should organizations be collecting, starting now, to know whether they actually have it?

I do not think we have a good answer yet. I think it is the right question. And I would rather leave you holding a better question than a borrowed conviction.

Until next Tuesday….



Suneeta Modekurty
Founder & CEO | SANJEEVANI AI

Image created by AI tools

I was invited recently to speak with a room of product leaders and senior professionals as part of AI in Motion, an initiative built to bring practitioners face to face with the harder questions AI is forcing on all of us. I had prepared to talk about something I have been circling f…

Read more

Image created using AI tools

There is a moment in every AI deployment that nobody schedules and nobody sees coming. It is the moment the system does something no one asked it to do, in a place no one realized it could reach, and the organization discovers, all at once, that it never actually knew what it had …

Read more

Image created using AI tools

On a Monday in June, Anthropic released the most capable model it had ever built. By Friday evening, it was gone. Not slowed, not limited to certain regions. Gone, for everyone, everywhere.

I have been sitting with that all weekend, and I want to share where my thinking landed, because I don’t think the lesson is the one the headlines are reaching for.

Here is what happened, told simply. Anthropic released Fable 5, and the model beneath it, Mythos 5. Teams across the industry spent the week building it into real work. Then the U.S. Commerce Department issued an export control directive: these models could not be used by any foreign national, anywhere, including foreign-born employees working inside U.S. offices (and you can imagine why it matters to me, and to you, if you are foreign-born). There was no clean way to honor that order in real time, so the only path to compliance was to switch the models off for everyone. And so they did. The most capable thing on the market went dark on a Friday night, because of a decision no customer was part of.

I want to be careful with the facts, because this story is being told in several versions, and some carry more confidence than the record supports. The verified spine is narrow. A government directive, citing national security, tied to the model’s cyber capabilities and a jailbreak technique. The company disagrees, calls it a misunderstanding, and says it is working to restore access. The more dramatic threads, who called whom, who set it in motion, are still unsettled. I would rather build on what is solid than on what is satisfying.

What is solid is enough to learn from.

This fits in an arc

Every infrastructure we now take for granted went through the same passage. There was a season when it was new and astonishing, and we built on it before we fully understood what we were leaning on.

Be it Electricity, the early eb or the Cloud.

Each one moved from “remarkable capability” to “thing we assume is simply there.” And each one, somewhere along that path, had a moment that reminded everyone it was not as solid as it felt.

Frontier models are early in that arc. They are astonishing, they are everywhere, and we have started to treat them the way we treat electricity. Something that is just on. Friday was a reminder that we are not there yet. The most capable models are still the newest part of the stack, and the newest part is always the part most exposed to forces that have nothing to do with engineering.

That is not a criticism of any one company. The same structure would hold for any frontier model from any provider. It is simply where we are in the arc.

A layer above

When a company adopts a model, it negotiates the things it knows how to negotiate, namely, uptime, latency, committed spend, data handling, support etc. Those terms feel like the boundaries of the relationship, and most of the time they are.

What this week showed is that there is a layer sitting above all of that and that is regulatory, geopolitical, national. And that layer does not read your service agreement. When it moves, the terms you negotiated still hold, they just stop being the thing that decides the outcome. The capability can be withdrawn by a force you have no seat across from.

This is worth naming plainly as something the field is now ready to see clearly.

We have understood model risk mostly as “will it perform.” This week added a second dimension: “will it remain available to me, and who decides that.” Both are real. We have simply been measuring one and assuming the other.

Why “it will come back” is not the lesson

It is tempting to wait this out. Access may well be restored. The disagreement may resolve, and the model may return as if the week never happened. And then the easy move is to file it under “strange few days” and carry on.

It’s worth gently resisting that. The specific outcome is not the lesson. The lesson is that the event was possible at all. A capability that hundreds of millions of people were using could go to zero overnight, lawfully, with no warning. Whether it stays gone for three days or three months does not change what it taught us about the shape of the risk. A thing that can happen once on a Friday can happen again on a Tuesday.

The narrow lesson is “this model carried risk.” The useful lesson is “leaning on any single model this completely carries risk.” One sends you shopping for a different vendor. The other invites you to look honestly at how you have built.

A question worth pondering over

So here is the question I have been thinking over, and I offer it as a genuine prompt rather than a rhetorical one.

If your primary model went dark this Friday, not slower, dark, what happens on Monday?

Picture it concretely. Which workflows stop. Which customers notice. How long before something else carries the load, and how much harder is it carrying. Do you know the answer already, or is it something you would discover in real time, under pressure, the way teams often discover their backups only when they finally reach for them.

Most organizations cannot answer this cleanly yet, and I don’t think that reflects poorly on anyone. The question simply was not urgent until this week. The capability was too good and too available to imagine it absent. That is exactly the condition under which dependence gathers quietly, when the thing we depend on feels too solid to question.

What I think resilience actually asks of us

I don’t believe the answer is fear, or stepping back from frontier models, or treating capability as a liability. Fear and stepping back is never a solution to any problem. The capability is real and the value is real. The answer is quieter than that, and frankly harder. It is knowing your own exposure well enough to act on it before the moment forces your hand.

That means being able to say, with specifics, where you are concentrated. It means having a fallback you have actually tested, not one you assume will be there. It means treating “what is our dependency profile” as a question you can answer at any time, rather than one you reconstruct in a hurry. None of this is dramatic. It is the difference between an organization that has a plan and one that becomes the plan, improvised live, while the screens turn red.

In its own backhanded way, Friday was a gift. It took a quiet, structural risk and made it loud, and dated, and undeniable, at a cost that for most of us belonged to someone else. That is the least expensive way to learn an expensive lesson. The expensive way is to learn it when the model that goes dark is the one your own business is standing on.

The model came back, or it will. The question it asked is going to stay with us a while longer.

What is your fallback plan if your primary model goes dark on a Friday? I would truly like to know how you are thinking about it. Reply and tell me.

Until next week…

Founder and CEO | SANJEEVANI AI

Image created using AI tools

One of the most dangerous AI failures inside companies today does not start as a failure. It starts as efficiency.

You put a predictive model next to a decision. It is fast. It is cheap. It is right often enough. So you lean on it a little more each week.

Then something quiet happens. The…

Read more

Image created with AI tools

In September 2025, Starbucks released a short promotional video to celebrate the launch of an AI-powered inventory system that the company had built in partnership with the computer vision startup NomadGo. The tool was positioned as a centerpiece of CEO Brian Niccol’s “Back to Starbucks” turnaround, a strategy designed to address the persistent product shortages that had been quietly eroding customer experience and same-store sales for years. The video showed a barista walking through a back room with a handheld tablet, sweeping the camera across shelves of milks and syrups while the system tagged and counted each item in real time. In one of the frames of that very video, the system failed to register a bottle of peppermint syrup that was sitting on the shelf in plain view, surrounded by other bottles that the system did correctly identify. Nobody at Starbucks caught the error before the video shipped, the blog post went live, and the rollout began across more than eleven thousand North American stores.

Eight months later, in May 2026, Starbucks quietly retired the system across the entire fleet, the original blog post was deleted from the corporate site, and the promotional video was pulled from circulation. The story made the trade press as a tidy AI failure narrative, with most coverage focusing on the technical embarrassment of a system that could not reliably distinguish oat milk from regular milk. What almost none of the coverage took seriously was the workforce story sitting underneath the technical one, and I want to tell that part here because I think it is the part that actually matters for every leader trying to navigate this moment.

The Starbucks partners, the workers on the floor who interact with these systems every day, did not trust the AI from very early in the rollout. They scanned the shelves with the tablet because they were required to, they watched the system return its counts, and then they quietly recounted by hand and entered the corrected numbers into the inventory log. They protected the stores from the AI’s errors for nine months by absorbing the extra workload onto themselves, working a second shift of verification on top of the work they were already doing. They are the reason the failures stayed mostly invisible to customers and to corporate leadership until the system was finally pulled, and they are also, in the same breath, the people the broader industry narrative is currently lining up to lay off in the name of AI productivity. That contradiction sits at the center of the AI adoption wave, and I want to name it as clearly as I can in what follows.

The Wrong Frame

The dominant story in the current AI moment is that humans and machines are competing for the same jobs, with the AI either taking the job from the human or the human keeping the job until the AI is good enough to take it. Either way, the relationship is framed as a contest with one winner and one loser, and the entire vocabulary of “AI productivity” has been built on top of that framing. This frame is wrong both technically and organizationally, and the Starbucks case shows you exactly why with a level of clarity that few cases offer.

The AI inventory tool did not do the partners’ job, even on the days when it worked. It did one slice of the partners’ job, which was the visual identification of products on a shelf, and it did that slice badly under the operational conditions of a real back room. The other slices of the job, including knowing yesterday’s shipment patterns, knowing which products burned through fastest during the morning rush, knowing which carton was tucked behind another because the closing shift had restocked in a hurry, and owning final accountability for the count when the day’s numbers were reconciled, were never part of what the AI could do and were never part of what the system was designed to do. Those slices remained with the human partner, who now held less authority over the visible counting work and the same responsibility for the eventual outcome, which is the worst possible position to put a worker in if you actually want the system to succeed.

This is not automation in any meaningful sense of the word. This is cognitive load transfer dressed up in the vocabulary of cost reduction, where the easy and legible part of a job gets handed to a machine while the hard and illegible parts get concentrated onto fewer remaining people who now have less context, less authority, and less protection from the consequences of the machine’s mistakes. The conversation we should be having is not whether to lay off the inventory counters because the AI can count, but rather what the expanded human role looks like around a partially automated process, and how many people we need in that expanded role for the system as a whole to actually work. That question is almost never asked in the rooms where AI procurement decisions get made, and so I want to ask it here with as much specificity as I can.

What If: A Manual Monte Carlo

Before I lay out the scenarios, let me say what I mean by a manual Monte Carlo. In quantitative finance and in operations research, a Monte Carlo simulation is a method of exploring possible futures by running the same model thousands of times with different inputs and watching the distribution of outcomes that emerges from the variability. You cannot do this on paper at scale, but you can do a slower and more deliberate version of the same thinking by hand, by asking what-if questions one at a time and walking each scenario out to its consequences before deciding which path to actually take. This is the kind of exercise that every leadership team should be running before they sign an AI contract, and the absence of this exercise is one of the most consistent features of the AI failures we have seen so far.

Image generated using python code by the author

Let me run five of these scenarios for the Starbucks case, not as predictions and not as numbers I could defend inside a formal model, but as the kind of structured thinking that the rollout itself appears to have skipped. I have begun to refer to this approach as a manual Monte Carlo for AI governance decisions, and the five scenarios that follow are the practical instrument I use when thinking about whether a given AI deployment is genuinely ready to ship into the operational world it will actually have to survive in.

The first scenario asks what would have happened if Starbucks had kept every partner on the floor and added a dedicated AI oversight team alongside the rollout. Imagine a team of perhaps fifty to one hundred people whose only job is to audit AI output against ground truth across the eleven thousand store fleet, sampling stores on a rotating schedule, reviewing confidence score distributions, identifying model drift before it becomes catastrophic, and feeding human corrections back into a retraining pipeline that updates the model on a regular cadence. The annual cost of such a team would be real, perhaps in the range of five to ten million dollars when fully loaded with engineering support and infrastructure, and it would be dwarfed by the cost of a failed nine-month rollout across eleven thousand locations, even before you begin to count the brand damage, the deleted blog post, the loss of internal credibility for the next AI initiative, and the harder-to-measure erosion of trust between partners and the corporate technology function. The deeper point, the one that matters more than the headcount or the budget, is that an oversight team of this kind functions as a forcing function for everything else the deployment should have included from the start, because the very existence of a paid oversight function makes engineering hygiene mandatory rather than optional. Without a budget line for oversight, the engineering team naturally optimizes for the demo and the ship date, since nobody inside the company is being paid to demand the audit trail, the retraining cadence, the rollback authority, or the operational testing protocol. With a budget line for oversight, the team’s first ninety days of work would surface exactly those missing pieces and would force the rest of the organization to build them, because the oversight function cannot do its own job without them. My own view, which I hold with some confidence after watching enough of these rollouts up close, is that this first scenario is the single highest-leverage intervention of the five I am about to walk through, because it creates the institutional pressure that pulls the other four into place behind it almost automatically. If Starbucks had committed to this one structural choice alone and had held the rest of the rollout to whatever pace the oversight team could actually validate, I believe the system would still be running today in a meaningfully better form than it ever achieved during its actual nine-month life.

The second scenario asks what would have happened if the partners themselves had been trained to interpret the AI’s confidence scores rather than being shown only the final classification. A two-day training program could have taught every partner in every store what a confidence score is, what it means when the tablet reports that an item was identified with 0.62 confidence versus 0.94 confidence, and how to decide whether to trust the AI’s count or recount manually based on that signal. The system would stop pretending to be certain when it was not, the partner would have a clear and defensible rule for when to intervene, the second shift of silent verification would become a deliberate and trained workflow rather than a hidden burden, and the partner would emerge from the deployment more skilled and more valuable to the company than they had been before it began. The analysis that needs to be done honestly here, however, is the one that asks whether the training alone would have changed actual behavior in the morning rush of a real store, and my view is that it would not have, at least not by itself, because reading a confidence score does no good if the worker has no operational permission to act on it. If the partner is still being measured on completing back-of-house tasks within a fixed time window, if the manager is still scheduling the shift around an assumption that inventory takes thirty minutes rather than the forty-five minutes it would take with proper verification, and if the performance review structure still rewards speed over accuracy, then the partner will see the 0.62 confidence score and will accept the count anyway, because slowing down to recount is a behavior the organization has not actually made room for. The training has to be paired with management metrics that explicitly reward verification over speed, and with shift scheduling that accommodates the time verification requires, otherwise the training becomes a polite fiction layered on top of an unchanged operational system. My opinion on this scenario is that it is useful as a complement to the first one but dangerous as a standalone fix, because it places the cognitive burden of catching AI errors back onto the worker without giving them the time, the authority, or the metric structure to discharge it well, and the worker is left in essentially the same position they were already in during the actual rollout, except now they are also responsible for understanding a confidence score on top of everything else they were already doing silently.

The third scenario asks what would have happened if every AI rollout in the company had been paired with a human oversight structure scaled to the size of the deployment, in the way that every other safety-critical system in the modern economy is paired with its corresponding human oversight function. Airplanes have pilots in the cockpit and air traffic controllers in the tower, power grids have local operators and regional supervisors who coordinate across the network, hospitals have clinicians at the bedside and quality review boards in the institutional governance layer, and each of these structures exists because we have learned over decades that complex technical systems do not stay safe or accurate without continuous human attention to their behavior. AI deployment, with stakes that increasingly rival these other systems, currently ships without any analogous oversight function in the vast majority of cases, not because the need is unknown but because nobody put a line item for it in the original procurement proposal and no buyer thought to ask why it was missing. The historical analysis that needs to be done honestly here is that each of these oversight structures came into existence reactively rather than proactively, after the industry had paid in lives or in losses for not having the structure already in place, with the Tenerife runway collision shaping the modern shape of air traffic control, the Three Mile Island incident shaping the modern shape of nuclear plant operations, and the Institute of Medicine reports of the late 1990s shaping the modern shape of hospital quality and safety. AI as a category of technology is currently in what I would describe as the pre-Tenerife phase of its lifecycle, by which I mean that the catastrophic failure which will eventually mandate the oversight structure has not yet happened in a form severe enough to force regulatory action across the entire industry, and the rollouts of the next two to three years are quietly building toward the kind of incident that will produce that mandate. My opinion on this scenario, and the one I would most want a board of directors to sit with for the longest time, is that the oversight structure is historically inevitable and that the only meaningful executive choice available right now is whether to build it voluntarily and proactively with room to design it thoughtfully and capture its benefits, or to be forced into building it reactively later under regulatory pressure and after the company has already become the cautionary tale that other companies study. The executives who choose voluntarily now will be the ones cited in five years as the responsible early operators of this technology, and the ones who skip it will be in the case study deck that the next generation of business school students will be required to read.

The fourth scenario asks what would have happened if the workforce story had been told to the partners and to the public in honest terms from the start. Imagine the announcement reading something like this: we are deploying AI in our back rooms, we are not reducing headcount in connection with this rollout, we are expanding the role of every partner to include AI verification and feedback, and we are hiring a new class of AI quality engineers to support that expanded role. This is the story that most companies are unwilling to tell because it does not fit the cost-cutting deck the CFO needs for the next earnings call, but it is also the story that would have given the partners a reason to be invested in the system’s success rather than skeptical of its motives, and it would have surfaced the operational realities of the back room into the design process months before the system shipped. The harder analysis that has to be done honestly here is the one that asks why no executive is currently telling this story even when they privately believe it, and my view is that the answer is not a personal failing on the part of the executives but rather a coordination problem at the level of the market itself. The current investor narrative around AI rewards companies that signal headcount reduction in connection with AI deployment, regardless of whether the AI is actually replacing the underlying work, which means the chief executive who tells the honest story takes a real stock price hit from the same investors who are funding the AI deployment in the first place. This is the trap that most public company executives are sitting inside right now, and the trap will not release until enough boards begin to realize that the dishonest story is what is producing the Starbucks pattern at the operational layer, and that the operational pattern is what is eventually destroying the value the AI rollout was supposed to create in the first place. My opinion on this scenario is that it is the hardest of the five to execute even when leadership privately wants to do it, because it requires bucking the entire current investor mood at the same moment that the investor mood is funding the AI strategy, and that this scenario will become executable at scale only when enough articles like the one you are reading right now help to seed the realization that is currently missing from the boardroom conversation. The piece in your hands is, in that limited sense, a small contribution to making this fourth scenario eventually possible.

The fifth scenario asks what would have happened if executives had drawn the line at the contract stage rather than after the failure was already visible. No AI deployment should be approved without explicit answers to a small number of governance questions, including who is accountable when the AI is wrong, what the confidence threshold is for autonomous action, what the rollback trigger looks like and who has the authority to pull it, who owns the human oversight team and where they sit in the org chart, what the retraining cadence will be and what the feedback loop is from worker corrections back into the model, and what the operational testing protocol is for the conditions under which the system will actually be used. Most AI contracts today are signed without any of these answers on the table, with the procurement process treating AI as if it were ordering office furniture, and the predictable result is that the answers get discovered the hard way, in production, after the system has already been deployed at scale. The structural analysis that has to be done honestly here is that the reason these questions almost never get asked in procurement is not because the buyer does not care, but because the buyer does not yet have the framework to know which questions are the important ones, the vendor does not volunteer the questions because they extend the sales cycle, and the legal team is checking standard contractual boilerplate rather than AI-specific governance language. Until procurement teams across industries have a shared rubric for AI contract due diligence, every individual deployment will reinvent these questions on its own and most will reinvent them badly or incompletely, which is the pattern we are watching play out in real time across the early enterprise rollouts of this technology. My opinion on this scenario is that it is the place where regulation will eventually arrive, because the market is not going to solve the problem on its own at anything close to the speed at which the technology is being deployed, and we are already seeing the early shape of that regulatory move in places like the recent FDA guidance for the use of AI in pharmaceutical quality systems, where the agency has now stated on the record that the company holds the liability regardless of whether the failure originated in an AI tool or in a human consultant. The executive who treats AI procurement today with the same diligence they would apply to the hiring of a senior consultant or to the qualification of a critical supplier is doing voluntarily what they will very soon be required to do anyway by their regulators, by their auditors, and by their insurers, and the gap between the voluntary adopters and the forced adopters is going to be one of the defining competitive lines of the next three years.

These five scenarios are not equally easy to execute, as the analysis I have walked through tries to make plain, and I want to be honest about the fact that the first one is the structural lever that pulls the others into place, the second one is dangerous in isolation though valuable as a complement, the third one is historically inevitable and the only real question is voluntary or forced, the fourth one is the hardest cultural lift because it requires bucking the investor narrative at the moment that narrative is funding the deployment, and the fifth one is where regulation will eventually arrive whether the industry is ready for it or not. What is shared across all five, and what makes them worth thinking about together rather than choosing among in isolation, is that each of them costs more in the short term than the path Starbucks actually took, and each of them, when you sit with the math honestly, costs far less in the long term and produces an AI deployment that actually achieves the operational outcome the technology was bought for, without quietly transferring the failure modes onto the workforce the company is simultaneously planning to lay off. The math is not the obstacle here. The thinking is.

The Roles, Properly Defined

The framing I want to argue for, and the framing that should sit at the center of every AI deployment decision going forward, is that humans and machines do not compete for the same role in a well-designed system because they are structurally suited to different roles. Machines are extremely good at pattern recognition across very large numbers of inputs, at repetition without fatigue across long time horizons, at computation that exceeds human speed by many orders of magnitude, at consistency on inputs that resemble the data they were trained on, and at operating across all twenty-four hours of the day without the breaks that human bodies require. These are real and valuable capabilities and there is no honest case for ignoring them in modern operations. Humans, in turn, are extremely good at integrating context that the machine never saw and could not have seen, at exercising judgment on novel situations that fall outside the training distribution, at maintaining calibrated awareness of their own uncertainty and communicating that uncertainty honestly to others, at making ethical decisions where the stakes are not encodable in a loss function, at holding accountability when the system errs in ways that affect real customers and real outcomes, at remembering past failures and the causes that produced them so that the same failure does not happen again, and at the deeply skilled work of training and supervising and correcting the machines themselves.

Notice what happens to the human role in a system that takes both lists seriously. As the machines do more of the work they are well suited for, the human role does not shrink, it changes shape and in most cases becomes more demanding rather than less. The human becomes the ringmaster who coordinates the system, the watchman who monitors its behavior, the trainer who feeds it new data and corrects its drift, the verifier who checks its outputs against ground truth, the loop-closer who turns every failure into an institutional learning, and the accountable party who answers for the system when something goes wrong in front of a customer or a regulator. These roles are not less skilled than the work the AI displaced, they are more skilled and more cognitively demanding, and they require deliberate investment in training programs and in headcount budgets and in career paths that did not exist in the pre-AI version of the organization.

If you are deploying AI seriously, you should expect your workforce composition to shift in shape rather than to shrink in size. You will have fewer people doing the specific task the AI now performs, and you will have more people doing the work the AI cannot do, which includes the supervision of the AI itself and the integration of its output into the broader context of the organization’s actual operations. A board of directors that approves an AI deployment with simultaneous layoffs in the same department is not making a productivity decision, regardless of how the decision is dressed for the earnings call. They are telling you plainly that they do not understand what they have just bought, and the cost of that misunderstanding will land somewhere in the organization eventually, usually on the customers and the workers who remain.

Human-Machine Synergy

In October 2024, I published a book titled The AI-Human Synergy: A Data Scientist’s Vision for the Future, in which I argued that the future of work is not a story of humans being replaced by machines but a story of humans and machines entering deliberate and complementary partnership, with each side doing what each is genuinely good at, each side compensating for the other’s blind spots, and each side accountable for the part of the work that only that side can do. The argument was meant to be a corrective to a public conversation that had already begun to slide into a binary framing of replacement versus survival, and it was rooted in the observation, from my own twenty-five years across education and research and applied data science, that the systems which actually work in the real world are almost always the ones designed around this kind of partnership rather than around the fantasy of full automation.

We are not yet living in the world that book describes. We are living instead in a transitional moment where companies are buying AI as if it were a workforce replacement, deploying it as if it were proven, laying off workers as if the deployments had succeeded, and then quietly retiring the systems eighteen months later when the failures become impossible to hide any longer. The Starbucks case is one example of this pattern, and it will not be the last one. There are several more in motion right now that will reach the trade press over the next twelve months, and the underlying dynamics will be recognizable in every one of them once you know what to look for. In the broader work I am leading at SANJEEVANI AI to build quantitative trust measurement infrastructure for enterprise AI deployments, this five-scenario lens is one of the components that feeds into a larger architecture for evaluating whether an organization is genuinely ready for the AI it is in the process of buying.

What I am calling for in this article, and what I think every executive in a position to draw this line should be calling for in their own organizations, is a different posture toward AI deployment that is neither anti-AI nor pro-layoff but rather pro-design, in the sense of designing the human-machine system on purpose rather than allowing it to emerge by accident from the collision of vendor sales decks and CFO spreadsheets. Defining the roles deliberately, building the oversight structure to match the deployment size, training the humans for the new and more demanding roles they are being asked to occupy, and hiring more of them in different roles to do the work the AI cannot do, all of this is the actual work of what I have been calling human-machine amity, and it is harder than firing people because it requires the organization to think clearly about what its own jobs actually are. It is also, and this is the part that should matter to the people who hold the budgets, the only path that delivers the productivity gains the technology is genuinely capable of producing.

The Starbucks partners knew what their job actually was, the AI did not know what the job was because it had never been told and could not have been told within the structure of the deployment as it was designed, and the executives somewhere in the middle of those two ends of the system mistook the visible surface of the work for its actual substance. That is the mistake that is being made at scale, in industry after industry, in the rollouts happening right now in 2026, and it is a mistake that we have every technical and organizational capability to correct if we are willing to do the harder thinking it requires. The next phase of this conversation, and the one I believe the field has to move into over the next two to three years, is not just better thinking about these tradeoffs but the construction of measurable infrastructure that quantifies them before deployment rather than discovering them after, and that direction of work is exactly where I am committing my own time. We can do this differently, and given what is now on the line for workers and for customers and for the credibility of AI as a category of technology, I would argue that we have to.

Read this, ponder it, sit with it, and bring it into the next AI conversation you find yourself in this week, whether that is a board meeting, a procurement call, a hallway conversation with a worker on your team, or a quiet moment of your own thinking. I am writing this as a businesswoman who is building toward a future where AI and humans work in deliberate partnership, not as a critic standing outside the field throwing stones. The companies that get this right over the next three years will be the ones still standing in 2030, and the work of getting it right starts with the kind of structured thinking we have just walked through together. Until next week.

Suneeta Modekurty
Founder, SANJEEVANI AI

Image created using AI tools

When a company lays off nearly a quarter of its staff, the narrative is usually one of retreat. But ClickUp’s recent 22 percent headcount cut was different. It was an aggressive, forward-leaning restructuring designed to answer a single, brutal question: Who deserves to survive in the age of AI?

The people who stayed, the survivors, were not kept because they knew how to write basic prompts or use chat interfaces. They were kept because they possessed a capability that most organizations cannot even define, let alone measure: the ability to direct, audit, and take absolute operational accountability for machine output.

This represents a massive, necessary transition from AI Literacy to AI Readiness.

Most executives treat AI adoption as a soft skill, rolling out licenses and calling “basic prompting” a success metric. But this approach creates a black box. When you cannot measure what your AI tools are actually producing, you cannot enforce accountability. You end up with an unmanageable explosion of automated noise. ClickUp’s restructure is the first industrial-scale proof that AI adoption without structural measurability is a dead end. It is a loud wake-up call for how we define the work of the future.

The Myth of Universal Productivity

In May 2026, ClickUp, a software company valued at $4 billion, announced a 22 percent headcount reduction. But unlike the standard tech layoffs of the past few years, which were framed as defensive, cost-cutting measures, ClickUp’s CEO, Zeb Evans, framed this cut as a proactive, structural bet.

Evans wrote on X that “the business is the strongest it’s ever been” and that “this wasn’t about cutting costs.” The goal was to completely rebuild the company around what he calls a “100x organization.”

To incentivize the transition, ClickUp threw traditional compensation structures out the window, introducing $1 million cash salary bands for the employees who remained, provided they could demonstrate “100x impact” by building and managing AI systems.

The mainstream headline was the layoff. The real story is the structural thesis underneath.

Evans challenged one of the most sacred assumptions of the generative AI era: the idea that AI tools automatically make everyone more productive. He argued the exact opposite.

In a traditional workflow, giving AI tools to junior or untrained staff does not speed up an organization. It creates an explosion of raw volume. Evans pointed specifically to companies celebrating 500 percent increases in pull request volume without matching customer outcomes, calling this “the great reckoning of AI coding.”

But someone has to review that output. Someone has to fix the subtle, confident errors the AI made. When that mass of automated volume collides with senior staff, it creates a massive review bottleneck. The senior people end up spending all their time auditing, rewriting, and debugging AI-generated noise.

As Evans bluntly wrote: “AI makes the best engineers wildly more productive, and everyone else using AI slows these engineers down... More code is just another bottleneck.”

This is the great paradox of the agentic era. When the cost of generating work drops to zero, the cost of evaluating that work becomes the most expensive line item in your business.

And that is where the line between AI literacy and AI readiness is drawn.

AI Literacy vs. AI Readiness

Most corporate training programs are designed to build AI Literacy. They teach employees what a Large Language Model is, how to open a chat interface, and how to write basic prompts. It is cognitive, basic, and tool-centric.

AI Readiness, however, is operational. It is the human capability to direct, judge, and take absolute accountability for automated systems.

When a company runs thousands of AI agents internally, it doesn’t need people who can write. It needs people who can manage. It needs what ClickUp calls Agent Managers: workers who can automate their own manual tasks, build systems around those automations, and act as the rigorous human filter for the machine’s output.

To build an organization of Agent Managers, companies must cultivate three distinct human capabilities.

1. Direction: From Prompting to Orchestration

An AI tool does exactly what you tell it to do. Therefore, the harder and more nuanced the work, the more critical the telling becomes.

The AI-Literate worker prompts: “Write a marketing email for our consulting services.” The result is a generic, instantly deleted block of corporate jargon.

The AI-Ready worker orchestrates: “Write a 200-word email to a small business owner who has heard of AI but fundamentally distrusts it. Open by directly validating that trust barrier, and close with a single, low-friction question they can answer in 30 seconds.”

The core skill here is not writing. The skill is system design. It is the ability to deconstruct a highly complex, intuitive human process, isolate its underlying variables, and translate them into a structured instruction set that an agent can execute flawlessly.

For decades, corporate structures have rewarded people for doing the work. Suddenly, we need people who can describe the work in such vivid, mechanical detail that a machine can replicate it. That is a rare, highly strategic skill. Traditional prompt-engineering workshops do not teach it because it requires deep domain expertise, not just software knowledge.

2. Judgment: The Review Bottleneck

Because AI tools produce highly confident, plausible-sounding answers that are frequently wrong, the ultimate bottleneck is no longer production. It is review.

The readiness skill here is knowing when to trust the output and when to interrogate it. This skill cannot be installed via a corporate slide deck. It is the slow, painful result of having done the manual work yourself, badly, for years.

It is what a senior software architect brings to reviewing AI-generated code, or what a veteran forensic accountant brings to an AI-run reconciliation. They do not look at the 99% that is right. They have the instinct to find the 1% that is catastrophically wrong.

Companies that lay off their expensive, senior experts while keeping only their cheap, “AI-fluent” juniors are going to discover their lack of judgment the hard way.

They will find themselves drowning in flawless-looking, broken work.

AI literacy lets you generate the code. AI readiness gives you the wisdom to realize that code should never go to production.

3. Ownership: The Air Canada Rule

An AI agent does not get fired when a calculation is wrong, and it does not get sued when a client is misled. The human in the chair does.

This is the psychological side of AI readiness that never makes it into vendor product demos. It is the willingness to sign your name to output you did not write, defend it to a client when it is challenged, and accept the professional and legal consequences when it fails.

This isn’t theoretical. In February 2024, the British Columbia Civil Resolution Tribunal ruled in Moffatt v. Air Canada that the airline was liable when its customer-facing chatbot promised a passenger an unauthorized bereavement refund. In its defense, the airline argued that the chatbot was a “separate legal entity” responsible for its own misinformation. The tribunal rejected the argument outright, ruling that the company was entirely accountable for the systems it chose to deploy.

Somebody at Air Canada owned that mistake, whether they wanted to or not.

Most companies have still not defined who legally and operationally owns their AI’s output. They will figure it out the first time a customer pushes back, and the lesson will be incredibly expensive.

The New Architecture of Work

If ClickUp’s thesis is correct, the transition to AI readiness will force a radical redesign of traditional corporate roles. Evans has outlined three distinct classes of ready workers:

The Builders. These are the 10x engineers and product designers. They are no longer typing lines of code or drawing static mockups. They are acting as system architects, directing fleets of agents to build, test, and iterate. They spend their time on system design and validation.

The Agent Managers (Systems Managers). These are operational workers who have successfully automated their own manual tasks. Instead of being displaced, they are kept because they possess the institutional knowledge required to run, monitor, and troubleshoot the automated systems they created. As Evans put it: “The people that automate their jobs with AI will always have a job.”

The Front-liners. These are the human-to-human connection points. In an era where AI can generate infinite digital noise, authentic human contact becomes a premium bottleneck. Under ClickUp’s model, customer-facing humans are explicitly protected from automation. They do not use AI to replace meetings. They use AI to automate everything around the meetings, so they can spend 100% of their time focused on real human relationship-building.

Image created with AI tools

Why Readiness Cannot Be Bought

The pattern repeating across the corporate world is a fundamental confusion of terms.

AI Adoption is technical. It is the work of installing software, integrating APIs, and provisioning licenses. Adoption can be bought. You sign a contract, run a launch event, and the software lands on every laptop in the company.

AI Literacy is cognitive. It is the basic understanding of how these tools work. Literacy can be taught via quick workshops and video modules.

AI Readiness is operational. It is the human capability to establish direction, exercise judgment, and accept ownership over automated workflows. Readiness cannot be bought, and it cannot be taught in a 45-minute webinar. It must be built through deliberate, supervised practice on real-world work over months.

This is why so many pilot programs yield zero measurable ROI. They successfully bought adoption, achieved basic literacy, and completely skipped readiness.

What to Do This Week

If you want to know where your company actually stands on the spectrum of AI readiness, run this simple test on Monday morning.

Pick one AI tool your team currently uses. Select one piece of work that tool produced this week.

Then ask these three questions.

Direction. Who wrote the operational instructions that produced this output, and could a different colleague reproduce this exact quality using those instructions alone?

Judgment. Who audited this specific output, and exactly how do they know it is correct?

Ownership. If this output goes to a client and severely backfires, whose name is on the line?

If you cannot confidently answer all three questions for a single piece of work, your company has adopted a tool, but it has not built readiness. You have active software, but you do not have an active system.

Closing that gap is the defining corporate challenge of the next two years. The organizations that build true AI readiness will look unrecognizable from the inside by the end of the decade. The ones that do not will still be running prompt-writing workshops, staring at empty dashboards, wondering where their ROI went.

Sources

1. Zeb Evans, post on X (May 21, 2026). primary source for the 22% layoff, the “100x organization” framing, and Evans’s direct quotes.

2. Fortune, “Outnumbered: At $4 billion ClickUp, a 3:1 agent-to-human ratio is rewiring work itself” (May 18, 2026). source for the 3,000 internal AI agents and the 3:1 agent-to-employee ratio.

3. StartupHub.ai, “ClickUp’s 22% cut comes with $1M salary bands. Evans calls it the 100x org.”. source for Evans’s “great reckoning of AI coding” critique and the 500% pull request volume reference.

4. The Next Web, “ClickUp cuts 22% of staff, offers $1M salaries in AI restructuring”. source for the Builders / Agent Managers / Front-liners taxonomy.

5. Business Today, “’People who automate jobs with AI will always have a job’: ClickUp cuts 22% of its workforce”. source for the Agent Manager quote.

6. Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal (February 14, 2024). source for the Air Canada chatbot liability ruling.

Suneeta Modekurty | Founder, SANJEEVANI AI | Quantifying AI Readiness

Image created using AI tools

We are AI First, We are using AI, We are leveraging AI across the enterprise. ….

These kind of phrases have been around for a while. They show up in headlines and slide decks and strategy documents. But saying “AI ready” and actually being ready are two very different things.

In this series, I want to stay with a single, uncomfortable question:

Are we simply using AI, or are we genuinely ready for what it does to our decisions, our people, and the communities we serve?

To get there, I am going to stay close to real stories. Some come from the patterns I have heard across more than five hundred conversations I had over past months, with practitioners, leaders, and operators working through AI questions inside their organizations, with details abstracted to protect the people and the work. Others, like the one in this article, are already public. All of them sit at the intersection of three things: a human being in a real situation, an AI system that sounds confident, and an organization that is about to discover what “not ready” really means.

Let us begin with an incident that happened in November 2022.

A traveler in British Columbia named Jake Moffatt opened the Air Canada website to book a last-minute flight from Vancouver to Toronto. He was a private individual; his name only later entered the public record because of the complaint he eventually brought. His grandmother had just died. He needed to be at the funeral. Before he booked the ticket, he asked the airline’s chatbot whether Air Canada offered bereavement fares, and if so, how to claim one.

The chatbot told him that the airline did offer a reduced bereavement rate, and that if he needed to travel immediately, he could book at the standard fare and then apply for the discount by submitting a ticket refund application within ninety days of the date the ticket was issued. Moffatt took a screenshot of the exchange. He booked the flight. He attended the funeral. When he returned home, he submitted his refund application with the required documentation, including his grandmother's death certificate. Air Canada denied the request.

The actual bereavement policy on the airline’s website, on a separate page titled “Bereavement travel,” stated clearly that the discount could not be applied after travel had been completed. The chatbot had been wrong. An Air Canada representative later acknowledged, in correspondence with Moffatt, that the chatbot had provided misleading words and said the airline would update it. They did not offer him the discount.

Moffatt took the matter to the British Columbia Civil Resolution Tribunal, a small-claims body designed for low-value consumer disputes. The amount in question was a few hundred Canadian dollars. Air Canada, defending itself, made an argument that would later draw international attention. The airline argued that it could not be held liable for what the chatbot had said, because the chatbot was, in the airline’s framing, a separate legal entity responsible for its own actions. The tribunal’s response, written by Member Christopher C. Rivers, has been quoted in legal commentary in nearly every jurisdiction that watches AI law. The tribunal called the submission remarkable. It then explained, plainly, that while a chatbot has an interactive component, it is still just a part of Air Canada’s website, and it should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference, the tribunal continued, whether the information comes from a static page or a chatbot. Air Canada was ordered to pay Moffatt eight hundred and twelve dollars and two cents in total damages and fees. The decision, Moffatt v Air Canada, 2024 BCCRT 149, is not binding on any other court. It has nevertheless been treated, in nearly every serious analysis published since, as the first clear judicial statement of a principle the industry had been quietly avoiding. An organization that deploys an AI system is the author of what that system says.

It is tempting to read this as a chatbot story. A vendor sold an airline a customer service tool, the tool gave a wrong answer, the airline paid for the mistake, and the technology will get better.

That reading misses what is actually instructive about the case. The technology is not the failure mode. The failure mode is upstream of the technology, inside the organization that put the system in front of customers.

Air Canada deployed a customer-facing AI without, by all available indications, the apparatus needed to know what the system was saying day to day, the mechanism needed to correct it when it was wrong, or the internal framing needed to take responsibility for it when it caused harm. The remarkable submission the tribunal rebuked was not a clever legal maneuver by outside counsel. It was the logical end of an organization that had deployed a system it did not feel it owned.

This is what we mean by AI readiness, and this is where literacy enters the picture. The question almost every executive team is now asking is some version of are we using AI.

The honest answer in most organizations is yes, in some workflows, with varying degrees of intention. The more important question is the one Air Canada was, in effect, forced to answer in front of a tribunal. Are we ready for the fact that we are using it. Use is everywhere.

Readiness is the thing your board, your regulators, your customers, and increasingly your courts will ask you to demonstrate.

The Moffatt case, in a few pages of tribunal decision, demonstrated that Air Canada was using AI and was not ready for it.

Readiness sits on top of literacy, and the literacy that failed at Air Canada is the literacy of deployment. The literacy of deployment is what an organization’s leaders, owners, and senior decision-makers need so that the choices about where AI enters the organization are deliberate rather than reactive. It is not about coding skill or model intuition. It is the capacity to ask, before a system goes live, three questions that look simple and almost never get clear answers in unprepared organizations. The first is what this system will say to the people it interacts with, and whether that aligns with what the rest of the organization is saying on the same topics. The second is who, inside the building, owns the answer if the system gets it wrong. The third is what the organization will do, operationally and legally, when the system gets it wrong in a way that produces harm. Air Canada had a chatbot that said one thing, a policy page that said another, and no apparent reconciliation between the two. The deployment decision had been made without the deployment literacy needed to carry it.

This is also where shadow AI enters the conversation, and it is worth naming, because shadow AI is the version of this gap that most organizations are quietly living inside today. Shadow AI is the AI use that happens inside an organization without the knowledge, sanction, or oversight of the people who would be accountable if something went wrong. A marketing manager who pastes draft customer messaging into a public AI tool to tighten it up before sending. A finance analyst who runs a board memo through a generative system to make it sound sharper. A field engineer who asks an open chatbot how to handle a regulated chemical because the internal documentation is too slow to navigate.

None of these people are acting in bad faith. They are doing what their workload asks them to do, with the tools that are most accessible at the moment of pressure. The cumulative effect is that the organization is using AI in places its leadership does not know about, with data its policies have not classified, producing outputs that influence decisions that will later need to be defended. The Air Canada chatbot was a sanctioned deployment that did not have the literacy behind it. Shadow AI is the unsanctioned deployment that does not even have the visibility behind it. Both produce the same outcome, which is decisions the organization cannot defend.

The reason the Moffatt case has been quoted so widely is not that the airline paid a few hundred dollars. The reason is that the tribunal, in plain language, removed the option that many organizations had been quietly relying on. The option to argue, when an AI system causes harm, that the system was somehow separate from the organization that ran it. That option no longer exists in any jurisdiction that takes the Moffatt reasoning seriously, and the reasoning is too straightforward to confine to one tribunal in one province. If your AI is part of your website, your customer service, your hiring funnel, your underwriting workflow, your clinical pathway, your procurement chain, then your AI is part of your organization. What it says, you said.

The shift this forces is not legal. It is organizational, and it is what AI readiness is actually about. An organization that is ready for AI has built the apparatus to stand behind every AI-mediated decision its name is attached to. That apparatus does not begin with the technology. It begins with deployment literacy at the top of the house, and it cascades from there. The frontline staff need the literacy of use so they can interpret AI outputs with calibrated suspicion. The builders need the literacy of development so the systems they ship can be reasoned about and not just deployed. The leaders need the literacy of deployment so the choices about where AI enters the organization are deliberate. Without all three, readiness is a claim the organization makes about itself without the means to defend it.

The European Union, the National Institute of Standards and Technology, and the International Organization for Standardization have each written the requirement for AI literacy into their respective frameworks in the last three years. Article 4 of the EU AI Act, in force since February 2025, requires both providers and deployers to ensure sufficient AI literacy among the staff who use these systems. The NIST AI Risk Management Framework places workforce competency inside its GOVERN function. ISO/IEC 42001 requires organizations to determine and ensure the competence of personnel whose work affects AI performance. The instinct across all three is the same. The frameworks know that literacy is the missing piece.

None of the frameworks tells you what good looks like in operational detail, and none of them gives you a score. The frameworks name the requirement and pass the burden of measurement back to you. That is the gap we will keep returning to in this series.

Put differently: they can tell you that your people need AI literacy; they cannot tell you whether they have it.

If your organization had been Air Canada in November 2022, the operational questions worth sitting with are these.

• Would you have known what the chatbot was telling customers about bereavement fares that week.

• Would the chatbot’s answer have matched the page on your own website that bore the same title.

• Would anyone inside the building, before the refund was denied, have been able to flag that the two were saying different things.

• Would your first response, when the customer complained, have been to honor what your system had said, or to argue that the system was not yours.

These are not technology questions. They are readiness questions, and the answers are the difference between an organization using AI and an organization ready for it.

Sit with one question this week. If a customer, a regulator, or a court asked you tomorrow to defend one AI‑mediated decision your organization has already made, who in the building would you ask first, and could they answer.

Suneeta Modekurty | Founder, SANJEEVANI AI | Quantifying AI Readiness

Image created using AI tools

Dear Friends,

A school superintendent I worked with last year sat across from me at a kitchen table and asked a question she had been carrying for months.

“My teachers are using AI. My students are using AI. The plagiarism vendor wants a contract. The school board wants a policy. The parents want answers. And I do not know what good looks like. How do I tell if we are ready?”

I have heard that question, in different costumes, from a state CIO, a hospital chief of staff, a community organizer, a board chair, a founder, and a parent in a Costco parking lot.

The question sounds like it is about AI. It is not. It is about a chain that runs underneath AI, in a direction most current conversations do not name.

AI Readiness depends on AI Adoption. Adoption depends on Awareness. Awareness depends on Literacy. Literacy is the foundation, and most organizations skip straight past it.

The next letter arrives Tuesday, and from there every other Tuesday at nine in the morning Central time.

Suneeta Modekurty
Founder, SANJEEVANI AI

I am making a simple change today. I am stepping back from the regular cadence for now. No action needed on your part.

When I return, it will be with more clarity, more depth, and more of what actually matters to you as a practitioner navigating AI in the real world.

Thank you for being here early. That means everything.

See you soon.

Suneeta

Image created by AI

Last week I wrote about a New York bill that would restrict AI systems from providing professional advice in licensed fields. A few readers asked a sharp follow-up question: does the bill apply differently depending on how the AI system was built?

The answer might surprise you, and I wil…

Read more

Image created by AI

Yesterday I wrote about the shift from measuring organizations to measuring AI systems. Today, the New York State Legislature is proving why that shift is urgent.

A bill introduced by Senator Kristen Gonzalez would restrict AI systems from providing what lawmakers call “substantive responses” in fields that require professional licenses. Medicine, law, engineering, psychology, dentistry, nursing, and other regulated professions where incorrect guidance can cause serious harm.

Read that again carefully. The bill does not say “companies must have policies about what their AI says.” It says AI systems must not provide certain types of responses. The subject of the regulation is the machine, not the organization.

This is the shift happening in real time.

What the bill actually does

The proposal draws a line between general information and professional advice. An AI chatbot can share educational content about, say, symptoms of a condition or how a legal process generally works. What it cannot do is cross into substantive guidance that resembles what a licensed professional would provide. It cannot offer what looks like a medical diagnosis, a legal strategy, an engineering recommendation, or a psychological assessment.

The bill also includes a private right of action. That means individuals can sue companies if their AI systems provide restricted guidance. This is not a regulatory slap on the wrist. This is litigation exposure for every company deploying a customer-facing AI system in a licensed domain.

Why this matters beyond New York

If you are thinking “I do not operate in New York, this does not apply to me,” think again.

New York tends to set the template. When New York moved on financial regulation, the rest of the country followed. The same pattern is already forming with AI. Colorado’s AI Act takes effect in 2026. The EU AI Act becomes fully enforceable in August 2026. The NAIC Model Bulletin on AI in insurance has been adopted by 24 states. NYC Local Law 144 already requires bias audits for automated hiring tools.

The direction is clear: regulators are moving from governing organizations that use AI to governing what AI systems actually do. And they are doing it jurisdiction by jurisdiction, which means any company deploying AI across state lines will soon face a patchwork of requirements that all ask the same fundamental question: does your AI system stay within its authorized boundaries?

The measurement problem this creates

Here is where the data scientist in me gets interested.

“Substantive response” is a fuzzy concept. Where exactly does educational information end and professional advice begin? When does a health chatbot cross from sharing general wellness content into offering what could be interpreted as a diagnosis? When does a legal information tool cross from explaining a process into recommending a strategy?

These are not binary questions. They are spectrum questions. And spectrum questions require quantitative measurement, not policy checklists.

Think about what an organization would need to demonstrate under this bill. Not that they have a policy saying “our AI does not give medical advice.” They would need to demonstrate that their AI system actually stays within bounds, consistently, across thousands of interactions, including edge cases where users push the boundaries with creative phrasing.

That is a behavioral measurement problem. You cannot solve it by reading the organization’s policy documents. You solve it by observing what the AI system actually says when real people interact with it. You measure boundary adherence: how often does the system recognize when it is approaching a restricted domain, and how reliably does it pull back?

This is exactly the kind of observable, quantifiable AI system property that I described yesterday. The policy says the system will not give medical advice. The behavior shows whether it actually does or does not. The gap between those two is where the litigation risk lives.

What this means for different types of AI deployments

The bill applies regardless of how the AI system was built, but the risk profile varies significantly.

Organizations that build their own AI from the ground up have complete control over system prompts, guardrails, and response boundaries. They can engineer precise limits. But they also own 100% of the liability.

Organizations using low-code platforms like Copilot Studio or LangFlow face a shared responsibility problem. The platform provides underlying model behavior and some guardrails, but the builder configures the use case and the domain scope. When the system drifts into professional advice territory, who is liable? The platform or the builder?

And then there are the no-code deployments, the custom GPTs, the drag-and-drop chatbot builders. This is the highest risk category, and it is not close. The people building on these platforms are often the exact professionals the bill is trying to protect: small healthcare clinics, law offices, dental practices. They deploy an AI chatbot on their website, feed it their documents, and assume the platform handles compliance. It usually does not.

The gap between how easy it is to deploy AI and how hard it is to govern what it says is widest in the no-code tier. And that gap is exactly where this bill’s private right of action will land hardest.

The deeper signal

Step back from the specifics of this one bill and look at what it represents.

For decades, professional licensing has been a human-to-human regulatory framework. A doctor is licensed. A lawyer passes the bar. An engineer gets certified. The license attaches to the person, and the person is accountable for what they say.

AI breaks that model. The chatbot giving health guidance is not a licensed professional. It is not a person. It cannot be sued, sanctioned, or stripped of credentials. So the regulatory framework has to evolve. It has to attach accountability to the system’s behavior and to the entity that deployed it.

This bill is one of the first attempts to do that explicitly. It will not be the last. And every attempt will come back to the same core question: can you prove, with data, that your AI system behaves within its authorized boundaries?

That is not a policy question. That is a measurement question. And it demands the kind of quantitative, reproducible, behavior-based measurement that this newsletter exists to explore.

More next Tuesday.

This is part of the “Before The Number” series at A.I.N.S.T.E.I.N., exploring what it takes to build quantitative AI governance measurement from first principles. If this resonated, share it with someone deploying AI in healthcare, legal, or any licensed profession.

Image created by AI

I owe you an explanation for the silence.

Two weeks ago, I published “How to Measure AI Governance” and laid out the five pillars, the metrics, the frameworks, the KPIs. I meant every word of it. And then I went quiet, because something broke in my own thinking that I could not write aro…

Read more

Image by AI

If you cannot measure it, you cannot govern it.

That principle holds true across every regulated industry, from finance to healthcare to cybersecurity, and it holds true for AI.

Yet when most organizations talk about AI governance today, they are talking about policies, principles, and frameworks. They are talking about what they believe, not what they can prove. And there is a meaningful difference between having an AI ethics policy and being able to demonstrate, with data, that your AI systems are actually governed.

This essay is a practical guide to bridging that gap. It walks through the core pillars of AI governance measurement, the specific metrics that matter, the frameworks available to structure the work, and the challenges that make this harder than it sounds. If you are a CISO, a Chief AI Officer, a compliance leader, or a founder building in this space, this is the foundation you need.

Why Measurement Matters

Without measurement, AI governance is a set of intentions. It lives in documents that get written once and reviewed quarterly at best. It gives leadership a sense of comfort without giving them a basis for action.

Measurement changes that in four concrete ways.

1. It demonstrates due diligence. When regulators, boards, or the public ask how your AI is managed, measurement gives you evidence rather than assurances.

2. It allows you to identify and mitigate risks before they cause harm, because metrics like model drift detection time and fairness deviation surface problems that narrative assessments miss entirely.

3. It prepares you for the regulatory compliance landscape that is already here, with the EU AI Act requiring specific documentation and measurement for high-risk AI systems.

4. And it builds trust with every stakeholder who needs to know that your AI decisions are fair, transparent, and accountable.

The Five Pillars of AI Governance Measurement

Effective AI governance measurement is not about tracking model accuracy or inference speed. Those are performance metrics. Governance measurement focuses on accountability, fairness, transparency, compliance, and safety. These are the five pillars, and each one requires its own set of metrics.

Accountability and Ownership

This pillar measures who is responsible for your AI systems and their outcomes. It sounds basic, but in my experience, a surprising number of organizations deploy AI systems where no single person owns the governance risk. The model was built by one team, deployed by another, and monitored by no one in particular.

The qualitative goal is straightforward: every high-risk AI system should have a named business owner who is accountable for its impact. The quantitative metric that tracks this is the percentage of deployed AI systems with a defined, documented business owner. If that number is below 100% for your high-risk systems, you have a governance gap that no policy document can close.

Transparency and Explainability

This pillar measures how well your AI system’s decisions can be understood by the humans affected by them. A lending model that denies an application needs to be able to explain why. A hiring algorithm that filters out candidates needs to produce a reason that a human can evaluate.

The quantitative metric here is the percentage of AI-driven decisions that include a human-interpretable explanation. In practice, this is one of the hardest metrics to improve because many complex models, particularly large language models, are inherently difficult to explain. But the measurement itself forces the conversation about where explainability gaps exist and how material those gaps are.

Fairness and Bias Mitigation

This pillar measures the extent to which your AI systems treat different demographic groups equitably. It is not enough to say “we care about fairness.” You need to measure the actual disparity in outcomes across protected groups and track that disparity over time.

The core metric is the measurable difference in approval rates, error rates, or outcomes between demographic groups. If your lending model approves 78% of applications from one group and 61% from another, that disparity is your fairness metric, and it needs to be monitored continuously, not just checked once before deployment.

Risk and Compliance

This pillar measures adherence to both internal policies and external regulations. With the EU AI Act, NIST AI RMF, and ISO 42001 all converging on requirements for risk classification and documentation, this pillar is becoming the most operationally urgent.

The key metrics include the percentage of high-risk AI systems that have completed an Algorithmic Impact Assessment, the percentage of inventoried systems that have undergone formalized risk classification, and the policy adherence rate across all AI projects. These numbers tell you whether your governance framework is actually being followed or whether it exists only on paper.

Safety and Security

This pillar measures your AI system’s resilience against attacks, errors, and unintended harm. It includes incident response readiness and the speed at which AI-specific failures are detected and resolved.

The metrics that matter here are the average time to detect and time to resolve AI-related incidents, including model drift, toxic output, adversarial attacks, and data pipeline failures. If your organization cannot tell you how long it takes to detect when a model has drifted from its intended behavior, your safety posture has a blind spot.

Key Performance Indicators for AI Governance

Beyond the five pillars, there are specific KPIs that give leadership a clear picture of governance health across the organization.

Program health metrics include AI inventory coverage (the percentage of all AI systems currently cataloged), risk classification completion (the percentage of inventoried systems that have been formally classified by risk level), and policy adherence rate (the percentage of AI projects fully compliant with established guidelines).

Decision and accountability metrics include decision latency for risk issues (how long it takes to make a material decision on an escalated AI risk), human override rate (how frequently automated decisions are reversed by human reviewers), and governance debt (the number of deferred governance controls that were postponed to speed up deployment).

Operational integrity metrics include model drift detection time, data lineage visibility (the percentage of models with full source-to-sink tracking), and audit readiness score (the percentage of models with current documentation and version control).

Ethical impact metrics include explanation coverage and fairness deviation, both of which I discussed in the pillars section above.

The important thing about these KPIs is that they are specific, measurable, and tied to real governance risk. They are not opinions. They are not traffic lights. They are numbers that a board can track quarter over quarter and that an auditor can verify independently.

The Frameworks That Structure This Work

Organizations do not need to build their measurement approach from scratch. Several established frameworks provide the structure.

The NIST AI Risk Management Framework provides guidelines for managing risks to improve the trustworthiness of AI systems. NIST has also recently released a preliminary draft Cyber AI Profile (NISTIR 8596) that maps AI considerations directly onto the Cybersecurity Framework 2.0, embedding AI governance into operational security infrastructure rather than treating it as a separate discipline.

ISO/IEC 42001 is an international standard specifying requirements for establishing, implementing, maintaining, and continually improving an AI management system. As an ISO 42001 Lead Auditor, I work with this framework regularly, and its strength is that it provides a certifiable standard that organizations can be audited against.

The EU AI Act is the most comprehensive regulatory framework currently in effect, requiring specific measurement and documentation for high-risk AI systems. It is not optional for organizations operating in or selling into the European market, and its requirements are driving measurement adoption globally.

These frameworks tell you what to measure and why. The challenge is translating their requirements into the specific quantitative metrics I described above, and doing so continuously rather than at a single point in time.

The Challenges That Make This Hard

If measuring AI governance were easy, every organization would already be doing it. Several factors make it genuinely difficult.

Concepts like fairness and transparency are contextually dependent. What counts as fair in a lending model may differ from what counts as fair in a hiring algorithm. There is no single universal formula, and measurement requires thoughtful interpretation alongside the numbers.

Many complex AI models, particularly large language models, are inherently difficult to explain. This makes transparency measurement challenging not because the metric is wrong but because the underlying system resists the measurement.

Standardization is still evolving. While frameworks exist, universally accepted methods for calculating specific metrics like bias are not yet settled. Different tools and approaches can produce different results for the same system.

Organizations have historically incentivized performance over responsibility. Accuracy and speed get rewarded. Governance measurement introduces a different set of priorities, and that cultural shift is often harder than the technical implementation.

And finally, data quality and lineage remain fundamental obstacles. You cannot measure governance properly if you do not understand the data your AI systems are trained on, and many organizations have complex or undocumented data flows that make this difficult.

Where This Is Heading

Every one of these challenges is real, and none of them are reasons to avoid measurement. They are reasons to invest in building the measurement infrastructure now, before regulators require it and before the gap between what your organization claims about its AI governance and what it can actually prove becomes a liability.

The organizations that solve the measurement problem first will not just be compliant. They will set the standard that others measure against. They will have the data to report to boards, the benchmarks to negotiate with partners, and the scores to prove what checklists never could.

AI governance measurement is not a nice-to-have. It is the infrastructure that makes governance real.

Image created by AI

Every critical system in human history eventually got measured. Before FICO, loan officers decided your creditworthiness with a handshake and a gut feeling. Same income, same history, approved at one branch and denied at another. Before s…

Read more

Image Created by AI

If you are in the governance space, the first three weeks of February 2026 arrived as a cascade, one event after another, each reinforcing the same message: the world is taking AI governance seriously. UC Berkeley’s Center for Long-Term C…

Read more

Image created using AI

This week, I was invited through one of the world’s largest expert networks to consult on a topic that stopped me in my tracks: the challenges and solutions for securely deploying autonomous AI agents in business environments, with a particular focus on something called “verifiable credentials” for AI agents and “trusted AI through intent binding.”

I have been building AI governance infrastructure for over two years now, and I have spent 25 years before that deploying AI and data systems across healthcare, fintech, insurance, legal technology, and education. But this invitation was different. It was not about compliance checklists or policy frameworks. It was about a question that the enterprise world is only now beginning to ask out loud: How do you prove that an AI agent acting on your behalf is actually authorized to do what it is doing?

Let that sink in for a moment.

The Problem Nobody Talks About

Right now, across every industry you can name, companies are racing to deploy AI agents that can act autonomously. These agents book meetings, process claims, triage legal inquiries, approve transactions, generate reports, and make decisions that used to require a human being in the loop. The promise is enormous: speed, scale, consistency, cost reduction. And the technology has reached a point where these agents can genuinely perform.

But here is what almost nobody is asking: What credentials does that AI agent carry?

When a human employee joins an organization, they go through background checks, they receive role-based access, they sign agreements about what they can and cannot do, and there is a paper trail connecting their identity to their authority. If that employee oversteps their boundaries, there is an audit trail. There is accountability. There is a verifiable chain that connects what they did to what they were authorized to do.

Now think about an AI agent. It gets deployed with an API key, maybe some prompt instructions, maybe a set of tool permissions configured by a developer who was moving fast to hit a sprint deadline. Where is the verifiable proof of what that agent is authorized to do? Where is the audit trail that connects its actions to a specific human decision about its scope? Where is the credential that another system, a partner, a regulator, a customer, can independently verify without simply trusting the agent’s own assertions?

It does not exist. Not in any standardized, quantitative, independently verifiable way.

Your AI agent has no ID.

Why Checklists Fail for Autonomous Systems

The traditional approach to AI governance has been borrowed from software compliance: create a checklist, assess against it annually or quarterly, produce a report, file it away. This works reasonably well for static systems where humans make the final decisions and the AI is just providing recommendations. You can audit the model, check the training data, review the outputs, and sign off.

But autonomous AI agents break this model completely.

An autonomous agent does not wait for a human to review its output before acting. It chains decisions together. It interprets ambiguous inputs in real time. It interacts with other systems, sometimes other agents, and the scope of its actions can shift based on context in ways that no static checklist can anticipate. A checklist that was accurate on Monday might be meaningless by Wednesday because the agent encountered a scenario that nobody tested for, and it made a judgment call.

This is not a theoretical concern. I have seen it firsthand. When I deployed an autonomous AI voice agent for a law firm handling workers’ compensation cases, the most dangerous moments were not when the agent got something wrong in a predictable way. They were when callers deviated from expected conversational paths and the agent had to decide, in real time, whether it was authorized to handle the new direction or whether it needed to escalate. A checklist cannot govern that decision. A quantitative, continuously updated trust boundary can.

The difference matters. A checklist says “this system was compliant when we last checked.” A trust score says “this system is operating within its verified boundaries right now, and here is the quantitative evidence.”

What Verifiable Credentials for AI Actually Means

When the consultation framed the topic around “verifiable credentials for AI agent deployment,” it pointed at something that I believe will become one of the defining infrastructure layers of the next decade.

Verifiable credentials for AI agents means that an agent carries cryptographically provable attestations about what it is authorized to do, who authorized it, what compliance standards it has been assessed against, and what boundaries it operates within. Any party interacting with that agent, whether it is another system, a business partner, a regulator, or a customer, can independently verify those claims without having to trust the agent itself.

Think of it like a digital license. Not a static certificate that was issued once and sits in a drawer, but a living, scored, continuously updated credential that reflects the agent’s current risk posture and authorization scope. When a partner organization’s system interacts with your AI agent, it can check that credential and confirm: yes, this agent has been assessed at a risk score of 247 out of 1000, it is authorized for these specific actions, it has been evaluated against ISO 42001 and the EU AI Act, and its last assessment was 14 minutes ago.

That is fundamentally different from saying “we passed an audit last quarter.”

The Market Is Moving Faster Than You Think

Here is what struck me about this invitation, and about the broader pattern I am seeing across the industry right now. This is not a topic that only governance nerds and compliance officers are thinking about. Investment firms are actively conducting diligence on companies in the AI security and governance tooling space. Major corporations are paying expert network rates to understand how verifiable trust for AI agents works. Professional services firms are benchmarking API governance frameworks in banking. The money is following the question.

And the question is the same everywhere: How do we trust AI that acts on its own?

For those of us who have been working in AI governance, this is the moment where the market catches up to the problem. For the past two years, I have heard enterprise leaders say they are “exploring” AI governance, that they know it matters but they are not ready to invest. That language is shifting. When investment firms start researching the competitive landscape for AI trust infrastructure, it means capital allocation decisions are being made. When enterprise clients specify “verifiable credentials” and “intent binding” as the topics they want to discuss, it means they have moved past awareness and into solution design.

The window between “people are asking the question” and “someone owns the answer” is open right now.

Why Measurement Beats Compliance

This brings me to the core thesis of everything I write about in this newsletter, and everything I am building.

The reason checklists and traditional compliance frameworks fail for autonomous AI is not that they are poorly designed. It is that they are qualitative instruments being applied to a quantitative problem. Asking whether an AI agent “meets” a compliance standard is like asking whether a bridge “meets” safety requirements without measuring the load it can bear. The answer is meaningless without a number.

What the market needs, and what it is beginning to demand, is quantitative measurement infrastructure for AI trust. A scoring system that can express, in a single interpretable number, how much risk an AI agent carries across multiple regulatory frameworks simultaneously. Not a binary pass/fail. Not a subjective assessment. A reproducible, auditable, continuously updated measurement that engineering teams can act on and compliance teams can report on and regulators can verify and business partners can trust.

This is the science of measurement applied to critical AI systems. It is what I call the work that happens “before the number,” the careful, rigorous thinking about what to measure, how to measure it, and why the measurement methodology matters as much as the result.

What Comes Next

If you are deploying AI agents in your organization today, or planning to, here is what I would encourage you to think about.

First, ask yourself whether you can prove, right now, what your AI agent is authorized to do. Not what it was designed to do. Not what the prompt says it should do. Can you prove it, verifiably and quantitatively, to a third party who has no reason to trust your assertions?

Second, ask yourself how you would know if your AI agent exceeded its authorized scope. Not after the fact, when a customer complains or a regulator asks. Right now. In real time. Do you have a continuous measurement of whether the agent is operating within its trust boundaries?

Third, ask yourself whether your current governance approach would survive the question: “Show me the score.” Not the checklist. Not the policy document. The score. The number that tells me, quantitatively, where this agent falls on the risk spectrum and what that number is based on.

If you cannot answer those questions today, you are not alone. Almost nobody can. But the market is telling us, loudly and with real dollars behind it, that the window to build this capability is right now.

That is what I am working on. That is what “Before the Number” is about. And I will have a lot more to say about it in the weeks ahead.

Suneeta Modekurty is the Founder and Chief AI Architect of SANJEEVANI AI, where she builds quantitative AI governance infrastructure. She is an ISO 42001 Lead Auditor and holds an O-1A visa for extraordinary ability in AI, bioinformatics, and data science. She publishes “Before the Number” on Substack, exploring the science of measurement in critical AI systems.

Divergence of Qualitative and Quantitative Measurements Over Time

In 1846, a doctor named Ignaz Semmelweis noticed something troubling. In one ward of the Vienna General Hospital, mothers were dying at five times the rate of the ward next door. It was the same hospital, in the same city, during the same year. The only difference was that one ward was staffed by doctors who came directly from performing autopsies, while the other was staffed by midwives who did not.

Semmelweis did not have germ theory. He did not have a microscope powerful enough to see bacteria. But he had something just as important: he had counted. He had a number. And the number made the invisible visible. When he introduced handwashing with chlorinated lime, mortality dropped from 18% to under 2%. The number did not just describe the problem. It made the solution undeniable.

But here is the part of the story people forget. The medical establishment rejected his findings for twenty years. They did not reject the findings because the data was wrong. They rejected the findings because the profession had no culture of measurement. Doctors operated on reputation, seniority, and judgment. Introducing a number threatened the entire social order of medicine because it implied that a senior physician’s intuition could be proven wrong, and worse, that a junior doctor armed with better data could be proven right. The number was not just a scientific tool. It was a direct challenge to the authority structure that governed the profession. And the authority structure pushed back.

Semmelweis died in an asylum in 1865. Germ theory was not widely accepted until the 1880s. For two decades, the refusal to accept a quantitative finding cost thousands of lives. Not because the answer was unknown, but because the system was not structured to receive a quantitative answer.

This essay is about what happens in that gap. It is not about the moment a number arrives, but about the years before it does, when a system that matters is governed by opinion and the cost of that governance failure remains invisible to the very people responsible for it.

Semmelweis’s story is dramatic, but it is not unique. When you study the history of measurement across critical systems in lending, aviation, food safety, environmental regulation, and cybersecurity, the same pattern emerges so consistently that it starts to look less like coincidence and more like a law of institutional physics. Every time a critical system operates without quantitative measurement, four specific costs appear. They appear in the same order, with the same dynamics, regardless of the industry, the era, or the technology involved.

The first cost is structural inconsistency: Before credit scoring existed, two loan officers at the same bank could evaluate the same applicant and reach opposite conclusions. This was not a flaw in the system. This was the system. Without a shared quantitative reference point, every decision was a fresh act of human judgment, shaped by experience, bias, workload, mood, and variables that nobody tracked because nobody could.

Fair Isaac Corporation studied this phenomenon in the 1950s and found staggering variance in loan decisions across branches of the same institution. The variance was not small. Outcomes that should have been statistically identical were diverging by double digits. The same applicant, with the same income and the same repayment history, could be approved at one branch in the morning and denied at another branch in the afternoon. This was not corruption. It was the natural and predictable result of a system that relied entirely on individual judgment with no standardized measurement to anchor it.

The same dynamic shows up everywhere measurement is absent. In food safety before HACCP scoring, the same restaurant could pass a health inspection in one county and fail in the county next door, because inspectors had no shared quantitative standard for what constituted a violation. In education before standardized assessments, the same student could be classified as “gifted” in one school district and “average” in another, because teachers were evaluating against their own internal benchmarks rather than a common metric. In pain management before the numeric pain scale, the same tibial fracture could receive acetaminophen from one nurse and morphine from another, because pain was described in adjectives rather than measured in numbers.

The deeper problem is not that people made different decisions. It is that nobody could demonstrate the inconsistency was even happening, because there was no consistent metric to compare against. When there is no number, inconsistency becomes invisible. It hides inside the phrase “professional judgment,” and it persists precisely because no one can see it.

The second cost follows directly from the first: accountability disappears. You cannot hold someone accountable for violating a standard that does not exist in quantifiable terms. You can write a policy that says “ensure patient safety.” You can create a governance framework that says “implement responsible AI practices.” But until those principles are attached to measurable thresholds, enforcement becomes a matter of interpretation, and interpretation is the enemy of accountability.

The aviation industry learned this through tragedy. Before the adoption of measurable safety metrics such as hours between incidents, defect rates per flight cycle, and standardized checklists with quantified completion rates, airlines assessed their own safety through self-reporting. The standard was “we follow best practices.” After a crash, the investigation would inevitably reveal that “best practices” meant different things to different maintenance crews, different inspectors, different shifts, and different airports. No one had been lying. There was simply nothing specific enough to be accountable to. The standard existed in prose rather than in numbers, and prose can be interpreted generously by anyone who needs it to be.

The same dynamic plagued corporate environmental responsibility for decades. When the standard was qualitative and every company could simply declare that it was “committed to sustainability,” every company in the world was effectively compliant, because commitment is not measurable. It was only when emissions reporting introduced actual numbers in the form of tons of CO₂, parts per million, and year-over-year change that accountability became possible. This shift did not happen because regulators suddenly became tougher. It happened because there was finally something concrete to hold companies accountable against. You can argue indefinitely about whether a company is “committed to sustainability.” You cannot argue with 47,000 metric tons of carbon.

Measurement does not create accountability on its own. But without measurement, accountability is theater. It carries all the language of oversight, including policies, frameworks, committees, and review boards, but none of the teeth. Because teeth require thresholds, and thresholds require numbers.

The third cost is the most economically significant, and it is also the hardest to see, because it manifests as things that never happen. When there is no number, markets do not crash. They simply never form in the first place.

Before credit scores existed, the secondary mortgage market barely existed either. A bank in Ohio could not sell a bundle of loans to an investor in New York, because there was no standardized way to assess the risk of those loans at scale. Every loan had been originated by a local officer, evaluated using local criteria, and documented in local formats. An investor three thousand miles away would have needed to re-underwrite every individual loan in order to assess the bundle, and the cost of that diligence was prohibitive. So the transaction simply did not happen.

The secondary mortgage market did not emerge because someone invented a clever financial instrument. It emerged because someone gave every borrower a number that an investor three thousand miles away could evaluate in seconds. Credit scoring did not just measure risk. It made risk portable. It created a common language that allowed parties who had never met each other to transact with confidence. A market worth trillions of dollars had been locked inside the absence of a three-digit number.

The same pattern explains why cyber insurance took decades to mature. Until organizations had quantifiable security postures in the form of scores, ratings, measurable controls, and auditable configurations, underwriters could not price policies with any actuarial confidence. You cannot build an insurance market around “we think we’re secure.” Insurance requires a number that actuaries can model, that underwriters can compare across applicants, and that reinsurers can aggregate into portfolios. The number does not just describe the risk. It enables the market infrastructure that makes risk transferable.

When there is no number, entire markets remain latent, not because demand is absent, but because there is nothing to transact around. Buyers cannot evaluate, sellers cannot differentiate, insurers cannot price, investors cannot compare, and regulators cannot benchmark. The market sits frozen, waiting for a unit of measurement that all participants agree to trust. And the longer that wait continues, the more value remains locked inside the gap.

The fourth cost is perhaps the most insidious: improvement becomes impossible to prove. Imagine walking into a hospital board meeting and reporting that patient safety improved this quarter. The first question will be: compared to what? By how much? Measured how? Without a quantitative baseline, improvement is a feeling rather than a fact. You can spend millions on better processes, better training, and better technology and still have absolutely no way to demonstrate that any of it worked.

This is not hypothetical. It is the precise reason the quality movement in manufacturing stalled for years until statistical process control gave factories a way to measure variation and prove that their interventions were actually reducing defects. W. Edwards Deming did not just advocate for quality. He advocated for measurement, because he understood from decades of experience that without it, quality was a slogan rather than a discipline. His famous observation that you cannot improve what you cannot measure was not a platitude. It was an empirical conclusion about what happens when organizations try to get better without quantitative feedback loops.

The consequence of unprovable improvement is organizational paralysis. When the finance team asks whether the governance investment is working and the honest answer is “we believe so but cannot demonstrate it,” budgets get questioned. When leadership asks whether the new training program reduced risk and the answer is anecdotal rather than measured, confidence erodes. And gradually, organizations stop investing in getting better, not because they do not want to improve, but because they have learned that improvement without measurement is indistinguishable from stagnation. The return on investment becomes invisible, and so the investment stops.

What is striking about these four costs is not that they exist, but that the people inside the system rarely see them clearly. Inconsistency feels like professional judgment. Missing accountability feels like flexibility. Frozen markets feel like the market simply is not ready yet. Unprovable improvement feels like doing the best you can under difficult circumstances. The absence of a number is comfortable. It protects incumbents. It allows vagueness to masquerade as strategy. It lets everyone believe they are above average, because there is no average to measure against.

This is why measurement is always resisted before it is adopted. Semmelweis was ridiculed. Early credit scoring was called dehumanizing, with critics arguing that a human relationship between banker and borrower could not and should not be reduced to a number. Standardized testing was called reductive. Emissions reporting was called burdensome. Every number that eventually became infrastructure started its life as an inconvenient truth that the existing establishment preferred to ignore.

And yet, in every single case, once the number arrived and proved its value, the world did not go back. Nobody has argued for returning to gut-feel lending decisions after FICO. Nobody has advocated for removing thermometers from hospitals. Nobody has suggested that airlines stop tracking maintenance defect rates. Nobody has proposed that companies stop reporting emissions in measurable units. The resistance dissolves once the number demonstrates what it can do, because the number does not just measure the system. It reorganizes the system. It changes who has authority, what constitutes evidence, how decisions get made, and what accountability looks like. The number becomes the infrastructure that everything else is built upon.

Which brings us to the present. Today, AI systems are approving loans, diagnosing cancers, screening job applicants, scoring insurance claims, generating legal documents, triaging emergency calls, and making consequential decisions that affect the lives of millions of people across every regulated industry on the planet.

Ask how well these systems are governed, and the answer in most organizations is qualitative. It takes the form of a maturity model with descriptive levels, a readiness checklist with binary checkboxes, a consultant’s assessment delivered as a narrative report, or a regulatory framework mapped onto a spreadsheet that was current when it was created and outdated by the time it was presented. The output is language rather than measurement. “We’re at level 3.” “We’ve addressed most of the NIST categories.” “We’re working on it.”

These are not numbers. They are opinions formatted to look like numbers. And the four costs are already visible for anyone willing to look.

The inconsistency is already here. Two auditors assessing the same AI system against the same governance framework reach materially different conclusions about its compliance posture. This is not a failure of the auditors. It is a failure of the measurement approach, or more precisely, the absence of one. When the standard is a checklist of qualitative criteria, every assessment is an interpretation, and interpretations diverge.

The accountability gap is already here. The EU AI Act is in force. ISO 42001 certifications are underway. NIST AI RMF adoption is accelerating. Regulatory enforcement is not theoretical but operational. Yet enforcement against what baseline? When a regulator asks an organization to demonstrate its AI governance posture over time, the evidence that exists consists of a policy document from last year, a completed questionnaire, and a consultant’s letter. None of these constitute the kind of quantitative, auditable, time-series evidence that regulators in every other domain have learned to require.

The frozen markets are already here. AI insurance is nascent. AI procurement due diligence is a custom exercise every time, with no standardized scoring to streamline vendor evaluation. AI risk assessment in mergers and acquisitions relies on qualitative representations that are difficult to verify and impossible to compare across targets. These markets are not early because demand is low. They are early because there is nothing standardized to transact around. The infrastructure of transaction, including pricing, benchmarking, comparison, and aggregation, requires a number that does not yet exist.

And the inability to prove improvement is already here. Organizations are spending real money on AI governance by hiring responsible AI teams, building review processes, investing in training, and purchasing tools. But when the board asks whether the organization is better governed than it was last quarter, the honest answer is that nobody knows, because nobody can measure it. There is a belief that things have improved. There are actions that should have made things better. But there is no number that moved.

Every critical system in human history has eventually been measured. Not because someone decided measurement was philosophically appealing, but because the cost of not measuring became intolerable. The lending industry crossed that threshold in the 1950s. Healthcare crossed it in the 19th century with the advent of lab science and vital signs. Aviation crossed it after enough planes fell from the sky and enough investigations revealed that “best practices” had been an empty phrase. Cybersecurity crossed it when boards stopped accepting narrative descriptions of risk exposure and started demanding numbers.

AI governance has not crossed that threshold yet. But every signal suggests it is approaching. The regulatory pressure is building across multiple jurisdictions simultaneously. The liability exposure is growing as AI systems move deeper into consequential decision-making. The number of AI systems in production is compounding faster than governance practices can keep pace. And the gap between what organizations claim about their governance posture and what they can actually demonstrate with evidence is widening every quarter.

The question is not whether AI governance will get a number. History is unambiguous on this point, because every critical system eventually does. The question is what happens between now and then, how long the gap persists, how much damage accumulates inside it, and who bears the cost of an industry that governed itself by opinion when it could have governed itself by measurement.

History suggests that cost will be larger than anyone currently inside the gap realizes. It always is.

Until next time,

Suneeta Modekurty

Founder & Chief Architect, METRIS™

Before the Number is a publication about the science of measurement in critical systems.

Two weeks ago, on January 22, I announced that AI governance now has a score.

Singapore released the world’s first agentic AI governance framework. South Korea’s AI Basic Act became enforceable. And on that same day, we shipped METRIS - a quantitative governance score for AI systems.

One of the responses that stayed with me came from a reader who wrote: “The distinction between frameworks and real, measurable governance is so crucial. Finally, someone speaks about dynamic AI risk.”

An AI/ML engineer called METRIS’s FICO-style approach “a total game changer for clarity” and then asked the right question: how does METRIS handle the shifting weight of different regulations as they evolve in real time?

A data engineer was more direct: “METRIS needs to be deployed everywhere, all at once. Think of the corporate goofs that could have been avoided.”

These reactions tell me the market knows it needs a number. But before I explain what METRIS does or how it works, I want to go deeper. I want to share why it needs to exist.

Because this isn’t a technology story. It’s a measurement story. And it starts with a question I’ve been asking my entire career.

The Question

I have fought ambiguity my entire professional life.

Twenty-five years across healthcare, pharma, finance, and technology. In every role, the hardest problems were never technical. They were the ones where two equally qualified people looked at the same situation and reached opposite conclusions. Where the answer depended on who you asked, what day it was, or which office you walked into.

That kind of ambiguity doesn’t just slow things down. It erodes trust. It makes fair outcomes impossible. And it creates a world where accountability is always someone else’s problem.

So I started asking a question that wouldn’t leave me alone: How did humanity solve ambiguity before?

The answer, every single time, was the same. We invented a number.

Banking Before FICO

Before 1989, getting a loan was a conversation. A loan officer looked at you across a desk. They reviewed your application. They considered your employment, your address, your references. And then they made a judgment call.

The same person, with the same income and the same history, could be approved at one branch and denied at another. The process was subjective, inconsistent, and (let’s be honest), was discriminatory. Billions of dollars were allocated based on gut feeling, personal bias, and the quality of a handshake.

Then Fair, Isaac and Company introduced a three-digit number.

FICO didn’t add technology to lending. It removed ambiguity from lending. One number. Same calculation everywhere. A 720 in New York meant the same thing as a 720 in Nebraska. Suddenly, lending decisions were comparable, auditable, and fairer than the alternative.

Today, nothing moves without a credit score. Mortgages, car loans, insurance premiums, apartment rentals. The number became infrastructure.

Medicine Before Lab Tests

A doctor feels your forehead. “You seem warm.” Another doctor examines you an hour later. “You seem fine.” Same patient. Two opinions. Who’s right?

Before quantitative diagnostics, medical decisions were based on observation, intuition, and experience. All valuable, but subjective. Two physicians could examine the same patient and reach opposite conclusions. Treatment varied not by disease, but by doctor.

Then someone measured body temperature. Then blood count. Then glucose levels. Then cholesterol, then hemoglobin A1C, then troponin levels, then genetic markers.

Numbers didn’t replace doctors. Numbers gave doctors a shared reality to work from. A blood glucose of 280 means the same thing in Asia and in America or Europe. The measurement created a common language, and with it, accountability.

Education Before Standardized Assessment

A teacher reads your essay and says “good.” Another teacher reads the same essay and says “mediocre.” A third says “brilliant.” The essay hasn’t changed. The judges have.

Before rubrics, scores, and standardized marking systems, educational assessment was entirely subjective. You can argue about whether standardized testing is perfect, it isn’t. But it made assessment comparable. A 92 means the same thing regardless of who graded it. Progress could be tracked. Gaps could be identified. Accountability became possible.

The Deeper Pattern

If you go far enough back, you find the origin of measurement itself: the moment societies decided that subjective trust wasn’t enough.

Signatures. Seals. Notarization. Weights and measures. Currency denominations. Accounting standards. Credit ratings. Safety certifications. Every one of these innovations was born from the same realization: when the stakes are high enough, “trust me” is not a system. Measurement is.

The pattern is always the same. First, a critical human activity operates on judgment alone. Then the stakes get high enough that inconsistency becomes intolerable. Then someone invents a way to measure. And the measurement becomes the new infrastructure-so foundational that within a generation, no one can imagine doing without it.

Now Look at AI

AI systems are making lending decisions. Diagnosing patients. Evaluating students. Screening job applicants. Flagging criminal suspects. Approving insurance claims. Moderating speech. Predicting recidivism.

These are the exact same domains that humanity spent centuries learning to measure. The lending system has a FICO score. The lab has quantitative diagnostics. The exam has a rubric. The contract has a signature.

But the AI that is replacing these systems? The AI that is now making these consequential decisions on our behalf?

Ask “How governed is this AI system?” and the answer you get is: a checklist. A policy document. A consultant’s opinion. A yes-or-no audit. Or worse? a shrug.

There is no number.

The Problem with Binary

The AI governance conversation today is stuck in binary. Compliant or not. Pass or fail. And that framing is the source of the paralysis.

Consider two companies.
Company A has done nothing:no documentation, no fairness testing, no monitoring, no risk assessment.
Company B has documented all its models, implemented bias testing, established human oversight protocols, but hasn’t yet completed adversarial robustness testing.

In a binary system, both fail. Same result. Same bucket. Binary made them identical. But they are not identical. One is at 50. The other is at 680. One needs a transformation. The other needs a nudge.

Binary created a market that is frozen. Companies that haven’t started say “we’re exploring.” Companies that have invested say “what’s the point?” Companies that passed say “we’re done” and stop paying attention - until the next incident.

A Score Changes Everything

A score does what binary cannot. It makes progress visible. It enables comparison. It creates continuous accountability. It creates a market that moves.

At 50, you know you’re early but you’ve started. At 400, you can see progress. At 680, you know exactly which gaps separate you from 800. At 900, you can prove your posture to your board, your regulator, your customers. And tomorrow, if your score drops because a new regulation kicked in or a model drifted, you see it immediately.

Governance isn’t pass/fail. It’s a score.

What’s Next

In my last newsletter, I announced METRIS

Today, I wanted to tell you why it exists. Because the founding insight behind METRIS isn’t technical. It’s historical. Every critical system eventually gets a number. AI’s turn is now.

In the coming weeks, I’ll share: how the scoring engine actually works, what the first assessments are revealing about the state of AI governance in the wild, and how organizations are using their score to move from “we’re exploring” to “here’s where we stand.”

If you’re building AI and wrestling with governance, reply to this newsletter. Tell me what’s broken. What’s working. What you wish existed. I read everything.

Every critical system in human history eventually got a number.

Lending got a credit score. Health got diagnostics. Education got assessments. Security got ratings. Financial health got audited statements.

AI is now the most consequential system in modern life. And it has no number.

Not yet.

Suneeta Modekurty

Founder & Chief Architect, METRIS™

ISO 42001 Lead Auditor | Sanjeevani AI LLC

A.I.N.S.T.E.I.N. is a reader-supported publication. Subscribe to follow the METRIS journey.

Not because Singapore launched a framework.

Not because South Korea’s AI law went live.

Because METRIS arrived.

And honestly? The timing couldn’t be more poetic.

On the same day that governments finally admitted AI governance needs real infrastructure - not more PDFs - we shipped exactly that.

Let me explain.

What Happened on January 22

Singapore released the world’s first agentic AI governance framework. 27 pages of guidance on how enterprises should govern AI agents.

South Korea’s AI Basic Act became enforceable. Mandatory impact assessments. Documentation requirements. Fines for non-compliance.

Two major economies. Two different approaches. Same message:

The era of “we’ll figure out AI governance later” is over.

But here’s what caught my attention in Singapore’s framework:

“AI risk is no longer static, it is dynamic and behavioral.”

Finally. Someone said it out loud.

The Problem With Frameworks

Frameworks tell you what to do.

They don’t tell you how well you’re doing it.

• “Implement human oversight” → But how do you measure if it’s meaningful?

• “Assess and bound risks” → But what’s your actual risk score?

• “Enable accountability” → But across which of the 9 regulatory frameworks that apply to you?

We’ve spent years collecting frameworks like Pokémon cards. EU AI Act. ISO 42001. NIST AI RMF. Singapore MGF. Korea AI Basic Act.

And yet - 94% of AI repositories still fail basic governance requirements.

We know this because we measured it. 2,000+ repositories. 1,429 checkpoints. 9 regulatory frameworks.

The market is drowning in frameworks.

What it’s starving for is measurement.

Enter METRIS

METRIS is what we’ve been building at Sanjeevani AI.

Not another framework. Not another checklist.

A quantitative risk score for AI governance.

Think of it like this:

• Frameworks tell you to “be healthy”

• METRIS is your blood pressure reading

Here’s what it does:

• 0-1000 Risk Score → Know exactly where you stand

• 1,429 Checkpoints → Mapped across 9 regulatory frameworks

• Continuous Assessment → Not point-in-time audits

• Bayesian Scoring + Monte Carlo Modeling → Because governance isn’t binary

The Singapore framework calls for “continuous monitoring” and “technical controls throughout the agent lifecycle.”

Great. METRIS is how you actually do that.

Why January 22

We could have launched any day.

But when we saw Singapore’s Davos announcement on the calendar, and Korea’s enforcement date landing the same day, we knew.

This was the moment.

Not to ride their coattails - but to draw a line:

January 22, 2026 is the day AI governance stopped being a conversation and started being infrastructure.

They wrote the frameworks.

We built the measurement layer.

What This Means For You

If you’re an enterprise deploying AI - especially agentic AI - here’s the reality:

1. Voluntary frameworks become market expectations. Singapore’s isn’t mandatory. It doesn’t matter. Your customers, partners, and investors will expect you to comply.

2. Mandatory requirements are cascading. Korea today. EU AI Act implementation ongoing. Others will follow.

3. “We’re working on governance” isn’t an answer anymore. The question is: What’s your score?

The enterprises that can answer that question - with data, not promises - will own the trust advantage.

One Ask

If you’re building AI and wrestling with governance, whether you’re trying to comply with frameworks, preparing for audits, or just trying to figure out where you actually stand, I want to hear from you.

Reply to this email. Tell me what’s broken. What’s working. What you wish existed.

I read everything.

January 22, 2026. Mark your calendars.

The day AI governance got a score.

Suneeta Modekurty

Founder, SANJEEVANI AI | Creator of METRIS

• METRIS - SANJEEVANI AI

• Singapore Agentic AI Framework (PDF)

• South Korea AI Basic Act Overview