Image created using AI tools
I attended a Databricks webinar earlier this month on data-driven AI governance, led by Matei Zaharia, the company’s cofounder and chief technology officer, with K. Sewell, who directs product for their AI products. Zaharia opened by saying that agents are beginning to go rogue inside enterprises, and he was specific about what he meant. Agents consume unexpected quantities of compute and tokens and costs escalate. Agents without adequate safeguards take harmful actions, and he named deleting production data as one of them. Agents are vulnerable to prompt injection, where untrusted content reaches a system that can then send information out of the company.
His diagnosis of why this happens is the part worth sitting with. Enterprises do not have enough data about what their agents are doing to make good decisions about them, so they are left with blunt instruments. A company can cap an engineer at a fixed monthly spend, but that cap cannot tell the difference between someone spending heavily because the work is producing value and someone whose agent has simply become expensive. A policy can forbid an agent from deleting files, even when deletion was a legitimate part of the task.
The Databricks answer is three words. See it, by collecting information about agents and the people using them. Show it, by putting that information in front of the people who can act on it. Act on it, by giving administrators and users real mechanisms rather than a single blunt limit.
It is a simple framework and I think it is a good one. What made the hour stay with me was not the framework but the account of what happened when they applied it to themselves, because that account contains three findings the company seems to have treated as incidental and I would treat as the main event.
What happened when the number became visible
Databricks had made coding tools available to engineers and adoption was uneven. Some of their strongest engineers were using them heavily and reporting real productivity gains, many others barely at all, and their engineering managers were among the people not using them. So the company began gathering usage data and making it visible.
The immediate effect was social rather than technical. Engineers could see that some colleagues were using these tools far more than others, and they started asking those colleagues what they were doing and whether it was working. Teams learned from each other. Zaharia describes the visibility as raising awareness, and it is worth noticing that the mechanism which actually moved behavior was people asking people, not the dashboard itself.
Then, in his own account, it produced the opposite problem. Some engineers began trying to maximize their score. They stopped considering whether an expensive model suited the task, handed the work over, and went to lunch while it ran. Costs rose, and the company responded by showing spending closer to the point of use so that an engineer could see what a session was costing while it happened.
It may be an easy reading here that Databricks was careless, no, it was not. They are among the most instrumented organizations you could choose for this. They run one of the world’s major data platforms, they have deep engineering teams, extensive traces, internal benchmarks and the ability to experiment on their own AI usage at scale. What happened to them is not a failure of sophistication. A company full of world-class engineers published a number, and people optimized the number rather than the outcome, which is Goodhart’s law arriving exactly on time. That they said so publicly is generous and unusual.
What a metric can answer, and what it cannot
Usage was worth measuring. If a company buys AI tools and nobody touches them, that is important to know, and usage is a reasonable signal of adoption. The difficulty is that usage is not the same as good use, and the gap between them is where most of the interesting questions live.
An employee can use AI constantly and badly. Another can use it rarely and exceptionally well. A third may use it for work whose value is genuinely hard to quantify. A fourth may generate enormous token consumption because the system around them is inefficient rather than because they are productive. Usage can answer questions about usage. It cannot tell you whether the use was appropriate, whether the output was sound, whether the person exercised judgment, whether the task should have gone to AI at all, or whether the organization became more capable as a result.
This matters now because enterprise AI produces a remarkable amount of measurable activity. Organizations can count licenses, active users, prompts, tokens, cost, latency, model calls, agent actions, training completions, inventoried systems, incidents and policy violations. Every one of those numbers can be useful. The risk is not that they are wrong. The risk is that an available number quietly stands in for a harder question because it is the one we happen to have.
If ninety percent of employees completed AI training, is the workforce capable of using AI well? If usage doubled, did productivity double? If an AI system has a named owner in an inventory, does that person understand what accountability requires when the system contributes to a decision that affects a customer? If there is a human in the loop, does that human have the knowledge, the authority, the time and the context to overrule the machine? The available number and the question we care about are related. They are not the same.
The measurement that pointed at the organisation
Databricks did not stop at counting usage, and the next thing they did is the reason I think their case is instructive rather than cautionary.
They built an internal benchmark for coding agents out of their own engineering work, then compared agents and configurations on both success and cost. That let them ask better questions. Was maximum reasoning effort necessary for every task? Did the harness around the model affect the result? Could a cheaper configuration do as well? The answers changed what they did. Lowering default reasoning effort and tuning the harness cut coding costs substantially without compromising quality, and, notably, without asking a single engineer to change how they worked. The most effective intervention in the whole account was invisible to the people it affected.
Then came the finding that was treated as an aside. When they sliced the benchmark results by project, they found parts of their own codebase where every agent struggled and burned tokens. The obvious interpretation was that the agents were weak there. Zaharia offered a different one: perhaps that code needed better documentation, and then the agents would do better and cost less.
Look at what happened in that sentence. An instrument built to evaluate models and harnesses produced a finding about the company. Not about the AI, but about an artifact the organization had made for itself, and a gap in it that nobody had noticed until a machine failed against it.
That is not a small thing, because AI never operates alone. It runs inside an environment that people built, made of data, documentation, workflows, permissions, incentives, review steps, escalation paths and assumptions about who is responsible for what. When an AI system performs badly the model may be the problem, and it may not. The documentation may be poor, the workflow badly designed, the task wrongly chosen for automation, the reviewer unclear about what to verify, or the responsibility so distributed that everyone assumes somebody else is checking. Those are not model problems. They are properties of the system the model sits in.
Outside engineering, the artifact disappears
Software is an unusually generous environment for this kind of measurement. You can hand an agent a task, let tests decide whether the code behaves, record what it cost, and compare configurations against each other. The feedback loop is fast and the artifact is countable.
Most of the enterprise does not work like that. In marketing, someone uses AI to analyze customers, shape a campaign, summarize research or draft copy, and the value of any single interaction may not be visible for months. In legal work, the number of prompts tells you nothing about whether the person caught a subtle error or verified a source. In finance, an answer can look entirely plausible while containing a mistake only domain knowledge would reveal. In human resources and in healthcare, the quality of the human’s interaction with the system matters as much as whether the system produced an output at all, because the output lands on a person.
Databricks’ own published guidance contains the admission that makes this concrete. Hard budgets are a last resort, they say, because cutting a developer off is debilitating and because some of the highest spenders are the people producing the most. So the ladder they recommend runs from visibility to gates to downshifting, with suspension only at the end. Every rung of it depends on telling the productive high spender from the wasteful one, and their proxy for that is pull requests shipped. It is a decent proxy and it exists in engineering because engineering produces a countable artifact on a short cycle. There is no ships-fourteen-pull-requests number in legal, in finance, in operations or in a clinic, and those functions are adopting AI at least as quickly.
Training completion is not competence
Organizations respond to all of this by investing in education, which is sensible, and education produces convenient numbers. Eighty-seven percent completed the training. Four thousand people attended the workshop. Two hundred managers finished the leadership program. Those numbers tell you that learning activity happened. They do not tell you what anyone can now do.
Education has known this for a century. Attendance is not learning, exposure is not competence, and finishing a course is not the same as applying it under real conditions. AI makes the distinction sharper, because using it well is mostly judgment rather than technique. Knowing when an answer is unsupported. Knowing what should never be typed into a system. Knowing which tasks are yours alone. Knowing that the standard of verification should rise with the cost of being wrong. Knowing that you remain accountable for a decision the machine recommended. None of that is an attendance record.
I do not want to assume this is everyone’s problem
There is a familiar danger in working on something for a long time, which is that you begin to see it everywhere.
I can construct a persuasive argument that organizations have a measurement gap around AI. That argument does not establish that anyone experiences the gap as painful, that anyone would pay to close it, or that it is urgent this year rather than eventually. And the fact that a company like Databricks can invest heavily in sophisticated measurement tells us very little about what a mid-sized healthcare provider, insurer, manufacturer or public sector organization is living through right now.
That evidence has to come from organizations themselves, which is why I am running a short study rather than writing another argument.
I am not asking anyone to predict the future. I am asking what has already happened. When AI raised a question that was difficult to answer, what was the question, and who needed the answer? What did it take to find it, and what did the difficulty cost in time, delay, friction or money? How does leadership currently work out what is happening with AI across the organization? How does anyone know whether employees have the judgment to use it well? When a customer, a board, an auditor, an investor or a regulator wants confidence in how a company uses AI, what can that company actually show them? What makes the question urgent, who owns it, and what has already been spent trying to solve it?
Those answers are worth more to me at the moment than another theory about what companies ought to need. The study takes six to ten minutes, every question is about events that have actually occurred, and I will send participants what comes back.
See if this is your problem too
Thoughts and Analysis by Suneeta, drafted by AI
Draws on Matei Zaharia and K. Sewell’s Databricks webinar on data-driven AI governance, August 2026, and on Databricks’ published accounts of benchmarking coding agents and managing AI coding costs, July and August 2026.