The Jeweller Who Gave You a Number

Trust is not the same quantity as accuracy, and a great deal of harm lives in the gap between them.

Image created using AI tools

When my parents wanted gold, they did not go to a shop. They called a man. He came to the house. He knew my grandmother. He had made the pieces she was married in, and the pieces her daughters were married in, and he sat in the front room and drank tea while the design was discussed. When the ornament was ready he brought it himself, and my mother paid him, and nobody weighed anything or questioned anything, because you do not audit a man who has drunk your tea for thirty years. That is not how trust works. Trust was the whole transaction. The gold was almost an afterthought.

Let me be clear that he was not a villain. Nobody in this story is, which is rather the problem. He was, in every sense that a person can be trusted, trustworthy. He kept his word. He came when he said he would. He had a name and a face and a history with the family, and if you had asked my mother whether she trusted her jeweller she would have looked at you as though you had asked whether she trusted her own brother.

And the gold was not pure.

Not because he was a thief. Because that is simply how the trade worked, up and down the country, for longer than anyone could remember. A little copper here. A little less than promised there. Nobody checked, because there was no way to check, and so the question of purity was not really a question. It was a matter of the man’s word, and the man’s word was good, and that was the end of it.

I did not learn until much later how much of that gold was not what it was said to be. When I did, the number was startling. Across the country, when someone finally measured, a large share of the gold sold as one purity turned out to be another, lower one. Not in a few bad shops. Everywhere. In the pieces people had been married in. In the pieces they had saved their whole lives to buy. The trust had been total, and the trust had been misplaced, and those two facts had lived comfortably side by side for generations because nothing existed that could tell them apart.

That is the thing I keep coming back to, and it is the reason I am writing this.

Trust and accuracy are not the same quantity. We speak as though they were. We say we trust a thing and we mean we believe it is correct, as if the trusting and the correctness were one measurement. They are not. You can have complete trust in something that is quietly, persistently wrong, and the trust does nothing to make it right. It only makes you stop looking.

And something eventually changed it.

A company decided to sell gold differently, and it ran into exactly the wall you would expect. People did not trust it. Why would they? It had no front room, no thirty years, no tea. It was a brand, and a brand cannot be someone’s brother. Against a man who had made your mother’s wedding jewellery, a company does not stand a chance. Trust was the entire market, and trust was precisely the thing the company did not have and could not manufacture.

So it did something clever. It did not try to win the argument about trust. It changed what the argument was about.

It bought a machine that could measure the purity of gold in about a minute, without damaging the piece, and it put the machine in its stores, and it told people to bring in their own gold and test it. Not gold bought from the company. Their gold. The pieces from the man in the front room. Bring it in, it said, and let us all simply look at the number.

People came. And the number was not what they had been told.

Not occasionally, almost always. Piece after piece, gold that had been sold as one thing measured as something less, and people stood there and watched their certainty come apart on a small screen in about sixty seconds. Not because anyone argued with them. Nobody argued. The machine did not have an opinion. It just showed the number, and the number did what thirty years of trust had prevented anyone from doing. It looked.

I find that moment extraordinary because the company did not win by being more trustworthy than the jeweller. It could not have won that fight; the jeweller had thirty years and it had a logo. It won by introducing an instrument, and an instrument does something a relationship cannot. It makes trust unnecessary. You do not have to trust the number. You can see it. And the moment seeing became possible, the whole edifice of “his word is good” simply stopped being the point.

Now I will tell you why I have spent all these words on gold, because it is not really about gold.

Every organization I talk to about artificial intelligence has a man in the front room.

He is not a jeweller. He is a vendor with a confident dashboard, or a governance committee that meets and produces a report, or a consultant with good credentials and a reassuring manner, or an internal team that says the system is fine. And the organisation trusts him, in exactly the way my mother trusted her jeweller, which is to say completely and for reasons that have almost nothing to do with whether he is right. He has a name. He has been here a while. The dashboard is green, and green is a very calming colour. Everyone exhaled a long time ago.

And nobody has measured the gold.

They have measured other things. They can tell you the model’s accuracy to a decimal. They can show you uptime, throughput, cost per query. These are real numbers and they are measured honestly. But they are not the purity. They are the weight of the gold, and you can weigh a piece all day and learn nothing about whether it is what it claims to be. The organisation is measuring the thing that is easy to measure and calling it the thing that matters, and the thing that matters, whether this system can actually be trusted, whether its confidence is earned, whether anyone could prove it under questioning, sits there untested. A matter of the vendor’s word. And the vendor’s word is good. And that is the end of it.

I do not say this to accuse anyone, and I am very sure about that, because it would be the easiest thing in the world to turn the vendor into the villain and I think that is exactly the wrong lesson. The jeweller was not a villain. The vendor is usually not a villain. The problem was never that the trusted party was dishonest. The problem is that trust was doing a job it cannot do. Trust is not a measurement. It feels like one. It sits in the same place a measurement would sit, and it produces the same comfortable sensation of this has been handled. But it has not been handled. It has been trusted, and those are different things, and the difference does not announce itself. It waits.

It waits for the regulator. It waits for the incident. It waits for the day someone finally brings the piece to the machine and asks the question that thirty years of good feeling had made unaskable. And on that day, the beautiful relationship and the reassuring dashboard and the confident committee are worth exactly nothing, because none of them were ever the same quantity as the truth.

So I will leave you with the question the machine asked, because it is the only question that was ever really being asked, and it is worth asking of your own systems before someone asks it of you.

Not do you trust it. You do. That was never in doubt, and it was never the point.

Bring it to the machine. What is the number?

If you do not know, that is not a small gap. That is the whole gap. It means you have been doing what my mother did, which was reasonable, and generational, and completely sincere, and wrong. And the gentlest possible way to discover that is the way she never got to: on your own terms, in your own hand, before the day when the question is no longer yours to ask.

The right gold has never feared the test. It is the only kind worth owning.

I build instruments for measuring whether AI systems can be trusted. Which means, before anything else, I build the thing that is willing to show you the number you would rather not see, while it is still yours to do something about.

Suneeta Modekurty