Your AI doesn't understand your business

The tools kept changing — first Excel, then dashboards, now AI. The pain stayed the same: deciding with data never got easy.

Almost every company already uses AI. Almost none see a real impact on their results.

In this article I explain why that gap doesn't close with more AI, but with a piece of work almost nobody does — building the machine a dictionary and a map of your business.

Without that, AI doesn't manage. It guesses. And it guesses with a confidence that's frightening.

Asking an AI agent questions is not the same as managing with (Artificial) Intelligence

I imagine something similar to what I see across many organizations is happening at your company too.

First it was Excel and macros. Then dashboards. Now AI. The tools kept changing, but the pain stayed the same — deciding with data never got easy, partly because our brain isn't built to process that much information at once. The numbers confirm it. According to McKinsey, nearly 90% of companies already use AI in some function, and only about 6% see a real impact on their results. That gap — between using it and it actually working — is what I want to talk about.

My thesis is uncomfortable. That gap doesn't close with more AI or better AI. It closes with a piece of work almost nobody is doing, one that's invisible, that never shows up in a demo, and that's the single thing that determines whether an Agentic AI solution will actually work for your business.

The first promise that went unfulfilled

It's worth pausing on the first two waves, because they explain what's happening now.

We asked Excel and dashboards to make the data speak for itself. To tell us where the problem was in three seconds, without anyone having to interpret it. That never happened. Dashboards multiplied, every department built its own, and decisions kept being made almost the same way as before. Gartner has been showing for years that most of the Business Intelligence (BI) licenses companies buy go unused. We bought the tool that was supposed to give us the answer, and left it in a drawer.

The same thing is about to happen with AI, and for the same underlying reason. The tool changed; the architectural data foundation it needs to actually work is still missing. The difference this time is that whoever builds that foundation is going to gain a competitive edge that's very hard to match.

We're asking AI to run a business it doesn't know

The comfortable answer is that we need more AI. I don't think that's it.

Think about it this way. If the most brilliant person in the market, with the best résumé, joined your department tomorrow, they wouldn't know how to run the operation on day one. Not because they're not capable — because they don't know the business yet. They don't know what you call "sales," they don't know how your operation is structured, they don't know who reports to whom. Before they can contribute, they need an onboarding into that organization's own business language in order to know how to make decisions.

The AI you connected to your data is exactly that. A brilliant newcomer with no onboarding. And onboarding for anyone joining a new business — whether a person or a machine — means learning two things: what things mean, and how they relate to each other.

The semantic layer and the ontological layer, in the Age of AI

These two concepts are the heart of any agentic AI implementation.

No AI reasons, at least not yet, well enough to know exactly what you mean when you say sales, market share, or turnover. Behind it there's a probabilistic model interpreting what you likely meant and building a response from that.

The semantic layer is the business dictionary — the definition of how each business variable must be interpreted. It's the practical meaning a human assigns to each piece of data. It's what makes sure that when someone says net sales, active customer, or turnover, that word means exactly one thing, with one formula and one calculation rule, across the entire company. Without that dictionary, AI interprets things differently every time someone asks, and returns a different result every time.

The ontological layer is the business map — the one that connects all those definitions to each other. It answers how each indicator connects to everything else: that an order belongs to a customer, that customer belongs to a route, that route is covered by a salesperson, that salesperson reports to a region. It's the interrelational model of every variable that explains how your business is actually built on the inside — the pieces and the threads that tie them together.

The dictionary says what a number is. The map says what it relates to and how. And here's the uncomfortable part: you can't buy that off the shelf. A new employee learns it after months of working inside the organization, with nobody writing it down for them. For AI, it has to be built, department by department, specifically for each company. That's why almost nobody has it. In a recent Deloitte survey, nearly half of companies said their main obstacle to automating with AI is that they can't find or reuse their own data. It's not a problem with the LLM. It's that the dictionary and the map are missing.

What happens when that foundation is missing

When that foundation isn't there, AI doesn't stay quiet. It hallucinates — and it hallucinates with a confidence that's frightening.

It hallucinates because of how it works under the hood. A large language model (LLM) doesn't calculate, it estimates. Faced with a question, it doesn't run a computation against your data — it predicts the most probable answer based on the specific text or query a user typed. A calculator calculates: ask it two plus two, it returns four. A model estimates, and an estimate that sounds right isn't the same as a correct number. For writing text, estimating the next word is a virtue. For telling you how much you sold last month, it's a serious flaw, because that number is a single, exact figure that has to be calculated precisely and delivered to the user — it's not a numerical approximation.

That's why ChatGPT, Claude, and/or Gemini are today a commodity for qualitative work — drafting, summarizing, organizing ideas. If I ask any of these technologies for a chocolate cake recipe, I get a different one every time and take it as good, because there isn't just one correct answer. But when I ask how much the northern region sold last month, there's only one right answer, no matter how I phrase the question. And that's exactly where they fail — on quantitative questions about business data.

And they fail in the worst possible way. If you feed an LLM several tables and ask it to calculate something, it doesn't tell you it can't. It gives you back a number, impeccably worded and with total confidence. It hallucinates silently. The problem isn't that it gets it wrong — it's that it gets it wrong without warning you, and you only find out when someone checks that number against the real data. A database sitting directly under a natural language model, with nothing in between, is one of the worst things that can happen to a business. And that's not an opinion. On real, raw enterprise schemas, even the best models in the world collapse — on the most demanding benchmark, they get it right on roughly one out of five queries (20% accuracy). And on the test that actually matters — the same battery of questions, with and without a layer — accuracy goes from close to 40% without it to more than 85% with a well-modeled semantic layer.

Same model, same question. The difference is the foundation.

MIT saw it at scale. Across hundreds of real deployments, 95% of generative AI pilots didn't move the business result. The very same people who use these tools all day for their individual work describe them as unreliable once they're plugged into company processes. They work for the individual. They break down at the organizational level. And they break down for the same reason — because inside the company, the dictionary and the map don't exist.

The value of an agent isn't in asking it something and getting an answer back. It's in the full value chain: taking raw data, turning it into an analysis that understands the business, and putting the person in a position to know what to do — how to act to create value. That chain is what will separate successful agentic AI implementations from the rest. And it doesn't hold up without the foundation.

The advantage that can't be bought

And here's what really matters.

The companies getting real value out of AI aren't the ones that handed every employee an assistant. They're the ones that put AI to work running the business, not just assisting people. Microsoft calls them frontier firms. In its annual survey, 71% of people already working at these companies feel their company is thriving, compared to 37% at the rest. That's self-reported perception, and it comes from the company selling the technology, so I take it as a signal, not proof. What is structural is the other part. An agent can only run a process if it understands it. And for it to understand it, someone had to build it the dictionary and the map.

That's the invisible advantage. It can't be bought, it can't be downloaded, it doesn't come with the license. It gets built inside each company, department by department, by people who truly know the business. It's slow and it's uncomfortable, and that's exactly why it's the hardest thing an organization has to copy today.

AI is going to be in every company. The foundation that makes it work won't be. That's where the winners will be decided — not in the model.

Sources

McKinsey, "The State of AI," March 2025. "their organizations aren't seeing a tangible impact on enterprise-level EBIT." https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-how-organizations-are-rewiring-to-capture-value

Deloitte, "Tech Trends 2026, Agentic AI strategy," 2025. "searchability of data (48%) and reusability of data (47%)." https://www.deloitte.com/us/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html

Spider 2.0 (Lei et al.), enterprise text-to-SQL benchmark, ICLR 2025. "they require significant improvement in order to achieve adequate performance for real-world enterprise usage." https://spider2-sql.github.io/

dbt Labs, "Semantic Layer vs. Text-to-SQL, 2026 Benchmark Update," April 2026. "the dbt Semantic Layer hits near-100% accuracy for covered queries." https://docs.getdbt.com/blog/semantic-layer-vs-text-to-sql-2026

MIT Media Lab, Project NANDA, "The GenAI Divide, State of AI in Business 2025," July 2025. "the vast majority remain stuck with no measurable P&L impact." https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf

Microsoft, "Work Trend Index 2025, The Year the Frontier Firm Is Born," April 2025. "71% of workers at these firms say their company is thriving." https://blogs.microsoft.com/blog/2025/04/23/the-2025-annual-work-trend-index-the-frontier-firm-is-born/