What We Learned Connecting AI to Enterprise Knowledge and Data
When Large Language Models (LLMs) first entered the mainstream, many organisations focused on the same opportunity: helping people find information more quickly.
That was certainly our starting point.
We weren't trying to build autonomous agents or digital workers. We simply wanted to make it easier for employees to access the information already sitting inside their organisation.
For most businesses, knowledge isn't the problem. Accessibility is. Policies live in SharePoint, procedures are buried in PDFs, and critical guidance often sits in documents that haven't been opened for years.
By combining enterprise knowledge with LLMs, we enabled people to ask questions in plain English instead of relying on traditional search. The results were impressive. Employees could quickly access answers from vast collections of documents they would never realistically search themselves.
Initially, it felt as though the problem had been solved.
AI Doesn't Create Information Problems. It Exposes Them.
The biggest surprise wasn't about AI at all.
Like many organisations, we expected to spend most of our time tackling hallucinations, prompt engineering and model selection. Instead, the AI started highlighting issues that already existed within the organisation.
Multiple versions of the same policy. Procedures that contradicted one another. Documents with no owner. Content that hadn't been reviewed for years.
The AI wasn't creating these problems. It was surfacing them faster than any human could.
In many ways, the assistant became the first organisation-wide test of whether knowledge was consistent, accurate and trustworthy.
The lesson was simple: the quality of any AI answer is directly linked to the quality of the information behind it. AI shines a very bright light on the state of your information estate.
Why Document Search Isn't Enough
As our work evolved, we discovered that document retrieval only solves part of the challenge.
Take two seemingly similar questions:
- What is our onboarding process?
- How many clients are currently in onboarding?
The first is a document question. The second is a data question.
Many early AI assistants perform well when retrieving information from documents but struggle when business questions require access to operational data.
Questions about revenue, capacity, active clients, outstanding actions or operational performance don't live in Word documents. They live in databases, data warehouses and business applications.
To answer questions about the business, not just its documentation, AI needs access to both knowledge and data.
The Rise of Hybrid Assistants
This realisation led us towards what we describe as hybrid assistants.
On one side sits unstructured information such as policies, procedures, correspondence and emails. On the other sits structured business data held in operational systems, databases and analytical platforms.
What's interesting is that users don't care where the information comes from. They don't know whether the answer sits in SharePoint, an ERP platform or a data warehouse. They simply want the right answer.
Hybrid assistants provide that unified experience.
They can retrieve knowledge from documents, access trusted operational data and increasingly combine the two to answer more complex business questions. The result feels far less like a search engine and far more like a digital colleague.
Trust, Governance and the Right Architecture
One of the most important lessons we've learned is that enterprise AI should not be given unrestricted access to business data.
A common instinct is to allow an LLM to generate database queries directly. In practice, this introduces risks around security, governance, consistency and explainability.
A more reliable approach is to separate responsibilities.
The AI understands intent. Trusted tools understand data.
For example, an assistant may access capabilities such as retrieving a client summary, checking revenue metrics, searching correspondence or identifying outstanding actions. The AI determines which capability is required, while the underlying platform executes trusted logic, governance controls and security policies.
This approach significantly improves reliability, transparency and trust.
Moving Beyond Search
Looking back, our journey has progressed through three distinct stages:
Search - helping people find information.
Knowledge - helping people understand information.
Reasoning - helping people answer questions that span documents, data and business systems.
Modern enterprise AI increasingly acts as an orchestration layer across knowledge repositories, data platforms, applications and workflows.
That is why we believe the future isn't simply better document search. It's AI that can reason across enterprise knowledge, enterprise data and enterprise processes.
Our first assistants answered questions about documents.
Our latest assistants answer questions about the business.
And that's where enterprise AI starts becoming genuinely transformational.
John Gamble delivered a version of this article at the Guernsey Chamber of Commerce AI Sprint on 7th October.
If you would like to discuss any of the ideas or experiences outlined above, please contact John Gamble directly.
