AI in engineering companies · The five levels of AI value

Your company already knows the answer. It just can't find it.

Catherine Ives-YimPublished 27 September 2026 · 7 min read

In every engineering organisation I have worked in, the most reliable way to find something out was to ask the person who had been there longest. The information existed somewhere: in a manual, a service report, a test log, an email thread from four years ago. Finding it took longer than asking, so people asked. When that person left, a surprising amount of the organisation's knowledge left with them.

AI now offers a practical way to fix this, through a technique called retrieval-augmented generation, usually shortened to RAG. I have been building RAG systems myself, most recently over my own body of writing. The first version taught me far more about what goes wrong than about what goes right, and that is the part most of what I read about RAG leaves out.

This article covers what RAG actually is, where it is genuinely useful in engineering and service organisations, and what to get right before you trust it.

What RAG is, precisely

It helps to be exact, because RAG is often confused with training a model on your data. It is not that.

A RAG system does two things. First, when someone asks a question, it searches your documents and retrieves the passages most likely to contain the answer. Second, it gives those passages to a language model and asks it to compose an answer based on them, with references back to the source.

The model is not changed. Your documents are not absorbed into it. They stay where they are, in an index you control, and when a manual is revised, you update the index rather than retraining anything. Every answer can point to the page it came from, so an engineer can check it.

That last point matters more than it first appears. An AI that answers from memory is guessing plausibly. An AI that answers from your documents, and shows you which ones, is doing something you can audit.

Where it earns its keep

Technical manuals and service procedures. A field engineer or workshop technician should be able to ask, in their own words, "what is the torque setting for the rear motor mount on the 2023 model?" and get the answer, the procedure it sits in, and the page reference. At the moment, most of them search a PDF, give up, and phone someone.

Standards, datasheets and specifications. Engineers spend a remarkable amount of time checking designs against documents that are long, dense and cross-referenced. A RAG system cannot replace the judgement, but it can find the relevant clause in seconds and put it next to the question being asked.

Design history. Why was this component chosen? What did we try before? What failed in testing? That knowledge usually lives in meeting notes, reports and old tickets. Made searchable in plain language, it stops teams repeating decisions that were settled, or mistakes that were already made.

Onboarding. A new engineer's first months are spent learning where things are and who to ask. A well-built system over the right documents shortens that considerably, and takes pressure off the people who would otherwise be interrupted.

Customer service logs: two different jobs

Service logs deserve their own section, because there are two distinct opportunities in them, and they need different approaches.

Answering. When a customer reports a problem, the chances are it has been seen and solved before. A RAG system over past cases, resolutions and service bulletins lets a support agent ask "has anyone seen this fault with this firmware version?" and find the cases that matched, what fixed them, and what did not. That improves first-time resolution, and it spreads the knowledge of your best support staff to everyone else.

Digesting. This is the larger prize, and strictly it is analysis rather than RAG. Thousands of free-text service records contain a detailed account of how your product actually behaves in the world. A language model can read every one of them, classify the fault, extract the product version, batch or conditions, and turn a pile of individual complaints into trends: which fault is growing, which firmware release introduced it, which component batch is over-represented. That is information engineering needs and rarely gets, because nobody has time to read the logs.

This second job is high-volume and repetitive, which makes it a good candidate for a local model. It is also where much of the value lies. A customer service log, properly digested, is one of the best sources of engineering feedback an organisation has.

What goes wrong

This is what I learned the hard way.

Your documents are messier than you think. Duplicates, superseded revisions sitting next to current ones, drafts that were never finalised. A RAG system will happily quote the 2019 version of a procedure if it is in the index. Version control of the source material is the single most important thing to get right, and it is a document management problem before it is an AI problem.

Chopping documents up breaks them. To be searchable, documents are split into passages. Split carelessly, a procedure loses its warnings, a table loses its headings, and step seven arrives without the safety note from step two. Technical documents need splitting that respects their structure. This is where most of the engineering effort in a good RAG system actually goes.

Not everything is text. Scanned pages, wiring diagrams, drawings and photographs need separate handling. Much of the most important engineering information is in exactly these forms.

People do not use the documents' words. Engineers ask with part numbers, nicknames and shorthand. Customers describe symptoms, not faults. If the search step cannot bridge that gap, the right passage is never found, and the answer is wrong.

When retrieval fails, the model still answers. This is the most dangerous failure. If the relevant passage is not found, a language model will often produce a fluent, confident answer anyway. A system used for engineering must be built to cite its sources and to say plainly when it cannot find the answer.

Access still matters. Service logs contain customer names and personal data. Some documents are restricted. A RAG system that lets anyone ask anything about everything has quietly bypassed your access controls and possibly your data protection obligations. Permissions have to follow the documents into the index.

You need to measure it. Before trusting a system, build a set of real questions with known answers and test it against them. Then keep testing as the documents and models change. Without that, you have an impressive demonstration, not a tool.

Where it should run

Everything in Where should your AI live?, later in this series, applies here, with one addition.

The index is a complete, searchable copy of your organisation's documents. It is arguably more sensitive than any single document, because it is all of them, organised. Building and holding it locally or on UK-hosted infrastructure is usually straightforward and inexpensive, because the models that create the index are small and efficient.

The answering step is a separate choice. A frontier model in the cloud writes better answers, but each question sends the retrieved passages to it. A local model keeps everything in the building, with somewhat weaker answers. Many organisations will reasonably choose differently for different document sets: cloud for the public manuals, local for the service logs and the design history. That decision should be made deliberately, collection by collection.

Start small

The most reliable way to do this is not a company-wide knowledge platform. It is one well-chosen document collection, one group of users, and one measurable improvement.

Pick a collection that people struggle with and that is reasonably well kept: the service manuals, or the last two years of support cases. Choose a measure that matters, such as time to find the right procedure or first-time fix rate. Build a test set of fifty real questions. Get that working properly, with citations and honest "I don't know" answers, and extend from there.

Four things worth taking seriously

For engineering and service leaders: your organisation's knowledge is already written down. The problem is access, not creation. RAG is the most practical way to fix access that has existed.

For whoever owns the documents: tidy the source before you index it. The system can only be as current and consistent as what it reads.

For anyone building one: invest in how documents are split, how questions are matched to them, and how the system behaves when it does not know. That is where the quality is.

For customer service: treat your logs as engineering data. The trends in them are worth more than any individual answer.

The knowledge that walks out of the door when an experienced engineer retires is not really in their head. Most of it was written down somewhere, by them or by someone they worked with. It was just never findable. That is now a solvable problem, and it is an engineering problem as much as an AI one.

I would be interested to hear where the hardest-to-find knowledge sits in your organisation, and what people currently do to find it.