Is your data ready for AI?

The short answer

Ready does not mean clean. It means you can reach the right data at the moment the decision happens, with the permissions attached, and it represents the situations the system will actually face.

Most teams fail the first test and spend their budget on the third. Cleaning a warehouse you cannot query in real time is expensive preparation for a system that still will not work.

What ready actually means

Three properties, in order of how often they block projects. Reachable: available at decision time, not overnight. Governed: you know who owns it, who may see it, and where it came from. Representative: it contains the cases the system will meet, including the awkward ones.

Perfectly clean data is not on that list. Production data is never clean, and systems that require it are systems that break the first week.

The four questions

Can I get this at the moment of the decision, or only in a nightly batch. Do I know who is allowed to see each record. Does this data include the hard cases, or only the ones that went well. And is there a source of truth, or three systems that disagree.

Answer those before scoping anything. The first one decides whether the project is a data pipeline project first, which we covered in legacy integration.

Reachability is the usual blocker

A decision that happens while a user waits needs data available while they wait. If your only interface is a batch export, the design is constrained no matter what you do downstream.

There are workarounds, and they are engineering projects: a read replica, an event stream, a cache with an invalidation strategy. All of them are legitimate. None of them are free, and all of them belong in the estimate rather than in the surprise column.

Permissions and lineage

If you cannot say who is allowed to see a record, you cannot safely put it behind a system that answers questions about it. This is the failure that shows up in a security review and freezes a project after the build.

Carry permissions into retrieval rather than filtering afterwards, and keep lineage so you can answer where an answer came from. That requirement is the same one described in guardrails, and it is much cheaper designed in than retrofitted.

Representativeness

Data collected from a working process encodes that process, including its blind spots. If a category was never handled well, the record of it is thin, and a system trained or grounded on that will be worst exactly where the stakes are highest.

Look specifically for the rare and the escalated. A dataset made of clean successes teaches a system to be confident everywhere, which is the failure mode that costs the most.

The cleaning trap

The instinct is to fix everything first. That is a multi year programme that delivers nothing until it is finished, and it will not be finished.

Clean the slice the decision touches, to the standard that decision requires. A narrow, well governed, current slice beats a broad, immaculate, stale warehouse every time, and it lets you learn what actually matters before you spend.

When the answer is no

Say so and reshape the project. A data pipeline phase with a clear deliverable is a legitimate first step, and framing it honestly is far better than starting an AI build that will stall in month three for reasons everyone could see in week one.

The wrong move is proceeding on a copy and discovering the constraint at integration, which is exactly how pilots end up unable to cross.

Sequencing data work with the build

Do not serialise them. Establish reachability and permissions first, on the narrow slice, then build against that while the broader work continues.

And instrument from the beginning: if you cannot see what data was used for a given answer, you cannot debug quality later. That is the point of observability, and it is part of how we build production AI, end to end.