For data teams

A data-quality copilot

Q: Is this just automated data tests?

No. Tests are static rules someone wrote once and forgot. A copilot reads your lineage, watches how data actually behaves, proposes new tests where the gaps are, then triages what breaks and explains why. The tests are an output, not the whole story.

Q: Does it fix problems on its own?

It does the legwork: spots the freshness drop, traces it to the upstream job, suggests a test to stop a repeat. A person still owns the call, because some incidents are a real signal about the business, not a bug to silence.

Q: How is this different from self-healing pipelines?

Closely related, different focus. Self-healing pipelines keep the plumbing running; a quality copilot watches what flows through it (values, freshness, distributions) and judges whether the data can be trusted, not just whether the job succeeded.

An agent that proposes tests, triages incidents, and explains root cause. Quality becomes a continuous conversation, not a quarterly cleanup.

Data-quality copilot

tests, triage, root cause

continuous

Tests, running continuously

not_null(customer_id) pass

unique(order_id) pass

freshness(orders) < 6h warn 8h

row_count drift < 5% warn 7%

copilot triage

orders are late: upstream export delayed 2h. proposed: add a retry and a freshness test at 9h.

Quality becomes a continuous conversation, not a quarterly cleanup.

For years, data quality was a project. Someone noticed the numbers looked wrong, a cleanup got scheduled, a sprint went into checks, things improved, then the estate drifted, new sources arrived, and quality decayed until the next cleanup. Quality was something you did to your data periodically, usually after a report had already embarrassed someone.

A data-quality copilot makes it continuous. It sits alongside the team, reading lineage and watching how data behaves, proposing tests where coverage is thin, triaging incidents the moment they appear, explaining root cause in plain language. Quality stops being a quarterly cleanup and becomes a conversation the team has with its data every day.

Why now

Incidents are frequent and costly. Modern estates pull from dozens of sources through many transformation steps, and any one can drift, stall, or change shape. A schema change, a late load, a unit that switched from cents to dollars: each lands silently and surfaces later as a wrong number in front of someone who trusted it. The real cost is the slow erosion of trust.
Agents can now do the watching. An agent can read lineage to see what feeds what, study the shape of data over time, and propose targeted tests rather than generic ones, holding the whole graph in view, which no on-call engineer can do at 2am. An estate too tangled to watch by hand, plus something that can watch it, is what makes continuous quality realistic now.

What it looks like

A daily revenue model is suddenly twelve hours stale.

Old world: someone notices the dashboard looks off, opens a thread, and a few people guess at which upstream job is to blame. An hour of archaeology later they find it, restart it, and nobody writes anything down.

With a copilot, the freshness drop is caught the moment it happens. The agent walks the lineage backward, isolates the one upstream job that failed to land, and posts a clear note: this model is stale because EU orders ingestion didn’t complete, here’s the failed step, and here’s the test that would have caught it earlier. It proposes a freshness check with a threshold drawn from how the job actually behaves. The on-call engineer reads the triage, agrees, and accepts the test in a click. Next time, the alert fires at the source, not three layers downstream in front of finance.

The win isn’t that a machine closed a ticket. It’s that the investigation, the explanation, and the fix-to-prevent-a-repeat happened in one motion, the same instinct behind self-healing pipelines, pointed at the data flowing through rather than the plumbing itself.

Where it’s heading

Toward quality that compounds. Every incident teaches the copilot something durable: which jobs are flaky, which sources drift, which thresholds are too tight, which alerts everyone ignores. Stored as living context rather than a forgotten Slack thread, that knowledge makes the next triage faster and the next test sharper. Coverage stops being a number you fight to hold steady and becomes something that grows as the system learns, which is exactly what company memory is for.

How we think about it

Quality is a conversation, not a control. The copilot is at its best doing the tedious, wide-angle work humans are bad at: watching everything at once, tracing lineage in seconds, remembering every past incident. But the call belongs to a person. A freshness drop might be a broken job, or a real holiday lull the data is correctly reporting. An outlier might be dirty data, or the most important thing that happened this week. An agent that silently suppresses anomalies is worse than no agent, because it hides exactly the signals a team needs to see.

So the copilot proposes, explains, and remembers; the team decides. Strike that balance and quality is no longer a thing you schedule and dread; it’s an ongoing dialogue with your data where the boring half is handled and the human half is the half that matters.

Questions