The pipeline passed. The data still needs proof.

Before rerunning the job, preserve the two states you need to compare. A rerun can create another state and make the original change harder to explain. The first useful question is: which two states are we actually comparing?

Pin down the before and after

“Yesterday” and “latest” sound precise until a file is replaced or a table advances to another snapshot.

Give both states durable identities. That might mean a file checksum, an object version, a Delta table version, an Iceberg snapshot, a catalog reference, or a pipeline run ID. Record the local time and timezone too. They help when logs and object histories use different clocks.

Suppose a Delta table's row-count drop occurred between versions 842 and 849. Those two numbers turn a vague report into a reproducible question. Without them, another engineer may inspect version 850 and reach a different conclusion.

Test the smallest useful explanation

An unexpected nullable column and a lower row count point to different first checks.

If a column became nullable, compare the two schemas before scanning rows. Check whether the field changed type, moved inside a nested structure, or was renamed. A CSV inference change and a Parquet schema change may look similar in a grid, but they are different events.

If the row count changed, group counts by the partition or date most likely to explain it. A full profile of every column may cost more than the original query while hiding the useful difference.

Delta and Iceberg add another question: which logical table state did the reader resolve? The newest objects in storage do not necessarily belong to the selected state. Check the version or snapshot, the metadata chain, and the data or delete files involved before blaming an individual file.

The aim is a smaller search area. A larger report can wait until the evidence calls for one.

State what the check can prove

The same finding means something different when it comes from metadata, a sample, or every relevant row.

Metadata can establish recorded schema, partition information, and available statistics. It cannot prove a claim about every row. A sample helps discover a pattern, but a missing value in the sample does not prove that the value is absent. A full scan supports stronger row-level claims, provided it covered every relevant partition and delete.

Record truncation, skipped objects, unsupported semantics, cancellation, and fallback behavior. If one of them prevents a conclusion, call the result incomplete. That is more useful than a clean-looking result with a hidden gap.

Leave evidence another engineer can challenge

A short record is enough when another engineer can understand the decision and repeat the check.

A fictional record might look like this:

Keep credentials, signed URLs, private paths, secret-bearing SQL, customer data, and private screenshots out of the record.

Before running the checks, define the finish line. Pass means the stated rule held over the recorded scope. Warning means the result needs review. Failure means the evidence disproved the rule. Incomplete means the available evidence cannot support a conclusion.

Leave enough evidence for another engineer to reproduce the result, disagree with it, and run the investigation again.