Seven Questions I Ask Before Touching Any New Dataset

Early in my career I spent three weeks building a churn model on a customer table before discovering that cancelled accounts were deleted from it every 90 days. My model was learning from survivors only. It was very accurate and completely useless.

Since then I start every project with the same seven questions. They take about an hour and have saved me weeks.

The checklist

1. Who created this data, and why? A table built for billing records different things than one built for marketing. The purpose shapes what got captured and what got ignored.

1. What does one row represent? One customer, one order, one order line, one daily snapshot? I have seen teams double-count revenue because they assumed rows were orders when they were order items.

1. What gets deleted or overwritten? This is the question that would have saved my churn model. Ask specifically about soft deletes, archival jobs, and fields that are updated in place, like a status column that only shows the latest value.

1. When did the definitions change? Almost every dataset older than a year has a moment where a field changed meaning. A lead in 2022 might have required a phone call. In 2024 it might be anyone who downloaded a PDF.

1. What is missing, and is the missingness random? Blank cells are rarely random. Blank income fields often mean the customer skipped an optional step, which itself says something about them.

1. Which numbers does the business already trust? Find one figure leadership already accepts, such as last quarter's revenue, and reproduce it from the raw data. If you cannot match it within a percent or two, stop and find out why.

1. What decision will this analysis inform? If nobody can answer this, the project is not ready, no matter how interesting the data is.

How I actually run it

I do not send these as a questionnaire. I book a 45-minute call with whoever owns the system and ask them in plain language. Engineers usually answer questions 2 and 3, finance answers 6, and a manager answers 7. I take notes in a short document that lives next to the code.

A quick sanity script

For question 2, a tiny check catches a lot.

SELECT order_id, COUNT(*) AS n
FROM orders
GROUP BY order_id
HAVING COUNT(*) > 1
LIMIT 20;

If that returns rows, your table is not one row per order, whatever its name suggests.

The payoff

Asking these questions can feel slow when a stakeholder wants results by Friday. But the hour you spend here is the cheapest hour of the project. Every assumption you surface now is one you will not have to explain away in a meeting later.

Cover: Photo by Isaac Smith on Unsplash.