Dataset Construction Checklist

A dataset construction checklist turns a modeling idea into an auditable dataset. It should be written before interpreting metrics, because most evaluation failures start as data-definition failures.

A compact construction pipeline is:

$$\mathcal{R}\stackrel{\text{curation}}{\to }{\mathcal{D}}_{\mathrm{raw}}\stackrel{P_{\varphi }}{\to }{\mathcal{D}}_{\mathrm{model}}\stackrel{s}{\to }({\mathcal{D}}_{\mathrm{train}},{\mathcal{D}}_{\mathrm{val}},{\mathcal{D}}_{\mathrm{test}})$$

where $\mathcal{R}$ is the source record set, $P_{\varphi }$ is the preprocessing contract, and $s$ is the split rule.

Checklist

• Define the example unit.
• Define the split unit.
• Write the dataset split contract.
• Write the data schema.
• State label semantics and missing-label policy.
• Record missing data, censored label, and weak label handling.
• Record source, license, collection process, and metadata provenance.
• Record data lineage from sources to final splits.
• Write the preprocessing contract.
• Decide the sampling strategy.
• Check duplicates, near-duplicates, and grouped leakage.
• Build a dataset card.
• Attach an evaluation protocol.

Molecular Modeling Checks

• Molecules: standardization, tautomer/protonation/stereo policy, scaffold grouping.
• Proteins: sequence identity, family grouping, structure source, confidence, missing regions.
• Protein-ligand complexes: receptor preparation, ligand preparation, pocket definition, pose source.
• Assays: target context, endpoint, threshold, units, censoring, and harmonization.
• Structure-based tasks: template leakage, ligand-defined pocket leakage, and deployment-available context.

Public Writing Boundary

For a public blog, describe dataset construction as a general method unless the dataset is public and the metadata is verified. Do not publish private dataset names, collaborator details, internal paths, unpublished results, or non-public task identifiers.

Related

• Data
• Dataset card
• Data lineage
• Example unit
• Split unit
• Dataset split contract
• Preprocessing contract
• Label semantics
• Missing data
• Censored label
• Weak label
• Leakage
• Evaluation protocol