How we approach data quality.

Scenario design

Every dataset starts as a written specification. Generation without a specification produces volume, not coverage.

• Clinical scope: which conditions, settings, and populations are in and out.
• Constraints and assumptions: stated explicitly so they can be checked, not inferred.
• Intended tasks: what the model is supposed to do with each record.
• Difficulty levels: a target distribution from routine to multi-constraint edge cases.
• Demographic and cultural representation where appropriate to the task.
• Uncertain or missing information, deliberately included so models learn to recognise it.
• Coverage of edge cases, enumerated rather than left to chance.

Synthetic generation

Generated records are fictional by construction. They can include patient-like scenarios, clinical task descriptions, structured model responses, comparative candidates, and evaluation annotations.

• Controlled, versioned workflows so a record can be traced to the generation run that produced it.
• Schema-bound outputs so downstream checks have something concrete to validate.
• Comparative candidates generated under varied conditions for preference data.

Automated validation

The Assessment Gate: a versioned, fingerprinted contract of deterministic rules that runs on every record and removes inconsistent or unsafe output before anyone spends time reviewing it. Our current diet-plan contract has 45 rules.

• Patient-record coherence: anthropometrics, age and sex physiology, mutually exclusive diagnoses, timelines.
• Weight-indexed dose bands, such as protein in kidney disease and in children.
• Contraindicated therapeutic diets and declared-allergen screens.
• Schema validity and field completeness.
• Unit consistency and deterministic numerical constraints where the task defines them.
• Duplicate and near-duplicate detection.
• Contradiction screening against the stated constraint list.
• Task-specific rubric scoring with pass, fail, and needs-review outcomes.
• Version tracking for generation, checks, and rubrics.

Human expert review

The intended review model for commercial datasets. Coverage depends on the dataset and the agreed engagement.

• Qualified domain-specific experts matched to the clinical domain of the records they review.
• Stratified random samples sized to the dataset and the risk profile of the task.
• Targeted review of high-risk, infeasible, or flagged cases in addition to sampling.
• Disagreement resolution with a documented adjudication path.
• Explicit reviewer qualifications and protocols recorded in the quality report.
• Documented limitations of what the review did and did not cover.

Quality reporting

A dataset quality report is the artifact a buyer can audit. It should capture:

• Number of generated and accepted records.
• Automated checks run and their results.
• Human-reviewed sample counts and reviewer qualifications.
• Error categories observed.
• Acceptance criteria used.
• Known limitations.
• Task coverage against the specification.
• Dataset version and provenance.

Training vs evaluation separation

Evaluation examples are held out from training by construction, and contamination is checked rather than assumed.

• Held-out sets are generated from the same specification but never released alongside training data.
• Contamination and leakage checks compare evaluation items against training records.
• Model improvement is measured against the model's own baseline, with repeatable evaluation runs.

Provenance and rights

Commercially licensed data needs documented provenance and rights checks. Synthetic output is not automatically rights-cleared because it was generated.

• Generation sources, prompts, and reference materials are documented per dataset version.
• Rights review precedes any sample or delivery.
• Customer patient data and sensitive healthcare information are not part of the offered training inventory.