What's Your Workflow?: Synthetic SDTM From an NCT ID

Reading time:

time

min

September 17, 2026

If you ran the same analysis twice and got two different answers back, which one would you be comfortable putting into a submission? That question is the reason this workflow exists in the form it does.

Anyone can ask a model to invent a lab dataset, and it will happily produce something that looks like LB, with plausible values and a header row that survives a glance. What it won’t produce is a full SDTM set that is conformant against real controlled terminology, carries the right required variables, and has cross-domain keys that join up. And it certainly won’t produce the same one twice. This is the third post in What’s Your Workflow?, and the first of two covering the entries Appsilon submitted to the CDISC AI Innovation Challenge 2026. Both are built in Mediforce, both are public, and both are trying to make a specific point about where AI belongs in a regulated pipeline.

Why you can’t just ask a model

Synthetic clinical data is genuinely useful, and there’s no shortage of reasons to want it. You need it to test pipelines before the real data turns up, to train people without exposing patient information, to develop validation rules against something realistic, and to run a demo without negotiating a data use agreement first.

The naive approach is to ask a model for it, and that falls over for three reasons which have a habit of compounding. The first is that conformance is structural, not cosmetic. SDTM has required variables, controlled terminology, and domain relationships, and a model producing plausible-looking text isn’t checking any of that. A dataset that fails CORE isn’t much use for testing anything that CORE is eventually going to run against. The second is that it has to be the study you claim it is, because a generic template with a study ID swapped into it isn’t a synthetic version of NCT01234567, it’s just a generic template. If the study is a twelve-week oncology trial with three arms then the data ought to reflect that, since the entire point is to test against something study-shaped.

The third reason is the one that really matters in this context, which is that generative output isn’t reproducible. Run the same prompt twice and you get two different datasets, and in a regulated setting that’s hard to defend. If you can’t regenerate the exact dataset your test suite was validated against, then the test suite has been validated against nothing in particular. The question that shapes the workflow is which parts of this problem a language model is good at, and which parts should never go anywhere near one. That’s a question about what kind of tool fits the job rather than about model quality.

How the work gets split

The workflow takes an NCT ID and returns conformant synthetic SDTM aligned to that study. It gets there by splitting the work across four different kinds of executor.

The input (human). Somebody provides a ClinicalTrials.gov NCT ID, and that’s the entire input.

Fetching the record (script). A deterministic step calls the ClinicalTrials.gov API, pulls the study record, and writes a study JSON holding the metadata. This finishes in seconds because it’s a script doing a script’s job, and there’s no good reason to involve a model in an API call.

The study configuration (agent). This is the step that genuinely needs a language model, since reading unstructured protocol prose and turning it into a structured specification is interpretation. Which domains, which visits, which arms, and what the treatment schedule looks like.

Reviewing the configuration (human). Before any data gets generated at all, a person inspects the specification and adjusts it where needed, which is worth its own section below.

Generation and validation (scripts). The approved configuration goes to synthsdtm, an open-source R package, with a fixed seed, and CDISC CORE then runs the conformance checks. Both are deterministic, so the same specification and the same seed produce the same datasets every time.

Reviewing the output (human). The reviewer sees the generated domains, the CORE findings, the conformance results, and downloadable reports, and then either approves or requests changes.

Note where the model sits in all that. It reads and interprets, and it doesn’t generate any of the data values. The data comes out of an R package with a seed attached. The reproducibility guarantee is a property of the architecture rather than a promise about how the model behaves on a given day.

Where the human sits

Mediforce sets a control mode on every step. The configuration step here runs under human review, and that gate, sitting in front of generation rather than after it, is the one carrying the weight in this workflow.

The reasoning is fairly simple. The specification determines everything downstream. If the agent misreads the protocol and configures the wrong number of arms or the wrong visit schedule, every dataset generated from it comes out wrong in a way that is internally coherent, and therefore hard to spot. You’d catch a corrupt file immediately, but you might not catch a well-formed dataset that happens to describe the wrong trial. Reviewing the spec is cheap, and reviewing 31 generated domains to work out that the spec was wrong is not, so the gate goes where the leverage is, which is before generation rather than after it. A gate in this sense means a named person at a defined step with the decision logged, which is the kind of documented oversight a regulated workflow needs.

There’s a nice bit of evidence for this in the workflow’s own history, which is that version 3 didn’t have the human review step at all. It got added later, because building the thing taught us where the gate needed to sit. The workflow definition is versioned and locked, older versions still run, and you can open V3 and V5 side by side to see exactly what changed and when. That makes it possible to answer what the process was six months ago without relying on anyone’s memory.

What a run costs

Some concrete numbers here, since they tend to be more useful than adjectives.

A current run costs somewhere around $2 in model spend. Earlier versions cost less, and the reason is worth explaining. V2 generated 5 domains, V3 generated 9, and the current version generates 31, including findings and trial design domains. More spec generation means more cost. That’s a per-run figure you can hold up against the time a programmer would spend building a synthetic dataset by hand. Mediforce tracks that spend per run and per step, so the cost doesn’t turn up as a surprise on a monthly bill. And because models get selected per step, the expensive model only runs on the step that needs interpretation. The API call and the data generation never touch a model at all. Controlled terminology integration is available when you supply a CDISC API key, and CORE conformance checking is optional, though it’s on by default in the reference workflow.

Where the model belongs

The demo is a synthetic data generator, but the design point is about architecture.

For teams working out where AI is allowed to sit inside a validated pipeline, the question that settles most of it is whether the step has to be reproducible. That tends to get skipped in favour of asking whether the model is accurate. If a downstream step has to produce reproducible, testable output, deterministic execution is easier to defend than unconstrained generation, even where the generative version looks more accurate on average. You can characterise what a script does across runs.

This workflow is a worked example of the alternative. Put the model where interpretation happens and where a human checks the result, and put deterministic code everywhere the output has to be defensible. What you end up with is a pipeline that uses AI meaningfully and still produces the same output for the same input, which means it can be validated the same way anything else gets validated. That pattern generalises well beyond synthetic data, into study setup, spec drafting, mapping proposals, and protocol deviation triage. Anywhere you’re reading prose and producing structure, the same split tends to apply.

What this means for your workflows

If you’re working out where generative AI fits into your clinical data stack, the question to bring to each step is whether it has to produce the same answer twice. Where the answer is yes, the step gets a script and a seed. Where it’s no, a model with a review gate in front of it is fine.

The workflow files, the synthsdtm package, and the generated outputs are all public, so you can clone it, point it at any NCT ID, and run it yourself.

git clone https://github.com/Appsilon/mediforce

‍

Next in this series, the second CDISC Challenge entry, which builds tables, figures, and listings where every number traces back to the study objective it answers.