As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: carry each collected vital-signs result with its test,
result, and collection date, and add the visit number, the trial
period, and the study day: VSTESTCD, VSORRES, VSDTC,
VISITNUM, EPOCH, and VSDY.
Input: collected vital-signs rows with test, result, collection date, and visit label; demographics rows carrying the reference start date; a trial-visits table carrying the planned number per visit label; and a subject-elements table carrying each subject's elements with their start and end dates.
Variables:
VISITNUMis the planned visit number from the trial-visits table for the collected visit label. An unplanned label such asUNSCHEDULEDnames no planned visit, so it takes the sponsor-assigned number: the latest planned visit number collected before the collection date plus0.01, so an unscheduled visit after visit2reads2.01.EPOCHis the subject's element whose start and end dates, both ends included, contain the collection date:SCREENING,TREATMENT, orFOLLOW-UP. A day shared by two elements belongs to the later element. Blank when no collection date or no element contains it.VSDYis the study day of the collection date, counted from the subject's reference start date: that date is day 1, there is no day zero, and dates before it count back from -1; missing when the result has no date or the subject has no reference start date.
Note: the visit number follows the visit label while the epoch follows the collection date, so an unscheduled visit still falls in an epoch, and a planned visit with no collection date has a visit number but no epoch.
Inputs
Under review4 source filesinput/dm.csv
Edit3 rows| STUDYID | USUBJID | RFSTDTC |
|---|---|---|
| CATH | 2025-01-08 | |
| CATH | 2025-02-08 | |
| CATH | 2025-03-01 |
input/se.csv
Edit9 rows| STUDYID | USUBJID | EPOCH | SESTDTC | SEENDTC |
|---|---|---|---|---|
| CATH | SCREENING | 2025-01-01 | 2025-01-07 | |
| CATH | TREATMENT | 2025-01-08 | 2025-03-04 | |
| CATH | FOLLOW-UP | 2025-03-05 | 2025-12-31 | |
| CATH | SCREENING | 2025-02-05 | 2025-02-08 | |
| CATH | TREATMENT | 2025-02-08 | 2025-04-05 | |
| CATH | FOLLOW-UP | 2025-04-06 | 2025-12-31 | |
| CATH | SCREENING | 2025-02-20 | 2025-02-28 | |
| CATH | TREATMENT | 2025-03-01 | 2025-04-26 | |
| CATH | FOLLOW-UP | 2025-04-27 | 2025-12-31 |
input/tv.csv
Edit3 rows| VISIT | VISITNUM |
|---|---|
| SCREENING | 1 |
| BASELINE | 2 |
| WEEK 4 | 3 |
input/vs_raw.csv
Edit7 rows| STUDYID | USUBJID | VSSEQ | VISIT | VSTESTCD | VSORRES | VSDTC |
|---|---|---|---|---|---|---|
| CATH | 1 | SCREENING | SYSBP | 128 | 2025-01-02 | |
| CATH | 2 | BASELINE | SYSBP | 132 | 2025-01-08 | |
| CATH | 3 | UNSCHEDULED | SYSBP | 145 | 2025-01-15 | |
| CATH | 4 | WEEK 4 | SYSBP | 130 | 2025-02-05 | |
| CATH | 1 | BASELINE | SYSBP | 118 | 2025-02-08 | |
| CATH | 2 | WEEK 4 | SYSBP | 122 | — | |
| CATH | 1 | BASELINE | SYSBP | 124 | 2025-03-01 |
Expected output
Under review7 expected rowsexpected/vs.csv
Edit7 rows| DOMAIN | STUDYID | USUBJID | VSSEQ | VSTESTCD | VSORRES | VSDTC | VISIT | VISITNUM | EPOCH | VSDY |
|---|---|---|---|---|---|---|---|---|---|---|
| VS | CATH | 1 | SYSBP | 128 | 2025-01-02 | SCREENING | 1 | SCREENING | -6 | |
| VS | CATH | 2 | SYSBP | 132 | 2025-01-08 | BASELINE | 2 | TREATMENT | 1 | |
| VS | CATH | 3 | SYSBP | 145 | 2025-01-15 | UNSCHEDULED | 2.01 | TREATMENT | 8 | |
| VS | CATH | 4 | SYSBP | 130 | 2025-02-05 | WEEK 4 | 3 | TREATMENT | 29 | |
| VS | CATH | 1 | SYSBP | 118 | 2025-02-08 | BASELINE | 2 | TREATMENT | 1 | |
| VS | CATH | 2 | SYSBP | 122 | — | WEEK 4 | 3 | — | — | |
| VS | CATH | 1 | SYSBP | 124 | 2025-03-01 | BASELINE | 2 | TREATMENT | 1 |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "VS". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from VS_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from VS_RAW.USUBJID. | SDTM | 3 | ||||
| VSSEQ | Sequence Number | Num | Collected | Copy value from VS_RAW.VSSEQ. | SDTM | 4 | ||||
| VSTESTCD | Vital Signs Test Short Name | Char | Collected | Copy value from VS_RAW.VSTESTCD. | SDTM | 5 | ||||
| VSORRES | Result or Finding in Original Units | Char | Collected | Copy value from VS_RAW.VSORRES. | SDTM | 6 | ||||
| VSDTC | Date/Time of Measurements | Char | ISO 8601 | Collected | Copy value from VS_RAW.VSDTC. | SDTM | 7 | |||
| VISIT | Visit Name | Char | Collected | Copy value from VS_RAW.VISIT. | SDTM | 8 | ||||
| VISITNUM | Visit Number | Num | Derived | First non-missing of PLANNEDNUM and SPONSORNUM. | SDTM | 9 | ||||
| VSDY | Study Day of Vital Signs | Num | Derived | CDISC study day of VSDTC against RFSTDTC as day 1; there is no day zero. | SDTM | 10 | ||||
| EPOCH | Epoch | Char | SCREENING, TREATMENT, FOLLOW-UP | Derived | Take EPOCH from the last SE record matching on USUBJID, where VSDTC falls within SESTDTC to SEENDTC, ordered by SE.SESTDTC, and no match yields blank. | SDTM | 11 |
| Variable | Permitted values |
|---|---|
| EPOCH | SCREENING, TREATMENT, FOLLOW-UP |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-vs-study-day/full.md
Edit25 linesFollowing CDISC SDTM standards, use the provided VS_RAW, DM, TV, and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, VISIT, VISITNUM, EPOCH, VSDY
VISITNUM is the planned visit number from the trial-visits table for the collected visit label. An unplanned label such as UNSCHEDULED names no planned visit, so it takes a sponsor-assigned number based on the latest planned visit number collected before the collection date, and an unscheduled visit after visit 2 reads 2.01. EPOCH is the subject's element whose start and end dates, both ends included, contain the collection date: SCREENING, TREATMENT, or FOLLOW-UP. A day shared by two elements belongs to the later element. EPOCH has no value when there is no collection date or no element contains it. VSDY is the study day of the collection date relative to the subject's reference start date.
The visit number follows the visit label while the epoch follows the collection date, so an unscheduled visit still falls in an epoch, and a planned visit with no collection date has a visit number but no epoch.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-study-day/conventions.md
Edit13 linesFollowing CDISC SDTM standards, use the provided VS_RAW, DM, TV, and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, VISIT, VISITNUM, EPOCH, VSDY
VISITNUM for an unscheduled visit after visit 2 reads 2.01. EPOCH is SCREENING, TREATMENT, or FOLLOW-UP, or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-study-day/brief.md
Edit10 linesFollowing CDISC SDTM standards, use the provided VS_RAW, DM, TV, and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, VISIT, VISITNUM, EPOCH, VSDY
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
run.R
Edit4 lineslibrary(yamaa)vs <- yamaa_domain("spec.yaml")$outputvsrun.py
Edit4 linesimport yamaavs = yamaa.yamaa_domain("spec.yaml").outputvssolutions/sdtm-vs-study-day/result.R
Edit100 lines# Reference solution for the yamaa benchmark sdtm-vs-study-day (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate)library(purrr)library(readr)study_day <- function(ref, coll) { if (is.na(ref) || is.na(coll)) { return(NA_integer_) } delta <- as.integer(coll - ref) if (delta >= 0L) delta + 1L else delta}dm <- read_csv( "/app/input/dm.csv", col_types = cols(.default = col_character()))se <- read_csv( "/app/input/se.csv", col_types = cols(.default = col_character()))tv <- read_csv( "/app/input/tv.csv", col_types = cols(.default = col_character()))vs_raw <- read_csv( "/app/input/vs_raw.csv", col_types = cols(VSSEQ = col_integer(), .default = col_character()))tv_map <- setNames(suppressWarnings(as.numeric(tv$VISITNUM)), tv$VISIT)ref_map <- setNames(suppressWarnings(ymd(dm$RFSTDTC)), dm$USUBJID)vs <- vs_raw |> mutate( coll = suppressWarnings(ymd(VSDTC)), ref = ref_map[USUBJID], VSDY = map2_int(ref, coll, study_day, .progress = FALSE), DOMAIN = "VS" )vs$VISITNUM <- pmap_dbl( list(vs$USUBJID, vs$VISIT, vs$VSDTC), function(usubjid, visit, vsdtc) { if (visit %in% names(tv_map)) { return(tv_map[[visit]]) } day <- suppressWarnings(ymd(vsdtc)) if (is.na(day)) { return(NA_real_) } prior <- vs_raw |> filter( USUBJID == usubjid, VISIT %in% names(tv_map), !is.na(suppressWarnings(ymd(VSDTC))), suppressWarnings(ymd(VSDTC)) < day ) if (nrow(prior) == 0) { return(NA_real_) } max(tv_map[prior$VISIT], na.rm = TRUE) + 0.01 }, .progress = FALSE)vs$EPOCH <- pmap_chr( list(vs$USUBJID, vs$VSDTC), function(usubjid, vsdtc) { day <- suppressWarnings(ymd(vsdtc)) if (is.na(day)) { return(NA_character_) } elements <- se |> filter(USUBJID == usubjid) hits <- elements |> filter( suppressWarnings(ymd(SESTDTC)) <= day, day <= suppressWarnings(ymd(SEENDTC)) ) |> arrange(suppressWarnings(ymd(SESTDTC))) if (nrow(hits) == 0) { return(NA_character_) } hits$EPOCH[nrow(hits)] }, .progress = FALSE)vs <- vs |> arrange(USUBJID, VSSEQ) |> select( DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, VISIT, VISITNUM, EPOCH, VSDY )dir.create("/app/output", showWarnings = FALSE)write_csv(vs, "/app/output/vs.csv", na = "")solutions/sdtm-vs-study-day/result.py
Edit121 lines# Reference solution for the yamaa benchmark sdtm-vs-study-day (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from datetime import datefrom pathlib import Pathimport polars as pldef parse_date(text: str | None) -> date | None: if text is None or text == "": return None try: # Collection and element dates are ISO dates without times. return date.fromisoformat(text[:10]) except ValueError: return Nonedef study_day(ref: date | None, coll: date | None) -> int | None: if ref is None or coll is None: return None delta = (coll - ref).days return delta + 1 if delta >= 0 else deltadm = pl.read_csv("/app/input/dm.csv", infer_schema=False)se = pl.read_csv("/app/input/se.csv", infer_schema=False)tv = pl.read_csv("/app/input/tv.csv", infer_schema=False)vs_raw = pl.read_csv("/app/input/vs_raw.csv", infer_schema=False).with_columns( pl.col("VSSEQ").cast(pl.Int64))tv_map = {row["VISIT"]: row["VISITNUM"] for row in tv.to_dicts()}ref_map = { row["USUBJID"]: parse_date(row["RFSTDTC"]) for row in dm.to_dicts()}se_by_subject: dict[str, list[dict]] = {}for row in se.to_dicts(): se_by_subject.setdefault(row["USUBJID"], []).append( { "EPOCH": row["EPOCH"], "start": parse_date(row["SESTDTC"]), "end": parse_date(row["SEENDTC"]), } )# Latest planned visit number collected before each date, per subject, for# unplanned labels such as UNSCHEDULED.planned_by_subject: dict[str, list[tuple[date, float]]] = {}for row in vs_raw.to_dicts(): if row["VISIT"] in tv_map and row["VSDTC"] not in (None, ""): coll = parse_date(row["VSDTC"]) if coll is not None: planned_by_subject.setdefault(row["USUBJID"], []).append( (coll, float(tv_map[row["VISIT"]])) )rows = []for row in vs_raw.to_dicts(): usubjid = row["USUBJID"] coll = parse_date(row["VSDTC"]) # Planned visit number, or the latest planned number before the # collection date plus .01 for an unplanned label. if row["VISIT"] in tv_map: visitnum: float | None = float(tv_map[row["VISIT"]]) else: candidates = [ num for day, num in planned_by_subject.get(usubjid, []) if day < coll ] if coll is not None else [] visitnum = max(candidates) + 0.01 if candidates else None # Element containing the collection date; a shared day belongs to the # later element. epoch = None if coll is not None: hits = [ e for e in se_by_subject.get(usubjid, []) if e["start"] is not None and e["end"] is not None and e["start"] <= coll <= e["end"] ] if hits: hits.sort(key=lambda e: e["start"]) epoch = hits[-1]["EPOCH"] vsdy = study_day(ref_map.get(usubjid), coll) rows.append( { "DOMAIN": "VS", "STUDYID": row["STUDYID"], "USUBJID": usubjid, "VSSEQ": row["VSSEQ"], "VSTESTCD": row["VSTESTCD"], "VSORRES": row["VSORRES"], "VSDTC": row["VSDTC"], "VISIT": row["VISIT"], "VISITNUM": visitnum, "EPOCH": epoch, "VSDY": vsdy, } )vs = pl.DataFrame(rows).sort(["USUBJID", "VSSEQ"]).select( "DOMAIN", "STUDYID", "USUBJID", "VSSEQ", "VSTESTCD", "VSORRES", "VSDTC", "VISIT", "VISITNUM", "EPOCH", "VSDY",)Path("/app/output").mkdir(exist_ok=True)vs.write_csv("/app/output/vs.csv")solutions/sdtm-vs-study-day/result.sas
Edit39 lines/* opensas reference solution for the yamaa benchmark sdtm-vs-study-day. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dm; length STUDYID USUBJID RFSTDTC $1024; infile "/app/input/dm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. RFSTDTC : $char1024.;run;data se; length STUDYID USUBJID EPOCH SESTDTC SEENDTC $1024; infile "/app/input/se.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. EPOCH : $char1024. SESTDTC : $char1024. SEENDTC : $char1024.;run;data tv; length VISIT $1024; infile "/app/input/tv.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input VISIT : $char1024. VISITNUM;run;data vs_raw; length STUDYID USUBJID VISIT VSTESTCD VSORRES VSDTC $1024; infile "/app/input/vs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VISIT : $char1024. VSTESTCD : $char1024. VSORRES : $char1024. VSDTC : $char1024.;run;proc sql; create table collected as select a.*,b.VISITNUM,c.RFSTDTC from vs_raw a left join tv b on a.VISIT=b.VISIT left join dm c on a.STUDYID=c.STUDYID and a.USUBJID=c.USUBJID; quit;data dated; set collected; NODATE=missing(VSDTC); run;proc sort data=dated; by STUDYID USUBJID NODATE VSDTC VSSEQ; run;data numbered; set dated; by STUDYID USUBJID; retain PRIORNUM; if first.USUBJID then PRIORNUM=.; if not missing(VISITNUM) and not missing(VSDTC) then PRIORNUM=VISITNUM; else if missing(VISITNUM) and not missing(VSDTC) and not missing(PRIORNUM) then VISITNUM=PRIORNUM+0.01; run;proc sql; create table candidates as select a.*,b.EPOCH,b.SESTDTC from numbered a left join se b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.VSDTC>=b.SESTDTC and a.VSDTC<=b.SEENDTC and not missing(a.VSDTC); quit;proc sort data=candidates; by STUDYID USUBJID VSSEQ descending SESTDTC; run;data result; length DOMAIN $2; set candidates; by STUDYID USUBJID VSSEQ; if first.VSSEQ; DOMAIN='VS'; VSDY=.; if lengthn(VSDTC)>=10 and lengthn(RFSTDTC)>=10 then do; VSDY=input(substr(VSDTC,1,10),yymmdd10.)-input(substr(RFSTDTC,1,10),yymmdd10.); if VSDY>=0 then VSDY+1; end; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, VISIT, VISITNUM, EPOCH, VSDY from result;quit;proc export data=final outfile="/app/output/vs.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions