As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: carry each collected vital-signs result with its test, result,
and collection date-time, and add the trial period from the subject's
own elements: VSTESTCD, VSORRES, VSDTC, and EPOCH.
Input: collected vital-signs rows with test, result, and collection date-time; and the subject's elements with the start and end of each element.
Variables:
VSDTCis the collection date-time, carried over unchanged; missing when no date was collected.EPOCHis the trial period of the element in progress at the collection date-time: the element whose start is on or before the collection and whose end is on or after it. It is missing when the reading has no date or only a partial one, or falls outside every element, such as before the first element starts.
Note: the first-dose day shows why a date-time matters: a pre-dose reading on that day belongs to screening and a post-dose reading to treatment. A date-only collection reads as the start of that day, so it falls in the element in progress at day-start; a reading exactly at a shared element boundary belongs to the later element.
Inputs
Under review2 source filesinput/se.csv
Edit8 rows| STUDYID | USUBJID | ETCD | ELEMENT | EPOCH | SESTDTC | SEENDTC |
|---|---|---|---|---|---|---|
| YAMAA-01 | SCRN | Screening | SCREENING | 2026-01-02T00:00 | 2026-01-06T09:30 | |
| YAMAA-01 | TRTA | Treatment A | TREATMENT | 2026-01-06T09:30 | 2026-02-02T00:00 | |
| YAMAA-01 | FUP | Follow-up | FOLLOW-UP | 2026-02-02T00:00 | 2026-02-17T00:00 | |
| YAMAA-01 | SCRN | Screening | SCREENING | 2026-01-03T00:00 | 2026-01-06T09:30 | |
| YAMAA-01 | TRTB | Treatment B | TREATMENT | 2026-01-06T09:30 | 2026-01-20T00:00 | |
| YAMAA-01 | FUP | Follow-up | FOLLOW-UP | 2026-01-20T00:00 | 2026-02-17T00:00 | |
| YAMAA-01 | SCRN | Screening | SCREENING | 2026-03-01T00:00 | 2026-03-05T10:00 | |
| YAMAA-01 | TRTA | Treatment A | TREATMENT | 2026-03-05T10:00 | 2026-03-19T00:00 |
input/vs_raw.csv
Edit10 rows| STUDYID | USUBJID | VSSEQ | VSTESTCD | VSORRES | VSDTC |
|---|---|---|---|---|---|
| YAMAA-01 | 1 | SYSBP | 128 | 2026-01-06T08:00 | |
| YAMAA-01 | 2 | SYSBP | 132 | 2026-01-06T10:30 | |
| YAMAA-01 | 3 | SYSBP | 130 | 2026-01-06 | |
| YAMAA-01 | 4 | SYSBP | 126 | — | |
| YAMAA-01 | 5 | SYSBP | 134 | 2026-02-02T00:00 | |
| YAMAA-01 | 6 | SYSBP | 120 | 2026-01 | |
| YAMAA-01 | 1 | SYSBP | 118 | 2026-01-06T09:30 | |
| YAMAA-01 | 2 | SYSBP | 122 | 2026-01-02 | |
| YAMAA-01 | 3 | SYSBP | 125 | 2026-01-25 | |
| YAMAA-01 | 1 | SYSBP | 135 | 2026-03-05T14:00 |
Expected output
Under review10 expected rowsexpected/vs.csv
Edit10 rows| DOMAIN | STUDYID | USUBJID | VSSEQ | VSTESTCD | VSORRES | VSDTC | EPOCH |
|---|---|---|---|---|---|---|---|
| VS | YAMAA-01 | 1 | SYSBP | 128 | 2026-01-06T08:00 | SCREENING | |
| VS | YAMAA-01 | 2 | SYSBP | 132 | 2026-01-06T10:30 | TREATMENT | |
| VS | YAMAA-01 | 3 | SYSBP | 130 | 2026-01-06 | SCREENING | |
| VS | YAMAA-01 | 4 | SYSBP | 126 | — | — | |
| VS | YAMAA-01 | 5 | SYSBP | 134 | 2026-02-02T00:00 | FOLLOW-UP | |
| VS | YAMAA-01 | 6 | SYSBP | 120 | 2026-01 | — | |
| VS | YAMAA-01 | 1 | SYSBP | 118 | 2026-01-06T09:30 | TREATMENT | |
| VS | YAMAA-01 | 2 | SYSBP | 122 | 2026-01-02 | — | |
| VS | YAMAA-01 | 3 | SYSBP | 125 | 2026-01-25 | FOLLOW-UP | |
| VS | YAMAA-01 | 1 | SYSBP | 135 | 2026-03-05T14:00 | TREATMENT |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | 2 | Assigned | Req | Set constant value "VS". | SDTM | 1 | ||
| STUDYID | Study Identifier | Char | 20 | Protocol | Req | Copy value from VS_RAW.STUDYID. | SDTM | 2 | ||
| USUBJID | Unique Subject Identifier | Char | 30 | Assigned | Req | Copy value from VS_RAW.USUBJID. | SDTM | 3 | ||
| VSSEQ | Sequence Number | Num | 8 | Assigned | Req | Copy value from VS_RAW.VSSEQ. | SDTM | 4 | ||
| VSTESTCD | Vital Signs Test Short Name | Char | 8 | Assigned | Req | Copy value from VS_RAW.VSTESTCD. | SDTM | 5 | ||
| VSORRES | Result or Finding in Original Units | Char | 20 | Assigned | Exp | Copy value from VS_RAW.VSORRES. | SDTM | 6 | ||
| VSDTC | Date/Time of Measurements | Char | ISO 8601 | Assigned | Exp | Copy value from VS_RAW.VSDTC. | SDTM | 7 | ||
| EPOCH | Epoch | Char | 10 | Epoch | Assigned | Exp | Take EPOCH from the last SE record matching on STUDYID and USUBJID, where VSDTM falls within SESTDTC to SEENDTC, ordered by SE.SESTDTC, and no match yields blank. | SDTM | 8 |
| Codelist ID | Name | Value | Decode | Codelist NCI alias | Value NCI alias | Extensible | Extended value | SAS format | Dictionary | Version |
|---|---|---|---|---|---|---|---|---|---|---|
| EPOCH | Epoch | SCREENING | Yes | |||||||
| EPOCH | Epoch | TREATMENT | Yes | |||||||
| EPOCH | Epoch | FOLLOW-UP | Yes |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-vs-epoch-from-subject-elements/full.md
Edit20 linesFollowing CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH
VSDTC is the collection date-time, carried over unchanged, and has no value when no date was collected. EPOCH is the trial period of the element in progress at the collection date-time: the element whose start is on or before the collection and whose end is on or after it. It has no value when the reading has no date or only a partial one, or when it falls outside every element, such as before the first element starts.
The first-dose day shows why a date-time matters: a pre-dose reading on that day belongs to screening and a post-dose reading to treatment. A date-only collection reads as the start of that day. A reading exactly at a shared element boundary belongs to the later element.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-epoch-from-subject-elements/conventions.md
Edit8 linesFollowing CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-epoch-from-subject-elements/brief.md
Edit8 linesFollowing CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
run.R
Edit4 lineslibrary(yamaa)vs <- yamaa_domain("spec.yaml")$outputvsrun.py
Edit4 linesimport yamaavs = yamaa.yamaa_domain("spec.yaml").outputvssolutions/sdtm-vs-epoch-from-subject-elements/result.R
Edit77 lines# Reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate)library(purrr)library(readr)parse_moment <- function(x) { if (is.na(x) || x == "") { return(as.POSIXct(NA, tz = "UTC")) } # A partial date such as YYYY-MM has no day, so it cannot be placed. if (nchar(x) == 7 && substr(x, 5, 5) == "-") { return(as.POSIXct(NA, tz = "UTC")) } if (nchar(x) == 4 && grepl("^[0-9]{4}$", x)) { return(as.POSIXct(NA, tz = "UTC")) } if (!grepl("T", x, fixed = TRUE)) { suppressWarnings(ymd_hms(paste0(x, "T00:00:00"), tz = "UTC")) } else { # Element bounds and collections use ISO datetimes without seconds. out <- suppressWarnings(ymd_hm(x, tz = "UTC")) if (is.na(out)) { out <- suppressWarnings(ymd_hms(x, tz = "UTC")) } out }}se <- read_csv( "/app/input/se.csv", col_types = cols(.default = col_character()))vs_raw <- read_csv( "/app/input/vs_raw.csv", col_types = cols(VSSEQ = col_integer(), .default = col_character()))se <- se |> mutate( start = vapply(SESTDTC, function(x) as.numeric(parse_moment(x)), numeric(1)), end = vapply(SEENDTC, function(x) as.numeric(parse_moment(x)), numeric(1)) )vs_raw <- vs_raw |> mutate(moment = vapply(VSDTC, function(x) as.numeric(parse_moment(x)), numeric(1)))vs <- vs_raw |> mutate( DOMAIN = "VS", EPOCH = pmap_chr( list(USUBJID, moment), function(usubjid, coll) { if (is.na(coll)) { return(NA_character_) } hits <- se |> filter(USUBJID == usubjid, start <= coll, coll <= end) |> arrange(start) if (nrow(hits) == 0) { return(NA_character_) } hits$EPOCH[nrow(hits)] }, .progress = FALSE ) ) |> arrange(USUBJID, VSSEQ) |> select(DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH)dir.create("/app/output", showWarnings = FALSE)write_csv(vs, "/app/output/vs.csv", na = "")solutions/sdtm-vs-epoch-from-subject-elements/result.py
Edit83 lines# Reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from datetime import datetimefrom pathlib import Pathimport polars as pldef parse_moment(text: str | None) -> datetime | None: if text is None or text == "": return None # A partial date such as YYYY-MM has no day, so it cannot be placed in # an element. if len(text) == 7 and text[4] == "-": return None if len(text) == 4 and text.isdigit(): return None try: # A date-only collection reads as the start of that day. if "T" not in text: return datetime.fromisoformat(text + "T00:00:00") return datetime.fromisoformat(text) except ValueError: return Nonese = pl.read_csv("/app/input/se.csv", infer_schema=False)vs_raw = pl.read_csv("/app/input/vs_raw.csv", infer_schema=False).with_columns( pl.col("VSSEQ").cast(pl.Int64))elements: dict[str, list[dict]] = {}for row in se.to_dicts(): elements.setdefault(row["USUBJID"], []).append( { "EPOCH": row["EPOCH"], "start": parse_moment(row["SESTDTC"]), "end": parse_moment(row["SEENDTC"]), } )rows = []for row in vs_raw.to_dicts(): coll = parse_moment(row["VSDTC"]) # Only a full date or datetime can fall in an element; empty and # partial dates stay empty, as do readings outside every element. epoch = None if coll is not None: hits = [ e for e in elements.get(row["USUBJID"], []) if e["start"] is not None and e["end"] is not None and e["start"] <= coll <= e["end"] ] if hits: hits.sort(key=lambda e: e["start"]) # A reading exactly at a shared boundary belongs to the later # element. epoch = hits[-1]["EPOCH"] rows.append( { "DOMAIN": "VS", "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "VSSEQ": row["VSSEQ"], "VSTESTCD": row["VSTESTCD"], "VSORRES": row["VSORRES"], "VSDTC": row["VSDTC"], "EPOCH": epoch, } )vs = pl.DataFrame(rows).sort(["USUBJID", "VSSEQ"]).select( "DOMAIN", "STUDYID", "USUBJID", "VSSEQ", "VSTESTCD", "VSORRES", "VSDTC", "EPOCH")Path("/app/output").mkdir(exist_ok=True)vs.write_csv("/app/output/vs.csv")solutions/sdtm-vs-epoch-from-subject-elements/result.sas
Edit25 lines/* opensas reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data se; length STUDYID USUBJID ETCD ELEMENT EPOCH SESTDTC SEENDTC $1024; infile "/app/input/se.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. ETCD : $char1024. ELEMENT : $char1024. EPOCH : $char1024. SESTDTC : $char1024. SEENDTC : $char1024.;run;data vs_raw; length STUDYID USUBJID VSTESTCD VSORRES VSDTC $1024; infile "/app/input/vs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VSTESTCD : $char1024. VSORRES : $char1024. VSDTC : $char1024.;run;data collections; set vs_raw; length DTC MOMENT $1024; DTC=VSDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); run;data elements; set se; length DTC MOMENT STARTMOMENT ENDMOMENT $1024; DTC=SESTDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); STARTMOMENT=MOMENT; DTC=SEENDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); ENDMOMENT=MOMENT; run;proc sql; create table candidates as select a.*,b.EPOCH,b.STARTMOMENT from collections a left join elements b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.MOMENT>=b.STARTMOMENT and a.MOMENT<=b.ENDMOMENT and not missing(a.MOMENT); quit;proc sort data=candidates; by STUDYID USUBJID VSSEQ descending STARTMOMENT; run;data result; length DOMAIN $2; set candidates; DOMAIN='VS'; by STUDYID USUBJID VSSEQ; if first.VSSEQ; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH from result;quit;proc export data=final outfile="/app/output/vs.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions