Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Goal: carry each collected vital-signs result with its test, result, and collection date-time, and add the trial period from the subject's own elements: VSTESTCD, VSORRES, VSDTC, and EPOCH.

Input: collected vital-signs rows with test, result, and collection date-time; and the subject's elements with the start and end of each element.

Variables:

  • VSDTC is the collection date-time, carried over unchanged; missing when no date was collected.
  • EPOCH is the trial period of the element in progress at the collection date-time: the element whose start is on or before the collection and whose end is on or after it. It is missing when the reading has no date or only a partial one, or falls outside every element, such as before the first element starts.

Note: the first-dose day shows why a date-time matters: a pre-dose reading on that day belongs to screening and a post-dose reading to treatment. A date-only collection reads as the start of that day, so it falls in the element in progress at day-start; a reading exactly at a shared element boundary belongs to the later element.

Inputs

Under review
2 source files

input/se.csv

Edit
8 rows
input/se.csv
STUDYIDUSUBJIDETCDELEMENTEPOCHSESTDTCSEENDTC
YAMAA-01SCRNScreeningSCREENING2026-01-02T00:002026-01-06T09:30
YAMAA-01TRTATreatment ATREATMENT2026-01-06T09:302026-02-02T00:00
YAMAA-01FUPFollow-upFOLLOW-UP2026-02-02T00:002026-02-17T00:00
YAMAA-01SCRNScreeningSCREENING2026-01-03T00:002026-01-06T09:30
YAMAA-01TRTBTreatment BTREATMENT2026-01-06T09:302026-01-20T00:00
YAMAA-01FUPFollow-upFOLLOW-UP2026-01-20T00:002026-02-17T00:00
YAMAA-01SCRNScreeningSCREENING2026-03-01T00:002026-03-05T10:00
YAMAA-01TRTATreatment ATREATMENT2026-03-05T10:002026-03-19T00:00

input/vs_raw.csv

Edit
10 rows
input/vs_raw.csv
STUDYIDUSUBJIDVSSEQVSTESTCDVSORRESVSDTC
YAMAA-011SYSBP1282026-01-06T08:00
YAMAA-012SYSBP1322026-01-06T10:30
YAMAA-013SYSBP1302026-01-06
YAMAA-014SYSBP126—
YAMAA-015SYSBP1342026-02-02T00:00
YAMAA-016SYSBP1202026-01
YAMAA-011SYSBP1182026-01-06T09:30
YAMAA-012SYSBP1222026-01-02
YAMAA-013SYSBP1252026-01-25
YAMAA-011SYSBP1352026-03-05T14:00

Expected output

Under review
10 expected rows

expected/vs.csv

Edit
10 rows
expected/vs.csv
DOMAINSTUDYIDUSUBJIDVSSEQVSTESTCDVSORRESVSDTCEPOCH
VSYAMAA-011SYSBP1282026-01-06T08:00SCREENING
VSYAMAA-012SYSBP1322026-01-06T10:30TREATMENT
VSYAMAA-013SYSBP1302026-01-06SCREENING
VSYAMAA-014SYSBP126——
VSYAMAA-015SYSBP1342026-02-02T00:00FOLLOW-UP
VSYAMAA-016SYSBP1202026-01—
VSYAMAA-011SYSBP1182026-01-06T09:30TREATMENT
VSYAMAA-012SYSBP1222026-01-02—
VSYAMAA-013SYSBP1252026-01-25FOLLOW-UP
VSYAMAA-011SYSBP1352026-03-05T14:00TREATMENT
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Variable NameVariable LabelTypeLengthControlled Terms or FormatOriginCoreConversion DefinitionVariable TypeVariable OrderComments for Define
DOMAINDomain AbbreviationChar2AssignedReqSet constant value "VS".SDTM1
STUDYIDStudy IdentifierChar20ProtocolReqCopy value from VS_RAW.STUDYID.SDTM2
USUBJIDUnique Subject IdentifierChar30AssignedReqCopy value from VS_RAW.USUBJID.SDTM3
VSSEQSequence NumberNum8AssignedReqCopy value from VS_RAW.VSSEQ.SDTM4
VSTESTCDVital Signs Test Short NameChar8AssignedReqCopy value from VS_RAW.VSTESTCD.SDTM5
VSORRESResult or Finding in Original UnitsChar20AssignedExpCopy value from VS_RAW.VSORRES.SDTM6
VSDTCDate/Time of MeasurementsCharISO 8601AssignedExpCopy value from VS_RAW.VSDTC.SDTM7
EPOCHEpochChar10EpochAssignedExpTake EPOCH from the last SE record matching on STUDYID and USUBJID, where VSDTM falls within SESTDTC to SEENDTC, ordered by SE.SESTDTC, and no match yields blank.SDTM8
Codelist IDNameValueDecodeCodelist NCI aliasValue NCI aliasExtensibleExtended valueSAS formatDictionaryVersion
EPOCHEpochSCREENINGYes
EPOCHEpochTREATMENTYes
EPOCHEpochFOLLOW-UPYes
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/sdtm-vs-epoch-from-subject-elements/full.md

Edit
20 lines

Following CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH

VSDTC is the collection date-time, carried over unchanged, and has no value when no date was collected. EPOCH is the trial period of the element in progress at the collection date-time: the element whose start is on or before the collection and whose end is on or after it. It has no value when the reading has no date or only a partial one, or when it falls outside every element, such as before the first element starts.

The first-dose day shows why a date-time matters: a pre-dose reading on that day belongs to screening and a post-dose reading to treatment. A date-only collection reads as the start of that day. A reading exactly at a shared element boundary belongs to the later element.

Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.

prompts/sdtm-vs-epoch-from-subject-elements/conventions.md

Edit
8 lines

Following CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH

Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.

prompts/sdtm-vs-epoch-from-subject-elements/brief.md

Edit
8 lines

Following CDISC SDTM standards, use the provided VS_RAW and SE datasets to create a VS dataset with one record per collected vital-signs result.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH

Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)vs <- yamaa_domain("spec.yaml")$outputvs

run.py

Edit
4 lines
import yamaavs = yamaa.yamaa_domain("spec.yaml").outputvs

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/sdtm-vs-epoch-from-subject-elements/result.R

Edit
77 lines
# Reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate)library(purrr)library(readr)parse_moment <- function(x) {  if (is.na(x) || x == "") {    return(as.POSIXct(NA, tz = "UTC"))  }  # A partial date such as YYYY-MM has no day, so it cannot be placed.  if (nchar(x) == 7 && substr(x, 5, 5) == "-") {    return(as.POSIXct(NA, tz = "UTC"))  }  if (nchar(x) == 4 && grepl("^[0-9]{4}$", x)) {    return(as.POSIXct(NA, tz = "UTC"))  }  if (!grepl("T", x, fixed = TRUE)) {    suppressWarnings(ymd_hms(paste0(x, "T00:00:00"), tz = "UTC"))  } else {    # Element bounds and collections use ISO datetimes without seconds.    out <- suppressWarnings(ymd_hm(x, tz = "UTC"))    if (is.na(out)) {      out <- suppressWarnings(ymd_hms(x, tz = "UTC"))    }    out  }}se <- read_csv(  "/app/input/se.csv",  col_types = cols(.default = col_character()))vs_raw <- read_csv(  "/app/input/vs_raw.csv",  col_types = cols(VSSEQ = col_integer(), .default = col_character()))se <- se |>  mutate(    start = vapply(SESTDTC, function(x) as.numeric(parse_moment(x)), numeric(1)),    end = vapply(SEENDTC, function(x) as.numeric(parse_moment(x)), numeric(1))  )vs_raw <- vs_raw |>  mutate(moment = vapply(VSDTC, function(x) as.numeric(parse_moment(x)), numeric(1)))vs <- vs_raw |>  mutate(    DOMAIN = "VS",    EPOCH = pmap_chr(      list(USUBJID, moment),      function(usubjid, coll) {        if (is.na(coll)) {          return(NA_character_)        }        hits <- se |>          filter(USUBJID == usubjid, start <= coll, coll <= end) |>          arrange(start)        if (nrow(hits) == 0) {          return(NA_character_)        }        hits$EPOCH[nrow(hits)]      },      .progress = FALSE    )  ) |>  arrange(USUBJID, VSSEQ) |>  select(DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH)dir.create("/app/output", showWarnings = FALSE)write_csv(vs, "/app/output/vs.csv", na = "")

solutions/sdtm-vs-epoch-from-subject-elements/result.py

Edit
83 lines
# Reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from datetime import datetimefrom pathlib import Pathimport polars as pldef parse_moment(text: str | None) -> datetime | None:    if text is None or text == "":        return None    # A partial date such as YYYY-MM has no day, so it cannot be placed in    # an element.    if len(text) == 7 and text[4] == "-":        return None    if len(text) == 4 and text.isdigit():        return None    try:        # A date-only collection reads as the start of that day.        if "T" not in text:            return datetime.fromisoformat(text + "T00:00:00")        return datetime.fromisoformat(text)    except ValueError:        return Nonese = pl.read_csv("/app/input/se.csv", infer_schema=False)vs_raw = pl.read_csv("/app/input/vs_raw.csv", infer_schema=False).with_columns(    pl.col("VSSEQ").cast(pl.Int64))elements: dict[str, list[dict]] = {}for row in se.to_dicts():    elements.setdefault(row["USUBJID"], []).append(        {            "EPOCH": row["EPOCH"],            "start": parse_moment(row["SESTDTC"]),            "end": parse_moment(row["SEENDTC"]),        }    )rows = []for row in vs_raw.to_dicts():    coll = parse_moment(row["VSDTC"])    # Only a full date or datetime can fall in an element; empty and    # partial dates stay empty, as do readings outside every element.    epoch = None    if coll is not None:        hits = [            e            for e in elements.get(row["USUBJID"], [])            if e["start"] is not None            and e["end"] is not None            and e["start"] <= coll <= e["end"]        ]        if hits:            hits.sort(key=lambda e: e["start"])            # A reading exactly at a shared boundary belongs to the later            # element.            epoch = hits[-1]["EPOCH"]    rows.append(        {            "DOMAIN": "VS",            "STUDYID": row["STUDYID"],            "USUBJID": row["USUBJID"],            "VSSEQ": row["VSSEQ"],            "VSTESTCD": row["VSTESTCD"],            "VSORRES": row["VSORRES"],            "VSDTC": row["VSDTC"],            "EPOCH": epoch,        }    )vs = pl.DataFrame(rows).sort(["USUBJID", "VSSEQ"]).select(    "DOMAIN", "STUDYID", "USUBJID", "VSSEQ", "VSTESTCD", "VSORRES", "VSDTC", "EPOCH")Path("/app/output").mkdir(exist_ok=True)vs.write_csv("/app/output/vs.csv")

solutions/sdtm-vs-epoch-from-subject-elements/result.sas

Edit
25 lines
/* opensas reference solution for the yamaa benchmark sdtm-vs-epoch-from-subject-elements.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data se;  length STUDYID USUBJID ETCD ELEMENT EPOCH SESTDTC SEENDTC $1024;  infile "/app/input/se.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. ETCD : $char1024. ELEMENT : $char1024. EPOCH : $char1024. SESTDTC : $char1024. SEENDTC : $char1024.;run;data vs_raw;  length STUDYID USUBJID VSTESTCD VSORRES VSDTC $1024;  infile "/app/input/vs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VSTESTCD : $char1024. VSORRES : $char1024. VSDTC : $char1024.;run;data collections; set vs_raw; length DTC MOMENT $1024; DTC=VSDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); run;data elements; set se; length DTC MOMENT STARTMOMENT ENDMOMENT $1024; DTC=SESTDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); STARTMOMENT=MOMENT; DTC=SEENDTC; MOMENT=''; if lengthn(DTC)=10 then MOMENT=cats(DTC,'T00:00:00'); else if lengthn(DTC)=16 then MOMENT=cats(DTC,':00'); else if lengthn(DTC)>=19 then MOMENT=substr(DTC,1,19); ENDMOMENT=MOMENT; run;proc sql; create table candidates as select a.*,b.EPOCH,b.STARTMOMENT from collections a left join elements b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.MOMENT>=b.STARTMOMENT and a.MOMENT<=b.ENDMOMENT and not missing(a.MOMENT); quit;proc sort data=candidates; by STUDYID USUBJID VSSEQ descending STARTMOMENT; run;data result; length DOMAIN $2; set candidates; DOMAIN='VS'; by STUDYID USUBJID VSSEQ; if first.VSSEQ; run;proc sql;  create table final as select DOMAIN, STUDYID, USUBJID, VSSEQ, VSTESTCD, VSORRES, VSDTC, EPOCH from result;quit;proc export data=final outfile="/app/output/vs.csv" dbms=csv replace; run;