As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: turn every collected vital-signs measurement into one
record that keeps the collected test, result, and unit, and adds
the result in the test's standard unit (as a number and as text)
plus a completion status: VSTESTCD, VSTEST, VSORRES,
VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, and VSSTAT.
Input: collected vital-signs rows, each with a test code and name, the result as collected with its unit, and study, subject, sequence, and visit identifiers.
Variables:
VSTESTCDis the test code as collected:HEIGHT,WEIGHT, orTEMP.VSTESTis the test name as collected:Height,Weight, orTemperature.VSORRESis the collected result written back as text; reading it as a number drops trailing zeros (a collected72.50reads72.5), but the value itself is never converted.VSORRESUis the unit exactly as collected:cm,kg,C,LB, orF. Blank when no result was collected.VSSTRESNis the result as a number in the test's standard unit: height passes through incm; weight inLBis multiplied by0.45359237to reachkg; temperature inFbecomes (VSORRES- 32) * 5 / 9 to reachC; a result already in the standard unit passes through unchanged; a result never collected stays missing.VSSTRESCis the same standardized value written as text, so it always agrees with the numeric result. Blank when no result was collected.VSSTRESUis the test's standard unit:cmfor height,kgfor weight,Cfor temperature. A unit that does not belong to the test (a weight incm) stops the run instead of being carried through as the standard unit. Blank when no result was collected.VSSTATisNOT DONEwhen no result was collected, blank otherwise.
Note: records are grouped by test rather than kept in
collection order: all heights first, then all weights, then all
temperatures, each in collection order. A record whose result was
never collected carries VSSTAT NOT DONE, no original unit,
and no standardized value.
Inputs
Under review1 source fileinput/vs_raw.csv
Edit10 rows| STUDYID | USUBJID | VSSEQ | VISIT | VSTESTCD | VSTEST | VSORRES | VSORRESU |
|---|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | SCREENING | HEIGHT | Height | 168.2 | cm | |
| YAMAA-01 | 2 | SCREENING | WEIGHT | Weight | 72.5 | kg | |
| YAMAA-01 | 3 | SCREENING | TEMP | Temperature | 36.8 | C | |
| YAMAA-01 | 4 | WEEK 2 | TEMP | Temperature | 98.6 | F | |
| YAMAA-01 | 5 | WEEK 2 | TEMP | Temperature | 37.0 | C | |
| YAMAA-01 | 1 | SCREENING | WEIGHT | Weight | 175 | LB | |
| YAMAA-01 | 2 | SCREENING | TEMP | Temperature | 100.4 | F | |
| YAMAA-01 | 3 | WEEK 2 | TEMP | Temperature | — | F | |
| YAMAA-01 | 4 | WEEK 2 | HEIGHT | Height | 165.5 | cm | |
| YAMAA-01 | 1 | BASELINE | HEIGHT | Height | 172 | cm |
Expected output
Under review10 expected rowsexpected/vs.csv
Edit10 rows| DOMAIN | STUDYID | USUBJID | VSSEQ | VISIT | VSTESTCD | VSTEST | VSORRES | VSORRESU | VSSTRESN | VSSTRESC | VSSTRESU | VSSTAT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VS | YAMAA-01 | 1 | SCREENING | HEIGHT | Height | 168.2 | cm | 168.2 | 168.2 | cm | — | |
| VS | YAMAA-01 | 4 | WEEK 2 | HEIGHT | Height | 165.5 | cm | 165.5 | 165.5 | cm | — | |
| VS | YAMAA-01 | 1 | BASELINE | HEIGHT | Height | 172 | cm | 172 | 172 | cm | — | |
| VS | YAMAA-01 | 2 | SCREENING | WEIGHT | Weight | 72.5 | kg | 72.5 | 72.5 | kg | — | |
| VS | YAMAA-01 | 1 | SCREENING | WEIGHT | Weight | 175 | LB | 79.37866475 | 79.37866475 | kg | — | |
| VS | YAMAA-01 | 3 | SCREENING | TEMP | Temperature | 36.8 | C | 36.8 | 36.8 | C | — | |
| VS | YAMAA-01 | 4 | WEEK 2 | TEMP | Temperature | 98.6 | F | 37 | 37 | C | — | |
| VS | YAMAA-01 | 5 | WEEK 2 | TEMP | Temperature | 37 | C | 37 | 37 | C | — | |
| VS | YAMAA-01 | 2 | SCREENING | TEMP | Temperature | 100.4 | F | 38 | 38 | C | — | |
| VS | YAMAA-01 | 3 | WEEK 2 | TEMP | Temperature | — | — | — | — | — | NOT DONE |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "VS". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from VS_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from VS_RAW.USUBJID. | SDTM | 3 | ||||
| VSSEQ | Sequence Number | Num | Collected | Copy value from VS_RAW.VSSEQ. | SDTM | 4 | ||||
| VISIT | Visit Name | Char | Collected | Copy value from VS_RAW.VISIT. | SDTM | 5 | ||||
| VSTESTCD | Vital Signs Test Short Name | Char | HEIGHT, WEIGHT, TEMP | Collected | Copy value from VS_RAW.VSTESTCD. | SDTM | 6 | |||
| VSTEST | Vital Signs Test Name | Char | Collected | Copy value from VS_RAW.VSTEST. | SDTM | 7 | ||||
| VSORRES | Result or Finding in Original Units | Char | Collected | Copy value from VS_RAW.VSORRES. | SDTM | 8 | ||||
| VSORRESU | Original Units | Char | cm, kg, C, LB, F | Derived | If VS_RAW.VSORRES IS NOT NULL then "VS_RAW.VSORRESU"; otherwise blank. | SDTM | 9 | |||
| VSSTRESN | Numeric Result/Finding in Standard Units | Num | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 10 | ||||
| VSSTRESC | Character Result/Finding in Std Format | Char | Derived | Copy value from VSSTRESN. | SDTM | 11 | ||||
| VSSTRESU | Standard Units | Char | cm, kg, C | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 12 | |||
| VSSTAT | Completion Status | Char | Derived | If VS_RAW.VSORRES IS NULL then "NOT DONE"; otherwise blank. | SDTM | 13 |
| Template | Include when | VSSTRESN | VSSTRESU |
|---|---|---|---|
| height | VS_RAW.VSTESTCD = 'HEIGHT' | If VS_RAW.VSORRESU = 'cm' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND VS_RAW.VSORRESU = 'cm' then "cm"; otherwise blank |
| weight | VS_RAW.VSTESTCD = 'WEIGHT' | If VS_RAW.VSORRESU = 'LB' then compute VS_RAW.VSORRES * 0.45359237; If VS_RAW.VSORRESU = 'kg' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND (VS_RAW.VSORRESU = 'kg' OR VS_RAW.VSORRESU = 'LB') then "kg"; otherwise blank |
| temp | VS_RAW.VSTESTCD = 'TEMP' | If VS_RAW.VSORRESU = 'F' then compute (VS_RAW.VSORRES - 32) * 5 / 9; If VS_RAW.VSORRESU = 'C' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND (VS_RAW.VSORRESU = 'C' OR VS_RAW.VSORRESU = 'F') then "C"; otherwise blank |
| Variable | Permitted values |
|---|---|
| VSTESTCD | HEIGHT, WEIGHT, TEMP |
| VSORRESU | cm, kg, C, LB, F |
| VSSTRESU | cm, kg, C |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-vs-units/full.md
Edit33 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
VSSEQ is the sequence number as collected. VSTESTCD is the test code as collected: HEIGHT, WEIGHT, or TEMP. VSTEST is the test name as collected: Height, Weight, or Temperature.
VSORRES is the collected result read as a number and written back as text, so trailing zeros after the decimal point are dropped and a whole number has no decimal point; the value itself is never converted. VSORRESU is the unit exactly as collected: cm, kg, C, LB, or F, and has no value when no result was collected. VSSTRESN is the result as a number in the test's standard unit, not rounded: height passes through in cm, weight collected in LB is multiplied by 0.45359237 to give kg, and temperature collected in F becomes (F - 32) * 5 / 9 in C; a result already in the standard unit passes through unchanged, and a result never collected has no value. VSSTRESC is the same standardized value written as text with at most 15 significant digits and no trailing zeros, so a whole number has no decimal point; it has no value when no result was collected. VSSTRESU is the test's standard unit: cm for height, kg for weight, C for temperature; it has no value when no result was collected. VSSTAT is NOT DONE when no result was collected, and has no value otherwise.
A record whose result was never collected carries VSSTAT NOT DONE, no original unit, and no standardized value.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-units/conventions.md
Edit19 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
VSTESTCD is HEIGHT, WEIGHT, or TEMP. VSTEST is Height, Weight, or Temperature.
VSORRES is the collected result read as a number and written back as text, so trailing zeros after the decimal point are dropped and a whole number has no decimal point. VSSTRESN is not rounded. VSSTRESC is the same standardized value written as text with at most 15 significant digits and no trailing zeros, so a whole number has no decimal point. VSSTAT is NOT DONE or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-units/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
run.R
Edit4 lineslibrary(yamaa)vs <- yamaa_domain("spec.yaml")$outputvsrun.py
Edit4 linesimport yamaavs = yamaa.yamaa_domain("spec.yaml").outputvssolutions/sdtm-vs-units/result.R
Edit73 lines# Reference solution for the yamaa benchmark sdtm-vs-units (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)fmt_text <- function(x) { if (is.na(x)) { return(NA_character_) } if (x == floor(x)) { formatC(x, format = "f", digits = 0) } else { format(x, scientific = FALSE, trim = TRUE, digits = 15) }}fmt_orres <- function(x) { if (is.na(x) || x == "") { return(NA_character_) } num <- suppressWarnings(as.numeric(x)) if (is.na(num)) { return(x) } fmt_text(num)}raw <- read_csv( "/app/input/vs_raw.csv", col_types = cols(VSSEQ = col_integer(), .default = col_character()))vs <- raw |> mutate( has_result = !is.na(VSORRES) & VSORRES != "", VSORRES = vapply(VSORRES, fmt_orres, character(1)), value = suppressWarnings(as.numeric(raw$VSORRES)), VSSTRESN = case_when( !has_result ~ NA_real_, VSTESTCD == "HEIGHT" ~ value, VSTESTCD == "WEIGHT" & VSORRESU == "LB" ~ value * 0.45359237, VSTESTCD == "WEIGHT" ~ value, VSORRESU == "F" ~ (value - 32) * 5 / 9, .default = value ), VSSTRESC = vapply(VSSTRESN, fmt_text, character(1)), VSSTRESU = case_when( !has_result ~ NA_character_, VSTESTCD == "HEIGHT" ~ "cm", VSTESTCD == "WEIGHT" ~ "kg", .default = "C" ), VSORRESU = if_else(has_result, VSORRESU, NA_character_), VSSTAT = if_else(has_result, NA_character_, "NOT DONE"), DOMAIN = "VS", test_ord = case_when( VSTESTCD == "HEIGHT" ~ 1L, VSTESTCD == "WEIGHT" ~ 2L, .default = 3L ) ) |> arrange(test_ord, USUBJID, VSSEQ) |> select( DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT )dir.create("/app/output", showWarnings = FALSE)write_csv(vs, "/app/output/vs.csv", na = "")solutions/sdtm-vs-units/result.py
Edit101 lines# Reference solution for the yamaa benchmark sdtm-vs-units (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pldef fmt_text(x: float | None) -> str | None: if x is None: return None import math if isinstance(x, float) and (math.isnan(x) or math.isinf(x)): return None fx = float(x) if fx.is_integer(): return str(int(fx)) text = f"{fx:.15g}" return textdef fmt_orres(x: str | None) -> str | None: if x is None or x == "": return None try: return fmt_text(float(x)) except ValueError: return xraw = pl.read_csv("/app/input/vs_raw.csv", infer_schema=False).with_columns( pl.col("VSSEQ").cast(pl.Int64),)rows = raw.to_dicts()for row in rows: orres = row["VSORRES"] has_result = orres is not None and orres != "" test = row["VSTESTCD"] unit = row["VSORRESU"] if not has_result: row["VSORRES"] = None row["VSORRESU"] = None row["VSSTRESN"] = None row["VSSTRESC"] = None row["VSSTRESU"] = None row["VSSTAT"] = "NOT DONE" continue row["VSORRES"] = fmt_orres(orres) value = float(orres) if test == "HEIGHT": std = value std_unit = "cm" elif test == "WEIGHT": std = value * 0.45359237 if unit == "LB" else value std_unit = "kg" else: std = (value - 32) * 5 / 9 if unit == "F" else value std_unit = "C" row["VSSTRESN"] = std row["VSSTRESC"] = fmt_text(std) row["VSSTRESU"] = std_unit # Original unit is empty when no result was collected; otherwise as # collected. row["VSORRESU"] = unit row["VSSTAT"] = Nonevs = ( pl.DataFrame(rows) .with_columns( DOMAIN=pl.lit("VS"), _ord=pl.col("VSTESTCD").replace_strict( {"HEIGHT": 1, "WEIGHT": 2, "TEMP": 3}, default=9, return_dtype=pl.Int64, ), ) .sort(["_ord", "USUBJID", "VSSEQ"]) .select( "DOMAIN", "STUDYID", "USUBJID", "VSSEQ", "VISIT", "VSTESTCD", "VSTEST", "VSORRES", "VSORRESU", "VSSTRESN", "VSSTRESC", "VSSTRESU", "VSSTAT", ))Path("/app/output").mkdir(exist_ok=True)vs.write_csv("/app/output/vs.csv")solutions/sdtm-vs-units/result.sas
Edit31 lines/* opensas reference solution for the yamaa benchmark sdtm-vs-units. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data vs_raw; length STUDYID USUBJID VISIT VSTESTCD VSTEST VSORRES VSORRESU $1024; infile "/app/input/vs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VISIT : $char1024. VSTESTCD : $char1024. VSTEST : $char1024. VSORRES : $char1024. VSORRESU : $char1024.;run;data result; set vs_raw; length DOMAIN $2 VSSTRESC VSSTRESU VSSTAT $1024; DOMAIN='VS'; VSSTRESN=.; VSSTRESC=''; VSSTRESU=''; VSSTAT='';if missing(VSORRES) then do; VSSTAT='NOT DONE'; VSORRESU=''; end;else do; VALUE=input(VSORRES,best32.); VSORRES=strip(put(VALUE,best15.)); VSSTRESN=VALUE;if VSTESTCD='HEIGHT' then VSSTRESU='cm'; else if VSTESTCD='WEIGHT' then do; VSSTRESU='kg'; if VSORRESU='LB' then VSSTRESN=VALUE*0.45359237; end;else do; VSSTRESU='C'; if VSORRESU='F' then VSSTRESN=(VALUE-32)*5/9; end;/* Fifteen significant digits; field width alone counts the decimal point. Explicit decimal precision avoids BEST32 exposing binary rounding noise. */if VSSTRESN=0 then VSSTRESC='0';else do; DECIMALS=max(0,14-floor(log10(abs(VSSTRESN)))); VSSTRESC=strip(putn(VSSTRESN,cats('32.',put(DECIMALS,best12.)))); if index(VSSTRESC,'.')>0 then do; VSSTRESC=prxchange('s/0+$//',1,VSSTRESC); VSSTRESC=prxchange('s/\.$//',1,VSSTRESC); end;end;end; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT from result;quit;proc export data=final outfile="/app/output/vs.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions