As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: turn every collected vital-signs measurement into one
record that keeps the collected test, result, and unit, and adds
the result in the test's standard unit (as a number and as text)
plus a completion status: VSTESTCD, VSTEST, VSORRES,
VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, and VSSTAT.
Input: collected vital-signs rows, each with a test code and name, the result as collected with its unit, and study, subject, sequence, and visit identifiers.
Variables:
VSTESTCDis the test code as collected:HEIGHT,WEIGHT, orTEMP.VSTESTis the test name as collected:Height,Weight, orTemperature.VSORRESis the collected result written back as text; reading it as a number drops trailing zeros (a collected72.50reads72.5), but the value itself is never converted.VSORRESUis the unit exactly as collected:cm,kg,C,LB, orF. Blank when no result was collected.VSSTRESNis the result as a number in the test's standard unit: height passes through incm; weight inLBis multiplied by0.45359237to reachkg; temperature inFbecomes (VSORRES- 32) * 5 / 9 to reachC; a result already in the standard unit passes through unchanged; a result never collected stays missing.VSSTRESCis the same standardized value written as text, so it always agrees with the numeric result. Blank when no result was collected.VSSTRESUis the test's standard unit:cmfor height,kgfor weight,Cfor temperature. A unit that does not belong to the test (a weight incm) stops the run instead of being carried through as the standard unit. Blank when no result was collected.VSSTATisNOT DONEwhen no result was collected, blank otherwise.
Note: records are grouped by test rather than kept in
collection order: all heights first, then all weights, then all
temperatures, each in collection order. A record whose result was
never collected carries VSSTAT NOT DONE, no original unit,
and no standardized value.
Inputs
Under review1 source fileinput/vs_raw.csv
Edit10 rows| STUDYID | USUBJID | VSSEQ | VISIT | VSTESTCD | VSTEST | VSORRES | VSORRESU |
|---|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | SCREENING | HEIGHT | Height | 168.2 | cm | |
| YAMAA-01 | 2 | SCREENING | WEIGHT | Weight | 72.5 | kg | |
| YAMAA-01 | 3 | SCREENING | TEMP | Temperature | 36.8 | C | |
| YAMAA-01 | 4 | WEEK 2 | TEMP | Temperature | 98.6 | F | |
| YAMAA-01 | 5 | WEEK 2 | TEMP | Temperature | 37.0 | C | |
| YAMAA-01 | 1 | SCREENING | WEIGHT | Weight | 175 | LB | |
| YAMAA-01 | 2 | SCREENING | TEMP | Temperature | 100.4 | F | |
| YAMAA-01 | 3 | WEEK 2 | TEMP | Temperature | — | F | |
| YAMAA-01 | 4 | WEEK 2 | HEIGHT | Height | 165.5 | cm | |
| YAMAA-01 | 1 | BASELINE | HEIGHT | Height | 172 | cm |
Expected output
Under review10 expected rowsexpected/vs.csv
Edit10 rows| DOMAIN | STUDYID | USUBJID | VSSEQ | VISIT | VSTESTCD | VSTEST | VSORRES | VSORRESU | VSSTRESN | VSSTRESC | VSSTRESU | VSSTAT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VS | YAMAA-01 | 1 | SCREENING | HEIGHT | Height | 168.2 | cm | 168.2 | 168.2 | cm | — | |
| VS | YAMAA-01 | 4 | WEEK 2 | HEIGHT | Height | 165.5 | cm | 165.5 | 165.5 | cm | — | |
| VS | YAMAA-01 | 1 | BASELINE | HEIGHT | Height | 172 | cm | 172 | 172 | cm | — | |
| VS | YAMAA-01 | 2 | SCREENING | WEIGHT | Weight | 72.5 | kg | 72.5 | 72.5 | kg | — | |
| VS | YAMAA-01 | 1 | SCREENING | WEIGHT | Weight | 175 | LB | 79.37866475 | 79.37866475 | kg | — | |
| VS | YAMAA-01 | 3 | SCREENING | TEMP | Temperature | 36.8 | C | 36.8 | 36.8 | C | — | |
| VS | YAMAA-01 | 4 | WEEK 2 | TEMP | Temperature | 98.6 | F | 37 | 37 | C | — | |
| VS | YAMAA-01 | 5 | WEEK 2 | TEMP | Temperature | 37 | C | 37 | 37 | C | — | |
| VS | YAMAA-01 | 2 | SCREENING | TEMP | Temperature | 100.4 | F | 38 | 38 | C | — | |
| VS | YAMAA-01 | 3 | WEEK 2 | TEMP | Temperature | — | — | — | — | — | NOT DONE |
Human review and assessment
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "VS". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from VS_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from VS_RAW.USUBJID. | SDTM | 3 | ||||
| VSSEQ | Sequence Number | Num | Collected | Copy value from VS_RAW.VSSEQ. | SDTM | 4 | ||||
| VISIT | Visit Name | Char | Collected | Copy value from VS_RAW.VISIT. | SDTM | 5 | ||||
| VSTESTCD | Vital Signs Test Short Name | Char | HEIGHT, WEIGHT, TEMP | Collected | Copy value from VS_RAW.VSTESTCD. | SDTM | 6 | |||
| VSTEST | Vital Signs Test Name | Char | Collected | Copy value from VS_RAW.VSTEST. | SDTM | 7 | ||||
| VSORRES | Result or Finding in Original Units | Char | Collected | Copy value from VS_RAW.VSORRES. | SDTM | 8 | ||||
| VSORRESU | Original Units | Char | cm, kg, C, LB, F | Derived | If VS_RAW.VSORRES IS NOT NULL then "VS_RAW.VSORRESU"; otherwise blank. | SDTM | 9 | |||
| VSSTRESN | Numeric Result/Finding in Standard Units | Num | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 10 | ||||
| VSSTRESC | Character Result/Finding in Std Format | Char | Derived | Copy value from VSSTRESN. | SDTM | 11 | ||||
| VSSTRESU | Standard Units | Char | cm, kg, C | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 12 | |||
| VSSTAT | Completion Status | Char | Derived | If VS_RAW.VSORRES IS NULL then "NOT DONE"; otherwise blank. | SDTM | 13 |
| Template | Include when | VSSTRESN | VSSTRESU |
|---|---|---|---|
| height | VS_RAW.VSTESTCD = 'HEIGHT' | If VS_RAW.VSORRESU = 'cm' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND VS_RAW.VSORRESU = 'cm' then "cm"; otherwise blank |
| weight | VS_RAW.VSTESTCD = 'WEIGHT' | If VS_RAW.VSORRESU = 'LB' then compute VS_RAW.VSORRES * 0.45359237; If VS_RAW.VSORRESU = 'kg' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND (VS_RAW.VSORRESU = 'kg' OR VS_RAW.VSORRESU = 'LB') then "kg"; otherwise blank |
| temp | VS_RAW.VSTESTCD = 'TEMP' | If VS_RAW.VSORRESU = 'F' then compute (VS_RAW.VSORRES - 32) * 5 / 9; If VS_RAW.VSORRESU = 'C' then "VS_RAW.VSORRES"; otherwise blank | If VS_RAW.VSORRES IS NOT NULL AND (VS_RAW.VSORRESU = 'C' OR VS_RAW.VSORRESU = 'F') then "C"; otherwise blank |
| Variable | Permitted values |
|---|---|
| VSTESTCD | HEIGHT, WEIGHT, TEMP |
| VSORRESU | cm, kg, C, LB, F |
| VSSTRESU | cm, kg, C |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-vs-units/full.md
Edit27 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
DOMAIN is "VS". VSSEQ is the sequence number as collected. VSTESTCD is the test code as collected: "HEIGHT", "WEIGHT", or "TEMP". VSTEST is the test name as collected: "Height", "Weight", or "Temperature". VSORRES is the collected result written back as text; read as a number it drops trailing zeros (a collected 72.50 reads 72.5), and the value itself is never converted. VSORRESU is the unit exactly as collected: "cm", "kg", "C", "LB", or "F", and it has no value when no result was collected. VSSTRESN is the result as a number in the test's standard unit: a height passes through in cm, a weight in LB is multiplied by 0.45359237 to reach kg, a temperature in F becomes (the result minus 32) times 5 divided by 9 to reach C, a result already in the standard unit passes through unchanged, and a result never collected has no value. VSSTRESC is the same standardized value written as text, so it always agrees with the numeric result, and it has no value when no result was collected. VSSTRESU is the test's standard unit: "cm" for height, "kg" for weight, and "C" for temperature, and it has no value when no result was collected. VSSTAT is "NOT DONE" when no result was collected and has no value otherwise.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-units/conventions.md
Edit15 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
DOMAIN is "VS". VSTESTCD is the test code as collected: "HEIGHT", "WEIGHT", or "TEMP". VSTEST is the test name as collected: "Height", "Weight", or "Temperature". VSORRES is the collected result written back as text; read as a number it drops trailing zeros (a collected 72.50 reads 72.5). VSSTAT is "NOT DONE" or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
prompts/sdtm-vs-units/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided VS_RAW dataset to create a VS dataset with one record per collected vital-signs measurement.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT
Read the source datasets from /app/input and save the completed dataset as /app/output/vs.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/sdtm-vs-units/result.R
Edit73 lines# Reference solution for the yamaa benchmark sdtm-vs-units (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)fmt_text <- function(x) { if (is.na(x)) { return(NA_character_) } if (x == floor(x)) { formatC(x, format = "f", digits = 0) } else { format(x, scientific = FALSE, trim = TRUE, digits = 15) }}fmt_orres <- function(x) { if (is.na(x) || x == "") { return(NA_character_) } num <- suppressWarnings(as.numeric(x)) if (is.na(num)) { return(x) } fmt_text(num)}raw <- read_csv( "/app/input/vs_raw.csv", col_types = cols(VSSEQ = col_integer(), .default = col_character()))vs <- raw |> mutate( has_result = !is.na(VSORRES) & VSORRES != "", VSORRES = vapply(VSORRES, fmt_orres, character(1)), value = suppressWarnings(as.numeric(raw$VSORRES)), VSSTRESN = case_when( !has_result ~ NA_real_, VSTESTCD == "HEIGHT" ~ value, VSTESTCD == "WEIGHT" & VSORRESU == "LB" ~ value * 0.45359237, VSTESTCD == "WEIGHT" ~ value, VSORRESU == "F" ~ (value - 32) * 5 / 9, .default = value ), VSSTRESC = vapply(VSSTRESN, fmt_text, character(1)), VSSTRESU = case_when( !has_result ~ NA_character_, VSTESTCD == "HEIGHT" ~ "cm", VSTESTCD == "WEIGHT" ~ "kg", .default = "C" ), VSORRESU = if_else(has_result, VSORRESU, NA_character_), VSSTAT = if_else(has_result, NA_character_, "NOT DONE"), DOMAIN = "VS", test_ord = case_when( VSTESTCD == "HEIGHT" ~ 1L, VSTESTCD == "WEIGHT" ~ 2L, .default = 3L ) ) |> arrange(test_ord, USUBJID, VSSEQ) |> select( DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT )dir.create("/app/output", showWarnings = FALSE)write_csv(vs, "/app/output/vs.csv", na = "")solutions/sdtm-vs-units/result.py
Edit95 lines# Reference solution for the yamaa benchmark sdtm-vs-units (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build the SDTM VS dataset from VS_RAW with unit standardization.Reads /app/input/vs_raw.csv and writes /app/output/vs.csv with onerecord per collected vital-signs measurement. Standard units: cm forheight, kg for weight, C for temperature."""import csvimport osINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"STANDARD_UNIT = {"HEIGHT": "cm", "WEIGHT": "kg", "TEMP": "C"}LB_TO_KG = 0.45359237COLUMNS = [ "DOMAIN", "STUDYID", "USUBJID", "VSSEQ", "VISIT", "VSTESTCD", "VSTEST", "VSORRES", "VSORRESU", "VSSTRESN", "VSSTRESC", "VSSTRESU", "VSSTAT",]def format_number(value): """Write a number as text, dropping trailing zeros (72.50 -> 72.5).""" rounded = round(value, 10) if rounded == int(rounded): return str(int(rounded)) return repr(rounded)def standardize(testcd, value, unit): """Return the result as a number in the test's standard unit.""" if testcd == "WEIGHT" and unit == "LB": return value * LB_TO_KG if testcd == "TEMP" and unit == "F": return (value - 32) * 5 / 9 return valuedef main(): with open(os.path.join(INPUT_DIR, "vs_raw.csv"), newline="") as f: rows = list(csv.DictReader(f)) out_rows = [] for row in rows: collected = row["VSORRES"].strip() testcd = row["VSTESTCD"] if collected: value = float(collected) std_value = standardize(testcd, value, row["VSORRESU"]) orres = format_number(value) orresu = row["VSORRESU"] stresn = format_number(std_value) stresc = stresn stresu = STANDARD_UNIT[testcd] stat = "" else: orres = "" orresu = "" stresn = "" stresc = "" stresu = "" stat = "NOT DONE" out_rows.append({ "DOMAIN": "VS", "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "VSSEQ": row["VSSEQ"], "VISIT": row["VISIT"], "VSTESTCD": testcd, "VSTEST": row["VSTEST"], "VSORRES": orres, "VSORRESU": orresu, "VSSTRESN": stresn, "VSSTRESC": stresc, "VSSTRESU": stresu, "VSSTAT": stat, }) os.makedirs(OUTPUT_DIR, exist_ok=True) with open(os.path.join(OUTPUT_DIR, "vs.csv"), "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=COLUMNS) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/sdtm-vs-units/result.sas
Edit31 lines/* opensas reference solution for the yamaa benchmark sdtm-vs-units. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data vs_raw; length STUDYID USUBJID VISIT VSTESTCD VSTEST VSORRES VSORRESU $1024; infile "/app/input/vs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VISIT : $char1024. VSTESTCD : $char1024. VSTEST : $char1024. VSORRES : $char1024. VSORRESU : $char1024.;run;data result; set vs_raw; length DOMAIN $2 VSSTRESC VSSTRESU VSSTAT $1024; DOMAIN='VS'; VSSTRESN=.; VSSTRESC=''; VSSTRESU=''; VSSTAT='';if missing(VSORRES) then do; VSSTAT='NOT DONE'; VSORRESU=''; end;else do; VALUE=input(VSORRES,best32.); VSORRES=strip(put(VALUE,best15.)); VSSTRESN=VALUE;if VSTESTCD='HEIGHT' then VSSTRESU='cm'; else if VSTESTCD='WEIGHT' then do; VSSTRESU='kg'; if VSORRESU='LB' then VSSTRESN=VALUE*0.45359237; end;else do; VSSTRESU='C'; if VSORRESU='F' then VSSTRESN=(VALUE-32)*5/9; end;/* Fifteen significant digits; field width alone counts the decimal point. Explicit decimal precision avoids BEST32 exposing binary rounding noise. */if VSSTRESN=0 then VSSTRESC='0';else do; DECIMALS=max(0,14-floor(log10(abs(VSSTRESN)))); VSSTRESC=strip(putn(VSSTRESN,cats('32.',put(DECIMALS,best12.)))); if index(VSSTRESC,'.')>0 then do; VSSTRESC=prxchange('s/0+$//',1,VSSTRESC); VSSTRESC=prxchange('s/\.$//',1,VSSTRESC); end;end;end; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, VSSEQ, VISIT, VSTESTCD, VSTEST, VSORRES, VSORRESU, VSSTRESN, VSSTRESC, VSSTRESU, VSSTAT from result;quit;proc export data=final outfile="/app/output/vs.csv" dbms=csv replace; run;
yamaa