As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: give each electrocardiogram (ECG) time-point record of a QT study
its time-matched baseline: flag the baseline records (ABLFL), carry the
baseline value (BASE), compute the change from baseline (CHG), and name
the baseline each record is compared with (BASETYPE).
Input: one record per subject, parameter, analysis period (APERIOD),
nominal day within the period (APERDAY) and nominal time point (ATPT,
with its number of hours after the reference dose time in ATPTN), each
holding the average of the replicate ECGs taken at that time (AVAL, unit
AVALU).
Variables:
ABLFL:Yon every record of day -1 of a period, that period's baseline day; no value on every other day.BASE: the value recorded on the same period's baseline day at the same nominal time point for the same subject and parameter; on the baseline day, the record's own value. No value when that baseline day did not record the time point or the period has no baseline day; nothing stands in for it, neither another time point, day or period nor the dosing day's pre-dose value.CHG:AVALminusBASE, not rounded; no value on the baseline day and whereverBASEhas no value.BASETYPE: "Period " followed by the period number, " Baseline Day " and the time point, for example "Period 2 Baseline Day 2 h", on every record, including those with no baseline.
Note: every period has its own baseline day, so a subject has one baseline record per period and time point, and each of them has a baseline type of its own.
Inputs
Under review1 source fileinput/adeg_raw.csv
Edit17 rows| STUDYID | USUBJID | PARAMCD | PARAM | APERIOD | APERDAY | ATPT | ATPTN | AVAL | AVALU |
|---|---|---|---|---|---|---|---|---|---|
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 0 h | 0 | 398.000 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 401.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 0 h | 0 | 399.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 407.000 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 0 h | 0 | 400.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 2 h | 2 | 409.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 392.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 398.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 2 | -1 | 2 h | 2 | 395.000 | msec | |
| TQT-01 | QTCF | QTcF Interval | 2 | 1 | 2 h | 2 | 404.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 0 h | 0 | 410.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 0 h | 0 | 412.000 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 418.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 2 h | 2 | 421.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 385.667 | msec | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 390.333 | msec | |
| TQT-01 | QTCF | QTcF Interval | 2 | 1 | 2 h | 2 | 393.000 | msec |
Expected output
Under review17 expected rowsexpected/adeg.csv
Edit17 rows| STUDYID | USUBJID | PARAMCD | PARAM | APERIOD | APERDAY | ATPT | ATPTN | AVAL | AVALU | ABLFL | BASE | CHG | BASETYPE |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 0 h | 0 | 398 | msec | Y | 398 | — | Period 1 Baseline Day 0 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 401.333 | msec | Y | 401.333 | — | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 0 h | 0 | 399.667 | msec | — | 398 | 1.6669999999999732 | Period 1 Baseline Day 0 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 407 | msec | — | 401.333 | 5.666999999999973 | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 0 h | 0 | 400.333 | msec | — | 398 | 2.333000000000027 | Period 1 Baseline Day 0 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 2 h | 2 | 409.667 | msec | — | 401.333 | 8.333999999999946 | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 392.667 | msec | Y | 392.667 | — | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 398.333 | msec | — | 392.667 | 5.666000000000054 | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 2 | -1 | 2 h | 2 | 395 | msec | Y | 395 | — | Period 2 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 2 | 1 | 2 h | 2 | 404.667 | msec | — | 395 | 9.666999999999973 | Period 2 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 0 h | 0 | 410.333 | msec | Y | 410.333 | — | Period 1 Baseline Day 0 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 0 h | 0 | 412 | msec | — | 410.333 | 1.6669999999999732 | Period 1 Baseline Day 0 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 418.667 | msec | — | — | — | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 2 | 2 h | 2 | 421.333 | msec | — | — | — | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | -1 | 2 h | 2 | 385.667 | msec | Y | 385.667 | — | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 1 | 1 | 2 h | 2 | 390.333 | msec | — | 385.667 | 4.666000000000054 | Period 1 Baseline Day 2 h | |
| TQT-01 | QTCF | QTcF Interval | 2 | 1 | 2 h | 2 | 393 | msec | — | — | — | Period 2 Baseline Day 2 h |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADEG | STUDYID | Study Identifier | character | Copy value from ADEG_RAW.STUDYID. | Predecessor | ||
| ADEG | USUBJID | Unique Subject Identifier | character | Copy value from ADEG_RAW.USUBJID. | Predecessor | ||
| ADEG | PARAMCD | Parameter Code | character | Copy value from ADEG_RAW.PARAMCD. | Predecessor | ||
| ADEG | PARAM | Parameter | character | Copy value from ADEG_RAW.PARAM. | Predecessor | ||
| ADEG | APERIOD | Period | numeric | Copy value from ADEG_RAW.APERIOD. | Predecessor | ||
| ADEG | APERDAY | Analysis Nominal Period Day | numeric | Copy value from ADEG_RAW.APERDAY. | Predecessor | ||
| ADEG | ATPT | Analysis Time Point | character | Copy value from ADEG_RAW.ATPT. | Predecessor | ||
| ADEG | ATPTN | Analysis Time Point (N) | numeric | Copy value from ADEG_RAW.ATPTN. | Predecessor | ||
| ADEG | AVAL | Analysis Value | numeric | Copy value from ADEG_RAW.AVAL. | Predecessor | ||
| ADEG | AVALU | Analysis Value Unit | character | Copy value from ADEG_RAW.AVALU. | Predecessor | ||
| ADEG | ABLFL | Baseline Record Flag | character | Y | If APERDAY = -1 then "Y"; otherwise blank. | Derived | |
| ADEG | BASE | Baseline Value | numeric | Aggregate ONLY(AVAL) over records grouped by STUDYID, USUBJID, PARAMCD, APERIOD, ATPTN, matching on the shared output keys, where ABLFL = 'Y'. | Derived | ||
| ADEG | CHG | Change from Baseline | numeric | If ABLFL IS NULL then compute AVAL - BASE; otherwise blank. | Derived | ||
| ADEG | BASETYPE | Baseline Type | character | Fill the template "Period {APERIODC} Baseline Day {ATPT}". | Derived |
| Variable | Permitted values |
|---|---|
| ABLFL | Y |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adeg-time-matched-baseline/full.md
Edit23 linesFollowing CDISC ADaM standards, use the provided ADEG_RAW dataset to create an ADEG dataset with one record per subject, parameter, analysis period, nominal day within the period, and nominal time point.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, APERIOD, APERDAY, ATPT, ATPTN, AVAL, AVALU, ABLFL, BASE, CHG, BASETYPE
- ABLFL is "Y" on every record of day -1 of a period, that period's baseline day; it has no value on every other day.
- BASE is the AVAL recorded at the same nominal time point on the same period's baseline day for the same subject and parameter; on the baseline day, BASE is the record's own AVAL.
- BASE has no value when that baseline day did not record the time point or the period has no baseline day; nothing stands in for it, neither another time point, day or period nor the dosing day's pre-dose value.
- CHG is AVAL minus BASE and is not rounded; CHG has no value on the baseline day or wherever BASE has no value.
- BASETYPE is "Period " followed by the period number, " Baseline Day " and ATPT on every record, including those with no baseline.
Read the source datasets from /app/input and save the completed dataset as /app/output/adeg.csv.
prompts/adam-adeg-time-matched-baseline/conventions.md
Edit13 linesFollowing CDISC ADaM standards, use the provided ADEG_RAW dataset to create an ADEG dataset with one record per subject, parameter, analysis period, nominal day within the period, and nominal time point.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, APERIOD, APERDAY, ATPT, ATPTN, AVAL, AVALU, ABLFL, BASE, CHG, BASETYPE
ABLFL is "Y" or has no value. CHG is not rounded. BASETYPE is "Period " followed by the period number, " Baseline Day " and ATPT.
Read the source datasets from /app/input and save the completed dataset as /app/output/adeg.csv.
prompts/adam-adeg-time-matched-baseline/brief.md
Edit10 linesFollowing CDISC ADaM standards, use the provided ADEG_RAW dataset to create an ADEG dataset with one record per subject, parameter, analysis period, nominal day within the period, and nominal time point.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, APERIOD, APERDAY, ATPT, ATPTN, AVAL, AVALU, ABLFL, BASE, CHG, BASETYPE
Read the source datasets from /app/input and save the completed dataset as /app/output/adeg.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-adeg-time-matched-baseline/result.R
Edit44 lines# Reference solution for the yamaa benchmark adam-adeg-time-matched-baseline (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)adeg <- read_csv( "/app/input/adeg_raw.csv", col_types = cols( APERIOD = col_integer(), APERDAY = col_integer(), ATPTN = col_double(), AVAL = col_double(), .default = col_character() ), na = "")# Day -1 of each period is that period's baseline day.baseline <- adeg |> filter(APERDAY == -1) |> select(STUDYID, USUBJID, PARAMCD, APERIOD, ATPTN, BASE = AVAL)# Time-matched: the same subject, parameter, period and nominal time on the# period's own baseline day; nothing stands in for a missing one.adeg_out <- adeg |> left_join( baseline, by = c("STUDYID", "USUBJID", "PARAMCD", "APERIOD", "ATPTN"), relationship = "many-to-one" ) |> mutate( ABLFL = if_else(APERDAY == -1, "Y", NA_character_), CHG = if_else(APERDAY == -1, NA_real_, AVAL - BASE), BASETYPE = paste0("Period ", APERIOD, " Baseline Day ", ATPT) ) |> select( STUDYID, USUBJID, PARAMCD, PARAM, APERIOD, APERDAY, ATPT, ATPTN, AVAL, AVALU, ABLFL, BASE, CHG, BASETYPE )dir.create("/app/output", showWarnings = FALSE)write_csv(adeg_out, "/app/output/adeg.csv", na = "")solutions/adam-adeg-time-matched-baseline/result.py
Edit63 lines# Reference solution for the yamaa benchmark adam-adeg-time-matched-baseline (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Create ADEG from ADEG_RAW with time-matched period baselines.For each subject, parameter and analysis period, the baseline day isAPERDAY -1. BASE is the AVAL recorded at the same nominal time point onthat period's baseline day; CHG is AVAL - BASE (blank on the baselineday itself and wherever no baseline value exists)."""import csvfrom decimal import DecimalINPUT_PATH = "/app/input/adeg_raw.csv"OUTPUT_PATH = "/app/output/adeg.csv"FIELDS = [ "STUDYID", "USUBJID", "PARAMCD", "PARAM", "APERIOD", "APERDAY", "ATPT", "ATPTN", "AVAL", "AVALU", "ABLFL", "BASE", "CHG", "BASETYPE",]def main(): with open(INPUT_PATH, newline="") as f: rows = list(csv.DictReader(f)) # AVAL on each period's baseline day, by subject/parameter/period/time point. baseline = {} for row in rows: if row["APERDAY"] == "-1": key = (row["USUBJID"], row["PARAMCD"], row["APERIOD"], row["ATPT"]) baseline[key] = row["AVAL"] out_rows = [] for row in rows: is_baseline_day = row["APERDAY"] == "-1" key = (row["USUBJID"], row["PARAMCD"], row["APERIOD"], row["ATPT"]) base = baseline.get(key, "") if is_baseline_day or base == "": chg = "" else: chg = str(Decimal(row["AVAL"]) - Decimal(base)) out = {name: row[name] for name in FIELDS[:10]} out["ABLFL"] = "Y" if is_baseline_day else "" out["BASE"] = base out["CHG"] = chg out["BASETYPE"] = ( "Period " + row["APERIOD"] + " Baseline Day " + row["ATPT"] ) out_rows.append(out) with open(OUTPUT_PATH, "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=FIELDS) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/adam-adeg-time-matched-baseline/result.sas
Edit48 lines/* opensas reference solution for the yamaa benchmark adam-adeg-time-matched-baseline. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data adeg; length STUDYID USUBJID PARAMCD PARAM ATPT AVALU $1024; infile "/app/input/adeg_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. PARAMCD : $char1024. PARAM : $char1024. APERIOD APERDAY ATPT : $char1024. ATPTN AVAL AVALU : $char1024.;run;/* Day -1 of each period is that period's baseline day; its records are the time-matched baselines, one per subject, parameter, period and time. */data baseline; length B_STUDYID B_USUBJID B_PARAMCD $1024; set adeg; if APERDAY = -1; B_STUDYID = STUDYID; B_USUBJID = USUBJID; B_PARAMCD = PARAMCD; B_APERIOD = APERIOD; B_ATPTN = ATPTN; BASE = AVAL; keep B_STUDYID B_USUBJID B_PARAMCD B_APERIOD B_ATPTN BASE;run;proc sql; create table matched as select a.*, b.BASE from adeg as a left join baseline as b on a.STUDYID = b.B_STUDYID and a.USUBJID = b.B_USUBJID and a.PARAMCD = b.B_PARAMCD and a.APERIOD = b.B_APERIOD and a.ATPTN = b.B_ATPTN;quit;/* No change on the baseline day, and none without a baseline. */data final; length ABLFL $1 BASETYPE $1024; set matched; if APERDAY = -1 then ABLFL = 'Y'; else ABLFL = ''; if APERDAY ne -1 and not missing(BASE) and not missing(AVAL) then CHG = AVAL - BASE; else CHG = .; BASETYPE = 'Period ' || strip(put(APERIOD, 8.)) || ' Baseline Day ' || strip(ATPT);run;proc sql; create table out as select STUDYID, USUBJID, PARAMCD, PARAM, APERIOD, APERDAY, ATPT, ATPTN, AVAL, AVALU, ABLFL, BASE, CHG, BASETYPE from final;quit;proc export data=out outfile="/app/output/adeg.csv" dbms=csv replace; run;
yamaa