Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: reviewedEditUnder review
What this benchmark means
README.md

Goal: summarize each subject-treatment, carrying EXTRT (treatment name) through and adding DOSECUM (cumulative dose), NDOSREC (exposure-record count), and NDOSVAL (recorded-dose count).

Input: a subject-treatment inventory, plus exposure records holding administered doses (EXDOSE) and exposure sequence numbers (EXSEQ).

Variables:

  • DOSECUM is the total administered dose across the treatment's exposure records; empty when no dose was ever recorded, while a recorded zero total is kept so a measured zero never reads as missing.
  • NDOSREC is the number of exposure records for the treatment; empty when the treatment has no exposure record at all.
  • NDOSVAL is the number of exposure records carrying a recorded dose; an explicitly recorded zero counts as a dose, and the count is zero (not empty) when records exist but every dose was left blank.

Note: a treatment whose doses were all left blank stays distinguishable from one that was never given: the first has exposure records with a zero NDOSVAL, while the second has nothing to count and leaves all three measures empty.

Inputs

Under review
2 source files

input/ex.csv

Edit
8 rows
input/ex.csv
STUDYIDUSUBJIDEXTRTEXSEQEXDOSE
YAMAA-01Carboplatin1400
YAMAA-01Carboplatin2350
YAMAA-01Paclitaxel1175
YAMAA-01Carboplatin1—
YAMAA-01Carboplatin2—
YAMAA-01Carboplatin10
YAMAA-01Carboplatin2—
YAMAA-01Carboplatin1—

input/subject_treatment.csv

Edit
6 rows
input/subject_treatment.csv
STUDYIDUSUBJIDEXTRT
YAMAA-01Carboplatin
YAMAA-01Paclitaxel
YAMAA-01Carboplatin
YAMAA-01Carboplatin
YAMAA-01Carboplatin
YAMAA-01Carboplatin

Expected output

Under review
6 expected rows

expected/adex.csv

Edit
6 rows
expected/adex.csv
STUDYIDUSUBJIDEXTRTDOSECUMNDOSRECNDOSVAL
YAMAA-01Carboplatin75022
YAMAA-01Paclitaxel17511
YAMAA-01Carboplatin—20
YAMAA-01Carboplatin021
YAMAA-01Carboplatin———
YAMAA-01Carboplatin—10
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADEXSTUDYIDStudy IdentifiercharacterCopy value from TRT.STUDYID.Predecessor
ADEXUSUBJIDUnique Subject IdentifiercharacterCopy value from TRT.USUBJID.Predecessor
ADEXEXTRTName of TreatmentcharacterCopy value from TRT.EXTRT.Predecessor
ADEXDOSECUMCumulative DosenumericAggregate SUM(EX.EXDOSE) over records matching on the shared output keys.Derived
ADEXNDOSRECNumber of Dose RecordsnumericAggregate COUNT(EX.EXSEQ) over records matching on the shared output keys.Derived
ADEXNDOSVALNumber of Nonmissing DosesnumericAggregate COUNT(EX.EXDOSE) over records matching on the shared output keys.Derived
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-adex-uncollected/full.md

Edit
18 lines

Following CDISC ADaM standards, use the provided TRT and EX datasets to create an ADEX dataset with one record per subject per treatment.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL

DOSECUM is the total administered dose across the treatment's exposure records; empty when no dose was ever recorded, while a recorded zero total is kept so a measured zero never reads as missing. NDOSREC is the number of exposure records for the treatment; empty when the treatment has no exposure record at all. NDOSVAL is the number of exposure records carrying a recorded dose; an explicitly recorded zero counts as a dose, and the count is zero (not empty) when records exist but every dose was left blank. NDOSVAL has no value when the treatment has no exposure record at all.

Read the source datasets from /app/input and save the completed dataset as /app/output/adex.csv.

prompts/adam-adex-uncollected/conventions.md

Edit
8 lines

Following CDISC ADaM standards, use the provided TRT and EX datasets to create an ADEX dataset with one record per subject per treatment.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL

Read the source datasets from /app/input and save the completed dataset as /app/output/adex.csv.

prompts/adam-adex-uncollected/brief.md

Edit
8 lines

Following CDISC ADaM standards, use the provided TRT and EX datasets to create an ADEX dataset with one record per subject per treatment.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL

Read the source datasets from /app/input and save the completed dataset as /app/output/adex.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)adex <- yamaa_domain("spec.yaml")$outputadex

run.py

Edit
4 lines
import yamaaadex = yamaa.yamaa_domain("spec.yaml").outputadex

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-adex-uncollected/result.R

Edit
36 lines
# Reference solution for the yamaa benchmark adam-adex-uncollected (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)trt <- read_csv(  "/app/input/subject_treatment.csv",  col_types = cols(.default = col_character()))ex <- read_csv(  "/app/input/ex.csv",  col_types = cols(EXDOSE = col_double(), .default = col_character()))# Per treatment: total dose, record count, and count of recorded doses.agg <- ex |>  group_by(STUDYID, USUBJID, EXTRT) |>  summarise(    DOSE_SUM = sum(EXDOSE, na.rm = TRUE),    NDOSREC = n(),    NDOSVAL = sum(!is.na(EXDOSE)),    .groups = "drop"  ) |>  mutate(DOSECUM = if_else(NDOSVAL > 0, DOSE_SUM, NA_real_)) |>  select(STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL)adex <- trt |>  left_join(agg, by = c("STUDYID", "USUBJID", "EXTRT")) |>  select(STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL)dir.create("/app/output", showWarnings = FALSE)write_csv(adex, "/app/output/adex.csv", na = "")

solutions/adam-adex-uncollected/result.py

Edit
37 lines
# Reference solution for the yamaa benchmark adam-adex-uncollected (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pltrt = pl.read_csv("/app/input/subject_treatment.csv", infer_schema=False)ex = pl.read_csv("/app/input/ex.csv", infer_schema=False).with_columns(    pl.col("EXDOSE").cast(pl.Float64, strict=False))# Per treatment: total dose, record count, and count of recorded doses.agg = (    ex.group_by("STUDYID", "USUBJID", "EXTRT")    .agg(        pl.col("EXDOSE").sum().alias("DOSE_SUM"),        pl.len().cast(pl.Int64).alias("NDOSREC"),        pl.col("EXDOSE").count().cast(pl.Int64).alias("NDOSVAL"),    )    .with_columns(        DOSECUM=pl.when(pl.col("NDOSVAL") > 0)        .then(pl.col("DOSE_SUM"))        .otherwise(None)    )    .select("STUDYID", "USUBJID", "EXTRT", "DOSECUM", "NDOSREC", "NDOSVAL"))adex = trt.join(    agg, on=["STUDYID", "USUBJID", "EXTRT"], how="left", maintain_order="left").select("STUDYID", "USUBJID", "EXTRT", "DOSECUM", "NDOSREC", "NDOSVAL")Path("/app/output").mkdir(exist_ok=True)adex.write_csv("/app/output/adex.csv")

solutions/adam-adex-uncollected/result.sas

Edit
25 lines
/* opensas reference solution for the yamaa benchmark adam-adex-uncollected.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ex;  length STUDYID USUBJID EXTRT $1024;  infile "/app/input/ex.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. EXTRT : $char1024. EXSEQ EXDOSE;run;data subject_treatment;  length STUDYID USUBJID EXTRT $1024;  infile "/app/input/subject_treatment.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. EXTRT : $char1024.;run;proc sql;  create table totals as select STUDYID,USUBJID,EXTRT,sum(EXDOSE) as DOSECUM,count(*) as NDOSREC,count(EXDOSE) as NDOSVAL from ex group by STUDYID,USUBJID,EXTRT;  create table result as select a.*,b.DOSECUM,b.NDOSREC,b.NDOSVAL from subject_treatment as a left join totals as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.EXTRT=b.EXTRT;quit;proc sql;  create table final as select STUDYID, USUBJID, EXTRT, DOSECUM, NDOSREC, NDOSVAL from result;quit;proc export data=final outfile="/app/output/adex.csv" dbms=csv replace; run;