As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: carry each administered combination component with its
dose, unit, administration interval, and adjustment reason, adding
EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, and
EXADJ.
Input: collected exposure records with one row per administered component, carrying treatment, dose, unit, administration interval, and adjustment reason.
Variables:
EXSEQis the order of the administration within the subject, numbered by start date and then, on the same date, by treatment name.EXDOSUis the dose unit as collected; milligrams (mg), milligrams per square metre (mg/m2), and area under the curve (AUC) remain distinct.EXADJis the adjustment reason as collected (for exampleDOSE INTERRUPTED); blank when none was reported.
Note: a saline placebo component is administered, so it keeps its
EXDOSE of 0; a zero dose is not an uncollected dose.
Inputs
Under review1 source fileinput/ex_raw.csv
Edit9 rows| STUDYID | USUBJID | EXTRT | EXDOSE | EXDOSU | EXSTDTC | EXENDTC | EXADJ |
|---|---|---|---|---|---|---|---|
| PILOT7 | Pembrolizumab | 200 | mg | 2025-01-02 | 2025-01-02 | — | |
| PILOT7 | Pemetrexed | 500 | mg/m2 | 2025-01-02 | 2025-01-02 | — | |
| PILOT7 | Carboplatin | 5 | AUC | 2025-01-02 | 2025-01-02 | — | |
| PILOT7 | Pembrolizumab | 200 | mg | 2025-01-23 | 2025-01-23 | DOSE INTERRUPTED | |
| PILOT7 | Pemetrexed | 500 | mg/m2 | 2025-01-23 | 2025-01-23 | — | |
| PILOT7 | Carboplatin | 5 | AUC | 2025-01-23 | 2025-01-23 | — | |
| PILOT7 | Saline placebo | 0 | mg | 2025-01-02 | 2025-01-02 | — | |
| PILOT7 | Pemetrexed | 500 | mg/m2 | 2025-01-02 | 2025-01-02 | — | |
| PILOT7 | Cisplatin | 75 | mg/m2 | 2025-01-02 | 2025-01-02 | — |
Expected output
Under review9 expected rowsexpected/ex.csv
Edit9 rows| DOMAIN | STUDYID | USUBJID | EXSEQ | EXTRT | EXDOSE | EXDOSU | EXSTDTC | EXENDTC | EXADJ |
|---|---|---|---|---|---|---|---|---|---|
| EX | PILOT7 | 2 | Pembrolizumab | 200 | mg | 2025-01-02 | 2025-01-02 | — | |
| EX | PILOT7 | 3 | Pemetrexed | 500 | mg/m2 | 2025-01-02 | 2025-01-02 | — | |
| EX | PILOT7 | 1 | Carboplatin | 5 | AUC | 2025-01-02 | 2025-01-02 | — | |
| EX | PILOT7 | 5 | Pembrolizumab | 200 | mg | 2025-01-23 | 2025-01-23 | DOSE INTERRUPTED | |
| EX | PILOT7 | 6 | Pemetrexed | 500 | mg/m2 | 2025-01-23 | 2025-01-23 | — | |
| EX | PILOT7 | 4 | Carboplatin | 5 | AUC | 2025-01-23 | 2025-01-23 | — | |
| EX | PILOT7 | 3 | Saline placebo | 0 | mg | 2025-01-02 | 2025-01-02 | — | |
| EX | PILOT7 | 2 | Pemetrexed | 500 | mg/m2 | 2025-01-02 | 2025-01-02 | — | |
| EX | PILOT7 | 1 | Cisplatin | 75 | mg/m2 | 2025-01-02 | 2025-01-02 | — |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "EX". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from EX_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from EX_RAW.USUBJID. | SDTM | 3 | ||||
| EXSEQ | Sequence Number | Num | Derived | Row number within each group of STUDYID, USUBJID, ordered by EX_RAW.EXSTDTC then EX_RAW.EXTRT. | SDTM | 4 | ||||
| EXTRT | Name of Treatment | Char | Collected | Copy value from EX_RAW.EXTRT. | SDTM | 5 | ||||
| EXDOSE | Dose | Num | Collected | Copy value from EX_RAW.EXDOSE. | SDTM | 6 | ||||
| EXDOSU | Dose Units | Char | Collected | Copy value from EX_RAW.EXDOSU. | SDTM | 7 | ||||
| EXSTDTC | Start Date/Time of Treatment | Char | ISO 8601 | Collected | Copy value from EX_RAW.EXSTDTC. | SDTM | 8 | |||
| EXENDTC | End Date/Time of Treatment | Char | ISO 8601 | Collected | Copy value from EX_RAW.EXENDTC. | SDTM | 9 | |||
| EXADJ | Reason for Dose Adjustment | Char | Collected | Copy value from EX_RAW.EXADJ. | SDTM | 10 |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-ex-combination/full.md
Edit18 linesFollowing CDISC SDTM standards, use the provided EX_RAW dataset to create an EX dataset with one record per administered combination component.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, EXADJ
EXSEQ numbers the administrations within the subject by start date and then, on the same date, by treatment name. EXDOSU is the dose unit as collected: milligrams (mg), milligrams per square metre (mg/m2), and area under the curve (AUC) remain distinct. EXADJ is the adjustment reason as collected, with no value when none was reported. A saline placebo component is administered, so it keeps its EXDOSE of 0; a zero dose is not an uncollected dose.
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
prompts/sdtm-ex-combination/conventions.md
Edit10 linesFollowing CDISC SDTM standards, use the provided EX_RAW dataset to create an EX dataset with one record per administered combination component.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, EXADJ
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
prompts/sdtm-ex-combination/brief.md
Edit10 linesFollowing CDISC SDTM standards, use the provided EX_RAW dataset to create an EX dataset with one record per administered combination component.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, EXADJ
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
run.R
Edit4 lineslibrary(yamaa)ex <- yamaa_domain("spec.yaml")$outputexrun.py
Edit4 linesimport yamaaex = yamaa.yamaa_domain("spec.yaml").outputexsolutions/sdtm-ex-combination/result.R
Edit32 lines# Reference solution for the yamaa benchmark sdtm-ex-combination (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)ex_raw <- read_csv( "/app/input/ex_raw.csv", col_types = cols(.default = col_character()))ex <- ex_raw |> mutate( DOMAIN = "EX", EXDOSE = suppressWarnings(as.numeric(EXDOSE)), EXSTDTC_DATE = suppressWarnings(as.Date(EXSTDTC)), EXADJ = na_if(EXADJ, "") ) |> arrange(STUDYID, USUBJID, EXSTDTC_DATE, EXTRT) |> group_by(STUDYID, USUBJID) |> mutate(EXSEQ = row_number()) |> ungroup() |> select( DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, EXADJ )dir.create("/app/output", showWarnings = FALSE)write_csv(ex, "/app/output/ex.csv", na = "")solutions/sdtm-ex-combination/result.py
Edit40 lines# Reference solution for the yamaa benchmark sdtm-ex-combination (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plex_raw = pl.read_csv("/app/input/ex_raw.csv", infer_schema=False)ex = ( ex_raw.with_columns( DOMAIN=pl.lit("EX"), EXDOSE=pl.col("EXDOSE").cast(pl.Float64, strict=False), EXSTDTC_DT=pl.col("EXSTDTC").str.to_date(strict=False), EXADJ=pl.col("EXADJ").map_elements( lambda v: None if v == "" or v is None else v, return_dtype=pl.String ), ) .sort(["STUDYID", "USUBJID", "EXSTDTC_DT", "EXTRT"]) .with_columns(EXSEQ=pl.col("USUBJID").cum_count().over(["STUDYID", "USUBJID"])) .with_columns(EXSEQ=pl.col("EXSEQ").cast(pl.Int64)) .select( "DOMAIN", "STUDYID", "USUBJID", "EXSEQ", "EXTRT", "EXDOSE", "EXDOSU", "EXSTDTC", "EXENDTC", "EXADJ", ))Path("/app/output").mkdir(exist_ok=True)ex.write_csv("/app/output/ex.csv")solutions/sdtm-ex-combination/result.sas
Edit21 lines/* opensas reference solution for the yamaa benchmark sdtm-ex-combination. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ex_raw; length STUDYID USUBJID EXTRT EXDOSU EXSTDTC EXENDTC EXADJ $1024; infile "/app/input/ex_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. EXTRT : $char1024. EXDOSE EXDOSU : $char1024. EXSTDTC : $char1024. EXENDTC : $char1024. EXADJ : $char1024.;run;data prepared; set ex_raw; length DOMAIN $2; DOMAIN='EX'; run;proc sort data=prepared; by STUDYID USUBJID EXSTDTC EXTRT; run;data result; set prepared; by STUDYID USUBJID; retain EXSEQ; if first.USUBJID then EXSEQ=0; EXSEQ+1;run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXSTDTC, EXENDTC, EXADJ from result;quit;proc export data=final outfile="/app/output/ex.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions