As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: build one Exposure (EX) record per constant-dose interval from
collected administration records, carrying EXSEQ, EXTRT, EXDOSE,
EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, and EXADJ.
Input: collected administration rows, one per dosing day, carrying treatment, dose, dose unit, dosing frequency, administration date, and the adjustment reason recorded when the dose changed.
Variables:
EXSEQnumbers the subject's intervals in the order they started, by start date then treatment. WithSTUDYIDandUSUBJIDit identifies the record.EXDOSEis the collected dose that defines the interval: the subject's administrations of one treatment at one dose, unit, and frequency form one interval.EXSTDTCis the first administration date at the interval's dose level.EXENDTCis the last administration date at the interval's dose level.EXADJis the adjustment reason recorded on any administration in the interval, for exampleDOSE REDUCEDorDOSE INTERRUPTED(the alphabetically first when they differ); blank when none was recorded.
Note: a dose interruption is its own interval with a zero dose and the reason recorded. A dose level that recurs later, for example one resumed after an interruption, is not split: its one record spans from the level's first to its last administration.
Provenance: one record per constant dosing interval follows the CDISC SDTMIG exposure assumption, with the adjustment reason on the record where the new dose took effect. All subjects, dates, doses, and reasons are invented fixtures.
Inputs
Under review1 source fileinput/ec_raw.csv
Edit13 rows| STUDYID | USUBJID | ECTRT | ECDOSE | ECDOSU | ECDOSFRQ | ECSTDTC | ECADJ |
|---|---|---|---|---|---|---|---|
| DOSEINT | Study Drug | 50 | mg | QD | 2025-01-02 | — | |
| DOSEINT | Study Drug | 50 | mg | QD | 2025-01-03 | — | |
| DOSEINT | Study Drug | 50 | mg | QD | 2025-01-04 | — | |
| DOSEINT | Study Drug | 100 | mg | QD | 2025-02-01 | — | |
| DOSEINT | Study Drug | 100 | mg | QD | 2025-02-02 | — | |
| DOSEINT | Study Drug | 75 | mg | QD | 2025-02-03 | DOSE REDUCED | |
| DOSEINT | Study Drug | 75 | mg | QD | 2025-02-04 | — | |
| DOSEINT | Study Drug | 200 | mg | QD | 2025-03-01 | — | |
| DOSEINT | Study Drug | 200 | mg | QD | 2025-03-02 | — | |
| DOSEINT | Study Drug | 0 | mg | QD | 2025-03-03 | DOSE INTERRUPTED | |
| DOSEINT | Study Drug | 0 | mg | QD | 2025-03-04 | — | |
| DOSEINT | Study Drug | 150 | mg | QD | 2025-03-05 | DOSE REDUCED | |
| DOSEINT | Study Drug | 150 | mg | QD | 2025-03-06 | — |
Expected output
Under review6 expected rowsexpected/ex.csv
Edit6 rows| DOMAIN | STUDYID | USUBJID | EXSEQ | EXTRT | EXDOSE | EXDOSU | EXDOSFRQ | EXSTDTC | EXENDTC | EXADJ |
|---|---|---|---|---|---|---|---|---|---|---|
| EX | DOSEINT | 1 | Study Drug | 50 | mg | QD | 2025-01-02 | 2025-01-04 | — | |
| EX | DOSEINT | 1 | Study Drug | 100 | mg | QD | 2025-02-01 | 2025-02-02 | — | |
| EX | DOSEINT | 2 | Study Drug | 75 | mg | QD | 2025-02-03 | 2025-02-04 | DOSE REDUCED | |
| EX | DOSEINT | 1 | Study Drug | 200 | mg | QD | 2025-03-01 | 2025-03-02 | — | |
| EX | DOSEINT | 2 | Study Drug | 0 | mg | QD | 2025-03-03 | 2025-03-04 | DOSE INTERRUPTED | |
| EX | DOSEINT | 3 | Study Drug | 150 | mg | QD | 2025-03-05 | 2025-03-06 | DOSE REDUCED |
Human review and assessment
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "EX". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from EC_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from EC_RAW.USUBJID. | SDTM | 3 | ||||
| EXTRT | Name of Treatment | Char | Collected | Copy value from EC_RAW.ECTRT. | SDTM | 4 | ||||
| EXDOSE | Dose | Num | Collected | Copy value from EC_RAW.ECDOSE. | SDTM | 5 | ||||
| EXDOSU | Dose Units | Char | mg | Collected | Copy value from EC_RAW.ECDOSU. | SDTM | 6 | |||
| EXDOSFRQ | Dosing Frequency per Interval | Char | QD | Collected | Copy value from EC_RAW.ECDOSFRQ. | SDTM | 7 | |||
| EXSTDTC | Start Date/Time of Treatment | Char | ISO 8601 | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 8 | |||
| EXENDTC | End Date/Time of Treatment | Char | ISO 8601 | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 9 | |||
| EXADJ | Reason for Dose Adjustment | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 10 | ||||
| EXSEQ | Sequence Number | Num | Derived | Row number within each group of STUDYID, USUBJID, ordered by EXSTDTC then EXTRT. | SDTM | 11 |
| Template | Include when | EXSTDTC | EXENDTC | EXADJ |
|---|---|---|---|---|
| dose_intervals | Aggregate MIN(EC_RAW.ECSTDTC) over records matching on the shared output keys | Aggregate MAX(EC_RAW.ECSTDTC) over records matching on the shared output keys | Aggregate MIN(EC_RAW.ECADJ) over records matching on the shared output keys |
| Variable | Permitted values |
|---|---|
| EXDOSU | mg |
| EXDOSFRQ | QD |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-ex-intervals/full.md
Edit21 linesFollowing CDISC SDTM standards, use the provided EC_RAW dataset to create an EX dataset with one record per constant-dose interval.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, EXADJ
DOMAIN is "EX". The subject's administrations of one treatment at one dose, dose unit, and dosing frequency form one interval, and each interval gives one record. EXSTDTC is the first administration date at the interval's dose level. EXENDTC is the last administration date at the interval's dose level. EXADJ is the adjustment reason recorded on any administration in the interval, for example "DOSE REDUCED" or "DOSE INTERRUPTED", and has no value when no administration in the interval records a reason. A dose interruption is its own interval, with a zero dose and the reason recorded. EXSEQ numbers the subject's intervals from 1 in the order they started, by start date then treatment.
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
prompts/sdtm-ex-intervals/conventions.md
Edit12 linesFollowing CDISC SDTM standards, use the provided EC_RAW dataset to create an EX dataset with one record per constant-dose interval.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, EXADJ
DOMAIN is "EX". EXADJ is an adjustment reason, for example "DOSE REDUCED" or "DOSE INTERRUPTED", or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
prompts/sdtm-ex-intervals/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided EC_RAW dataset to create an EX dataset with one record per constant-dose interval.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, EXADJ
Read the source datasets from /app/input and save the completed dataset as /app/output/ex.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/sdtm-ex-intervals/result.R
Edit50 lines# Reference solution for the yamaa benchmark sdtm-ex-intervals (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)ec_raw <- read_csv( "/app/input/ec_raw.csv", col_types = cols(.default = col_character()))ex <- ec_raw |> mutate( EXDOSE_NUM = suppressWarnings(as.numeric(ECDOSE)), ECSTDTC_DT = suppressWarnings(as.Date(ECSTDTC)), ECADJ_CLEAN = na_if(ECADJ, "") ) |> group_by(STUDYID, USUBJID, ECTRT, EXDOSE_NUM, ECDOSU, ECDOSFRQ) |> summarise( START_DT = min(ECSTDTC_DT, na.rm = TRUE), END_DT = max(ECSTDTC_DT, na.rm = TRUE), EXADJ = { vals <- sort(unique(ECADJ_CLEAN[!is.na(ECADJ_CLEAN)])) if (length(vals) == 0) NA_character_ else vals[1] }, .groups = "drop" ) |> mutate( DOMAIN = "EX", EXTRT = ECTRT, EXDOSE = EXDOSE_NUM, EXDOSU = ECDOSU, EXDOSFRQ = ECDOSFRQ, EXSTDTC = START_DT, EXENDTC = END_DT ) |> arrange(STUDYID, USUBJID, START_DT, EXTRT) |> group_by(STUDYID, USUBJID) |> mutate(EXSEQ = row_number()) |> ungroup() |> select( DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, EXADJ )dir.create("/app/output", showWarnings = FALSE)write_csv(ex, "/app/output/ex.csv", na = "")solutions/sdtm-ex-intervals/result.py
Edit106 lines# Reference solution for the yamaa benchmark sdtm-ex-intervals (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build the EX dataset (one record per constant-dose interval) from EC_RAW.Reads /app/input/ec_raw.csv and writes /app/output/ex.csv with columns:DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ,EXSTDTC, EXENDTC, EXADJ."""import csvimport osINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"def main(): with open(os.path.join(INPUT_DIR, "ec_raw.csv"), newline="") as f: rows = list(csv.DictReader(f)) # Administrations of one subject and treatment, in date order. rows.sort(key=lambda r: (r["USUBJID"], r["ECTRT"], r["ECSTDTC"])) # Consecutive administrations at one dose, dose unit, and dosing # frequency form one interval. A dose interruption (dose 0 with a # recorded reason) is its own interval like any other dose level. intervals = [] for r in rows: key = ( r["STUDYID"], r["USUBJID"], r["ECTRT"], r["ECDOSE"], r["ECDOSU"], r["ECDOSFRQ"], ) if intervals and intervals[-1]["key"] == key: iv = intervals[-1] iv["end"] = r["ECSTDTC"] if not iv["adj"] and r["ECADJ"]: iv["adj"] = r["ECADJ"] else: intervals.append( { "key": key, "start": r["ECSTDTC"], "end": r["ECSTDTC"], "adj": r["ECADJ"], } ) # EXSEQ numbers each subject's intervals from 1 in the order they # started, by start date then treatment. by_subject = {} for iv in intervals: by_subject.setdefault(iv["key"][1], []).append(iv) out_rows = [] for usubjid in sorted(by_subject): subject_intervals = sorted( by_subject[usubjid], key=lambda iv: (iv["start"], iv["key"][2]) ) for seq, iv in enumerate(subject_intervals, start=1): studyid, _, trt, dose, dosu, dosfrq = iv["key"] out_rows.append( { "DOMAIN": "EX", "STUDYID": studyid, "USUBJID": usubjid, "EXSEQ": seq, "EXTRT": trt, "EXDOSE": dose, "EXDOSU": dosu, "EXDOSFRQ": dosfrq, "EXSTDTC": iv["start"], "EXENDTC": iv["end"], "EXADJ": iv["adj"], } ) fieldnames = [ "DOMAIN", "STUDYID", "USUBJID", "EXSEQ", "EXTRT", "EXDOSE", "EXDOSU", "EXDOSFRQ", "EXSTDTC", "EXENDTC", "EXADJ", ] os.makedirs(OUTPUT_DIR, exist_ok=True) with open(os.path.join(OUTPUT_DIR, "ex.csv"), "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/sdtm-ex-intervals/result.sas
Edit24 lines/* opensas reference solution for the yamaa benchmark sdtm-ex-intervals. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ec_raw; length STUDYID USUBJID ECTRT ECDOSU ECDOSFRQ ECSTDTC ECADJ $1024; infile "/app/input/ec_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. ECTRT : $char1024. ECDOSE ECDOSU : $char1024. ECDOSFRQ : $char1024. ECSTDTC : $char1024. ECADJ : $char1024.;run;proc sql;create table intervals as select STUDYID,USUBJID,ECTRT as EXTRT,ECDOSE as EXDOSE,ECDOSU as EXDOSU,ECDOSFRQ as EXDOSFRQ,min(ECSTDTC) as EXSTDTC,max(ECSTDTC) as EXENDTC from ec_raw group by STUDYID,USUBJID,ECTRT,ECDOSE,ECDOSU,ECDOSFRQ;create table reasons as select STUDYID,USUBJID,ECTRT,ECDOSE,ECDOSU,ECDOSFRQ,min(ECADJ) as EXADJ from ec_raw where not missing(ECADJ) group by STUDYID,USUBJID,ECTRT,ECDOSE,ECDOSU,ECDOSFRQ;create table prepared as select 'EX' as DOMAIN,a.*,b.EXADJ from intervals a left join reasons b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.EXTRT=b.ECTRT and a.EXDOSE=b.ECDOSE and a.EXDOSU=b.ECDOSU and a.EXDOSFRQ=b.ECDOSFRQ; quit;proc sort data=prepared; by STUDYID USUBJID EXSTDTC EXTRT; run;data result; set prepared; by STUDYID USUBJID; retain EXSEQ; if first.USUBJID then EXSEQ=0; EXSEQ+1;run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, EXSEQ, EXTRT, EXDOSE, EXDOSU, EXDOSFRQ, EXSTDTC, EXENDTC, EXADJ from result;quit;proc export data=final outfile="/app/output/ex.csv" dbms=csv replace; run;
yamaa