As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: derive TR01SDT, TR01EDT, TR02SDT, TR02EDT,
TRT01A, TRT02A, and WASHDUR to capture each treatment
period and the washout between them, with one record per
subject.
Input: demographics listing each subject, plus exposure
records carrying the epoch (EPOCH), the treatment
given (EXTRT), start and end dates (EXSTDTC, EXENDTC), and
a sequence number (EXSEQ).
Variables:
TR01SDTis the earliestEXSTDTCamong the subject's exposure records in theTREATMENT 1epoch and a recorded start date; empty when the subject has no such record.TR01EDTis the latestEXENDTCamong the subject's exposure records in theTREATMENT 1epoch and a recorded end date; empty when the subject has no such record.TR02SDTis the earliestEXSTDTCamong the subject's exposure records in theTREATMENT 2epoch and a recorded start date; empty when the subject has no such record.TR02EDTis the latestEXENDTCamong the subject's exposure records in theTREATMENT 2epoch and a recorded end date; empty when the subject has no such record.TRT01Ais theEXTRTfrom the subject's earliest exposure record in theTREATMENT 1epoch, earliest byEXSTDTCwith the lowerEXSEQbreaking ties on the same day and a record with no start date sorting last; eitherVITAMIN D3orPLACEBO, and empty when the subject has no period-one exposure.TRT02Ais theEXTRTfrom the subject's earliest exposure record in theTREATMENT 2epoch, chosen the same way; eitherVITAMIN D3orPLACEBO, and empty when the subject has no period-two exposure.WASHDURis the number of days strictly betweenTR01EDTandTR02SDT, counting neither endpoint, so periods that touch give zero; empty when either date is missing.
Note: every value comes only from exposure records in its
own period, so period-one dates and treatment never mix with
period-two records. A subject with no period-two exposure
leaves TR02SDT, TR02EDT, TRT02A, and WASHDUR empty
together.
Inputs
Under review2 source filesinput/dm.csv
Edit6 rows| STUDYID | USUBJID |
|---|---|
| YAMAA-01 | |
| YAMAA-01 | |
| YAMAA-01 | |
| YAMAA-01 | |
| YAMAA-01 | |
| YAMAA-01 |
input/ex.csv
Edit17 rows| STUDYID | USUBJID | EXSEQ | EPOCH | EXTRT | EXSTDTC | EXENDTC |
|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | TREATMENT 1 | VITAMIN D3 | 2025-01-06 | 2025-01-16 | |
| YAMAA-01 | 2 | TREATMENT 1 | VITAMIN D3 | 2025-01-17 | 2025-01-26 | |
| YAMAA-01 | 3 | TREATMENT 2 | PLACEBO | 2025-02-10 | 2025-03-02 | |
| YAMAA-01 | 3 | TREATMENT 2 | PLACEBO | 2025-02-10 | 2025-03-02 | |
| YAMAA-01 | 1 | TREATMENT 1 | PLACEBO | 2025-01-08 | 2025-01-28 | |
| YAMAA-01 | 2 | TREATMENT 2 | VITAMIN D3 | 2025-02-12 | 2025-02-22 | |
| YAMAA-01 | 3 | TREATMENT 2 | VITAMIN D3 | 2025-02-23 | 2025-03-04 | |
| YAMAA-01 | 1 | TREATMENT 1 | VITAMIN D3 | 2025-01-10 | 2025-01-20 | |
| YAMAA-01 | 2 | TREATMENT 1 | VITAMIN D3 | 2025-03-10 | 2025-03-15 | |
| YAMAA-01 | 5 | TREATMENT 1 | PLACEBO | 2025-03-10 | 2025-03-20 | |
| YAMAA-01 | 1 | TREATMENT 2 | PLACEBO | 2025-04-01 | 2025-04-15 | |
| YAMAA-01 | 1 | TREATMENT 1 | PLACEBO | 2025-04-01 | 2025-04-10 | |
| YAMAA-01 | 2 | TREATMENT 2 | VITAMIN D3 | 2025-04-11 | 2025-04-25 | |
| YAMAA-01 | 1 | TREATMENT 1 | VITAMIN D3 | 2025-05-01 | 2025-05-10 | |
| YAMAA-01 | 2 | TREATMENT 1 | PLACEBO | — | 2025-05-12 | |
| YAMAA-01 | 3 | TREATMENT 2 | PLACEBO | 2025-06-01 | — | |
| YAMAA-01 | 4 | TREATMENT 2 | PLACEBO | 2025-06-02 | 2025-06-20 |
Expected output
Under review6 expected rowsexpected/adsl.csv
Edit6 rows| STUDYID | USUBJID | TR01SDT | TR01EDT | TR02SDT | TR02EDT | TRT01A | TRT02A | WASHDUR |
|---|---|---|---|---|---|---|---|---|
| YAMAA-01 | 2025-01-06 | 2025-01-26 | 2025-02-10 | 2025-03-02 | VITAMIN D3 | PLACEBO | 14 | |
| YAMAA-01 | 2025-01-08 | 2025-01-28 | 2025-02-12 | 2025-03-04 | PLACEBO | VITAMIN D3 | 14 | |
| YAMAA-01 | 2025-01-10 | 2025-01-20 | — | — | VITAMIN D3 | — | — | |
| YAMAA-01 | 2025-03-10 | 2025-03-20 | 2025-04-01 | 2025-04-15 | VITAMIN D3 | PLACEBO | 11 | |
| YAMAA-01 | 2025-04-01 | 2025-04-10 | 2025-04-11 | 2025-04-25 | PLACEBO | VITAMIN D3 | 0 | |
| YAMAA-01 | 2025-05-01 | 2025-05-12 | 2025-06-01 | 2025-06-20 | VITAMIN D3 | PLACEBO | 19 |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADSL | STUDYID | Study Identifier | character | Copy value from DM.STUDYID. | Predecessor | ||
| ADSL | USUBJID | Unique Subject Identifier | character | Copy value from DM.USUBJID. | Predecessor | ||
| ADSL | TR01SDT | Date of First Exposure in Period 01 | numeric | ISO 8601 | Aggregate MIN(EX.EXSTDTC) over records matching on the shared output keys, where EX.EPOCH = 'TREATMENT 1'. | Derived | |
| ADSL | TR01EDT | Date of Last Exposure in Period 01 | numeric | ISO 8601 | Aggregate MAX(EX.EXENDTC) over records matching on the shared output keys, where EX.EPOCH = 'TREATMENT 1'. | Derived | |
| ADSL | TR02SDT | Date of First Exposure in Period 02 | numeric | ISO 8601 | Aggregate MIN(EX.EXSTDTC) over records matching on the shared output keys, where EX.EPOCH = 'TREATMENT 2'. | Derived | |
| ADSL | TR02EDT | Date of Last Exposure in Period 02 | numeric | ISO 8601 | Aggregate MAX(EX.EXENDTC) over records matching on the shared output keys, where EX.EPOCH = 'TREATMENT 2'. | Derived | |
| ADSL | TRT01A | Actual Treatment for Period 01 | character | VITAMIN D3, PLACEBO | Copy value from EX.EXTRT where EX.EPOCH = 'TREATMENT 1', keeping the first ordered by EX.EXSTDTC, nulls last then EX.EXSEQ. | Predecessor | |
| ADSL | TRT02A | Actual Treatment for Period 02 | character | VITAMIN D3, PLACEBO | Copy value from EX.EXTRT where EX.EPOCH = 'TREATMENT 2', keeping the first ordered by EX.EXSTDTC, nulls last then EX.EXSEQ. | Predecessor | |
| ADSL | WASHDUR | Washout Duration (Days) | numeric | Whole days from TR01EDT to TR02SDT, counting neither endpoint. | Derived |
| Variable | Permitted values |
|---|---|
| TRT01A | VITAMIN D3, PLACEBO |
| TRT02A | VITAMIN D3, PLACEBO |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adsl-crossover-periods/full.md
Edit36 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, TR01SDT, TR01EDT, TR02SDT, TR02EDT, TRT01A, TRT02A, WASHDUR
- TR01SDT is the earliest EXSTDTC among the subject's exposure records in the "TREATMENT 1" epoch with a recorded start date, and has no value when the subject has no such record.
- TR01EDT is the latest EXENDTC among the subject's exposure records in the "TREATMENT 1" epoch with a recorded end date, and has no value when the subject has no such record.
- TR02SDT is the earliest EXSTDTC among the subject's exposure records in the "TREATMENT 2" epoch with a recorded start date, and has no value when the subject has no such record.
- TR02EDT is the latest EXENDTC among the subject's exposure records in the "TREATMENT 2" epoch with a recorded end date, and has no value when the subject has no such record.
- TRT01A is the EXTRT of the subject's earliest exposure record in the "TREATMENT 1" epoch, earliest by EXSTDTC, with the lower EXSEQ breaking ties between records that start on the same day and a record with no start date counting as later than any record with one; TRT01A is either "VITAMIN D3" or "PLACEBO", and has no value when the subject has no exposure record in the "TREATMENT 1" epoch.
- TRT02A is the EXTRT of the subject's earliest exposure record in the "TREATMENT 2" epoch, chosen the same way as TRT01A; TRT02A is either "VITAMIN D3" or "PLACEBO", and has no value when the subject has no exposure record in the "TREATMENT 2" epoch.
- WASHDUR is the number of days strictly between TR01EDT and TR02SDT, counting neither endpoint, so periods that touch give zero, and has no value when either date has no value.
- A subject with no exposure record in the "TREATMENT 2" epoch has no value for TR02SDT, TR02EDT, TRT02A, and WASHDUR.
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
prompts/adam-adsl-crossover-periods/conventions.md
Edit13 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, TR01SDT, TR01EDT, TR02SDT, TR02EDT, TRT01A, TRT02A, WASHDUR
- TRT01A is either "VITAMIN D3" or "PLACEBO", or has no value.
- TRT02A is either "VITAMIN D3" or "PLACEBO", or has no value.
- WASHDUR is the number of days strictly between TR01EDT and TR02SDT, counting neither endpoint.
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
prompts/adam-adsl-crossover-periods/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, TR01SDT, TR01EDT, TR02SDT, TR02EDT, TRT01A, TRT02A, WASHDUR
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-adsl-crossover-periods/result.R
Edit66 lines# Reference solution for the yamaa benchmark adam-adsl-crossover-periods (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)dm <- read_csv( "/app/input/dm.csv", col_types = cols(.default = col_character()))ex <- read_csv( "/app/input/ex.csv", col_types = cols( EXSEQ = col_integer(), EXSTDTC = col_date(), EXENDTC = col_date(), .default = col_character() ))ex1 <- ex |> filter(EPOCH %in% "TREATMENT 1")ex2 <- ex |> filter(EPOCH %in% "TREATMENT 2")agg1 <- ex1 |> group_by(STUDYID, USUBJID) |> summarise( TR01SDT = if (all(is.na(EXSTDTC))) as.Date(NA) else min(EXSTDTC, na.rm = TRUE), TR01EDT = if (all(is.na(EXENDTC))) as.Date(NA) else max(EXENDTC, na.rm = TRUE), .groups = "drop" )agg2 <- ex2 |> group_by(STUDYID, USUBJID) |> summarise( TR02SDT = if (all(is.na(EXSTDTC))) as.Date(NA) else min(EXSTDTC, na.rm = TRUE), TR02EDT = if (all(is.na(EXENDTC))) as.Date(NA) else max(EXENDTC, na.rm = TRUE), .groups = "drop" )# The treatment on the earliest record in each period: earliest start date# with the lower sequence number breaking ties, records without a start# date sorting last.first1 <- ex1 |> arrange(STUDYID, USUBJID, is.na(EXSTDTC), EXSTDTC, EXSEQ) |> distinct(STUDYID, USUBJID, .keep_all = TRUE) |> transmute(STUDYID, USUBJID, TRT01A = EXTRT)first2 <- ex2 |> arrange(STUDYID, USUBJID, is.na(EXSTDTC), EXSTDTC, EXSEQ) |> distinct(STUDYID, USUBJID, .keep_all = TRUE) |> transmute(STUDYID, USUBJID, TRT02A = EXTRT)adsl <- dm |> left_join(agg1, by = c("STUDYID", "USUBJID")) |> left_join(agg2, by = c("STUDYID", "USUBJID")) |> left_join(first1, by = c("STUDYID", "USUBJID")) |> left_join(first2, by = c("STUDYID", "USUBJID")) |> mutate(WASHDUR = as.integer(TR02SDT - TR01EDT) - 1L) |> select( STUDYID, USUBJID, TR01SDT, TR01EDT, TR02SDT, TR02EDT, TRT01A, TRT02A, WASHDUR )dir.create("/app/output", showWarnings = FALSE)write_csv(adsl, "/app/output/adsl.csv", na = "")solutions/adam-adsl-crossover-periods/result.py
Edit99 lines# Reference solution for the yamaa benchmark adam-adsl-crossover-periods (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build ADSL (one record per subject) from DM and EX.Derives the two crossover treatment periods per subject: earliest startand latest end per epoch, the treatment of the earliest record in eachepoch, and the washout duration strictly between the periods."""import csvfrom datetime import dateINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"COLUMNS = [ "STUDYID", "USUBJID", "TR01SDT", "TR01EDT", "TR02SDT", "TR02EDT", "TRT01A", "TRT02A", "WASHDUR",]def parse_day(text): text = (text or "").strip() if not text: return None return date.fromisoformat(text)def read_rows(path): with open(path, newline="") as handle: return list(csv.DictReader(handle))def period(rows): """Return (start, end, treatment) for one subject's records in one epoch.""" starts = [d for d in (parse_day(r["EXSTDTC"]) for r in rows) if d] ends = [d for d in (parse_day(r["EXENDTC"]) for r in rows) if d] start = min(starts) if starts else None end = max(ends) if ends else None treatment = "" if rows: def earliest_key(row): day = parse_day(row["EXSTDTC"]) # A record with no start date counts as later than any with one. return (day is None, day or date.max, float(row["EXSEQ"])) treatment = min(rows, key=earliest_key)["EXTRT"] return start, end, treatmentdef main(): dm = read_rows(f"{INPUT_DIR}/dm.csv") ex = read_rows(f"{INPUT_DIR}/ex.csv") by_subject = {} for row in ex: by_subject.setdefault((row["STUDYID"], row["USUBJID"]), []).append(row) out = [] for subject in dm: key = (subject["STUDYID"], subject["USUBJID"]) records = by_subject.get(key, []) t1 = [r for r in records if r["EPOCH"] == "TREATMENT 1"] t2 = [r for r in records if r["EPOCH"] == "TREATMENT 2"] tr01sdt, tr01edt, trt01a = period(t1) tr02sdt, tr02edt, trt02a = period(t2) washdur = "" if tr01edt is not None and tr02sdt is not None: # Days strictly between the two dates, counting neither endpoint. washdur = str((tr02sdt - tr01edt).days - 1) def fmt(day): return day.isoformat() if day is not None else "" out.append({ "STUDYID": subject["STUDYID"], "USUBJID": subject["USUBJID"], "TR01SDT": fmt(tr01sdt), "TR01EDT": fmt(tr01edt), "TR02SDT": fmt(tr02sdt), "TR02EDT": fmt(tr02edt), "TRT01A": trt01a, "TRT02A": trt02a, "WASHDUR": washdur, }) with open(f"{OUTPUT_DIR}/adsl.csv", "w", newline="") as handle: writer = csv.DictWriter(handle, fieldnames=COLUMNS) writer.writeheader() writer.writerows(out)if __name__ == "__main__": main()solutions/adam-adsl-crossover-periods/result.sas
Edit33 lines/* opensas reference solution for the yamaa benchmark adam-adsl-crossover-periods. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dm; length STUDYID USUBJID $1024; infile "/app/input/dm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024.;run;data ex; length STUDYID USUBJID EPOCH EXTRT EXSTDTC EXENDTC $1024; infile "/app/input/ex.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. EXSEQ EPOCH : $char1024. EXTRT : $char1024. EXSTDTC : $char1024. EXENDTC : $char1024.;run;data exposure; set ex; NODATE=missing(EXSTDTC); run;proc sort data=exposure; by STUDYID USUBJID EPOCH NODATE EXSTDTC EXSEQ; run;data firstex; set exposure; by STUDYID USUBJID EPOCH; if first.EPOCH; run;proc sql;create table starts1 as select STUDYID,USUBJID,min(EXSTDTC) as TR01SDT from ex where EPOCH='TREATMENT 1' and not missing(EXSTDTC) group by STUDYID,USUBJID;create table ends1 as select STUDYID,USUBJID,max(EXENDTC) as TR01EDT from ex where EPOCH='TREATMENT 1' group by STUDYID,USUBJID;create table starts2 as select STUDYID,USUBJID,min(EXSTDTC) as TR02SDT from ex where EPOCH='TREATMENT 2' and not missing(EXSTDTC) group by STUDYID,USUBJID;create table ends2 as select STUDYID,USUBJID,max(EXENDTC) as TR02EDT from ex where EPOCH='TREATMENT 2' group by STUDYID,USUBJID;create table joined as select a.*,b.TR01SDT,c.TR01EDT,d.TR02SDT,e.TR02EDT,f.EXTRT as TRT01A,g.EXTRT as TRT02A from dm aleft join starts1 b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJIDleft join ends1 c on a.STUDYID=c.STUDYID and a.USUBJID=c.USUBJIDleft join starts2 d on a.STUDYID=d.STUDYID and a.USUBJID=d.USUBJIDleft join ends2 e on a.STUDYID=e.STUDYID and a.USUBJID=e.USUBJIDleft join firstex f on a.STUDYID=f.STUDYID and a.USUBJID=f.USUBJID and f.EPOCH='TREATMENT 1'left join firstex g on a.STUDYID=g.STUDYID and a.USUBJID=g.USUBJID and g.EPOCH='TREATMENT 2'; quit;data result; set joined; WASHDUR=.; if not missing(TR01EDT) and not missing(TR02SDT) then WASHDUR=max(0,input(TR02SDT,yymmdd10.)-input(TR01EDT,yymmdd10.)-1); run;proc export data=result(keep=STUDYID USUBJID TR01SDT TR01EDT TR02SDT TR02EDT TRT01A TRT02A WASHDUR) outfile="/app/output/adsl.csv" dbms=csv replace; run;
yamaa