yamaa Review ask
As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
EditUnder reviewWhat this benchmark meansREADME.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: build one EC (Exposure as Collected) record per dosing-log
day, carrying ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE,
ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, and
ECADJ.
Input: a daily dosing log, one row per subject per day, carrying the date, tablets taken, whether the dose was taken, the reason a dose was missed, and any dose adjustment.
Variables:
ECSEQnumbers the subject's dosing days in date order. WithSTUDYIDandUSUBJIDit identifies the record.ECMOODisPERFORMEDon every record: these are administrations as collected.ECOCCURisYwhen the subject took the day's dose,Nwhen the dose was not taken.ECREASNDis the reason the dose was not taken, for exampleSUBJECT FORGOT DOSE; blank when the dose was taken.ECDOSEis the tablets taken that day; blank on a day the dose was not taken.ECSTDTCandECENDTCare the dosing day, since one record covers one day.ECADJis the adjustment reason the log records for that day, for exampleDOSE REDUCED; blank when the day records none.
Note: a dose reduced after an adverse event carries the adjustment
reason (DOSE REDUCED), not the event. A missed day keeps the
treatment's unit, form, frequency, and route with no dose and the
missed reason.
Inputs
Under review1 source fileinput/dosing_log.csv
Edit10 rows| STUDYID | USUBJID | LOGDATE | TABLETS | DOSE_TAKEN | MISSREAS | DOSE_ADJ |
|---|---|---|---|---|---|---|
| YAMAA-01 | 2025-01-02 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-03 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-04 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-05 | — | N | SUBJECT FORGOT DOSE | — | |
| YAMAA-01 | 2025-01-06 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-02 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-03 | 2 | Y | — | — | |
| YAMAA-01 | 2025-01-04 | 1 | Y | — | DOSE REDUCED | |
| YAMAA-01 | 2025-01-05 | 1 | Y | — | DOSE REDUCED | |
| YAMAA-01 | 2025-01-06 | 1 | Y | — | DOSE REDUCED |
Expected output
Under review10 expected rowsexpected/ec.csv
Edit10 rows| DOMAIN | STUDYID | USUBJID | ECSEQ | ECTRT | ECMOOD | ECOCCUR | ECREASND | ECDOSE | ECDOSU | ECDOSFRM | ECDOSFRQ | ECROUTE | ECSTDTC | ECENDTC | ECADJ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EC | YAMAA-01 | 1 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-02 | 2025-01-02 | — | |
| EC | YAMAA-01 | 2 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-03 | 2025-01-03 | — | |
| EC | YAMAA-01 | 3 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-04 | 2025-01-04 | — | |
| EC | YAMAA-01 | 4 | Study Drug | PERFORMED | N | SUBJECT FORGOT DOSE | — | tablet | TABLET | QD | ORAL | 2025-01-05 | 2025-01-05 | — | |
| EC | YAMAA-01 | 5 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-06 | 2025-01-06 | — | |
| EC | YAMAA-01 | 1 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-02 | 2025-01-02 | — | |
| EC | YAMAA-01 | 2 | Study Drug | PERFORMED | Y | — | 2 | tablet | TABLET | QD | ORAL | 2025-01-03 | 2025-01-03 | — | |
| EC | YAMAA-01 | 3 | Study Drug | PERFORMED | Y | — | 1 | tablet | TABLET | QD | ORAL | 2025-01-04 | 2025-01-04 | DOSE REDUCED | |
| EC | YAMAA-01 | 4 | Study Drug | PERFORMED | Y | — | 1 | tablet | TABLET | QD | ORAL | 2025-01-05 | 2025-01-05 | DOSE REDUCED | |
| EC | YAMAA-01 | 5 | Study Drug | PERFORMED | Y | — | 1 | tablet | TABLET | QD | ORAL | 2025-01-06 | 2025-01-06 | DOSE REDUCED |
Shaded columns are derived— = missing value
Human review and assessment
Not under review
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "EC". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from LOG.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from LOG.USUBJID. | SDTM | 3 | ||||
| ECTRT | Name of Treatment | Char | Assigned | Set constant value "Study Drug". | SDTM | 4 | ||||
| ECMOOD | Mood | Char | Assigned | Set constant value "PERFORMED". | SDTM | 5 | ||||
| ECOCCUR | EC Occurrence | Char | Collected | Copy value from LOG.DOSE_TAKEN. | SDTM | 6 | ||||
| ECREASND | Reason Not Dosed | Char | Collected | Copy value from LOG.MISSREAS. | SDTM | 7 | ||||
| ECDOSE | Dose | Num | Derived | If ECOCCUR = 'Y' then "LOG.TABLETS"; otherwise blank. | SDTM | 8 | ||||
| ECDOSU | Dose Units | Char | Assigned | Set constant value "tablet". | SDTM | 9 | ||||
| ECDOSFRM | Dose Form | Char | Assigned | Set constant value "TABLET". | SDTM | 10 | ||||
| ECDOSFRQ | Dosing Frequency per Interval | Char | Assigned | Set constant value "QD". | SDTM | 11 | ||||
| ECROUTE | Route of Administration | Char | Assigned | Set constant value "ORAL". | SDTM | 12 | ||||
| ECSTDTC | Start Date/Time of Treatment | Char | ISO 8601 | Collected | Copy value from LOG.LOGDATE. | SDTM | 13 | |||
| ECENDTC | End Date/Time of Treatment | Char | ISO 8601 | Collected | Copy value from LOG.LOGDATE. | SDTM | 14 | |||
| ECADJ | Reason for Dose Adjustment | Char | Collected | Copy value from LOG.DOSE_ADJ. | SDTM | 15 | ||||
| ECSEQ | Sequence Number | Num | Derived | Row number within each group of STUDYID, USUBJID, ordered by LOG.LOGDATE. | SDTM | 16 |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-ec-collected-exposure/full.md
Edit25 linesFollowing CDISC SDTM standards, use the provided DOSING_LOG dataset to create an EC dataset with one record per dosing-log day.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ
- DOMAIN is "EC" and ECTRT is "Study Drug" on every record.
- ECMOOD is "PERFORMED" on every record.
- ECSEQ numbers each subject's dosing days from 1 in date order.
- ECOCCUR is "Y" when the subject took the day's dose and "N" when the dose was not taken.
- ECREASND is the reason the log records for a dose that was not taken and has no value when the dose was taken.
- ECDOSE is the number of tablets the subject took that day and has no value on a day the dose was not taken.
- ECDOSU is "tablet", ECDOSFRM is "TABLET", ECDOSFRQ is "QD", and ECROUTE is "ORAL" on every record, including a day the dose was not taken.
- ECSTDTC and ECENDTC are both the dosing day.
- ECADJ is the adjustment reason the log records for the day and has no value when the day records none.
Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.
prompts/sdtm-ec-collected-exposure/conventions.md
Edit14 linesFollowing CDISC SDTM standards, use the provided DOSING_LOG dataset to create an EC dataset with one record per dosing-log day.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ
- DOMAIN is "EC" and ECTRT is "Study Drug" on every record.
- ECMOOD is "PERFORMED" on every record.
- ECOCCUR is "Y" or "N".
- ECDOSFRM is "TABLET" and ECROUTE is "ORAL" on every record.
Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.
prompts/sdtm-ec-collected-exposure/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided DOSING_LOG dataset to create an EC dataset with one record per dosing-log day.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ
Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/sdtm-ec-collected-exposure/result.R
Edit41 lines# Reference solution for the yamaa benchmark sdtm-ec-collected-exposure (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)log <- read_csv( "/app/input/dosing_log.csv", col_types = cols(.default = col_character()))ec <- log |> mutate( DOMAIN = "EC", ECTRT = "Study Drug", ECMOOD = "PERFORMED", ECOCCUR = DOSE_TAKEN, ECREASND = ifelse(DOSE_TAKEN == "Y", NA_character_, na_if(MISSREAS, "")), ECDOSE = suppressWarnings(as.integer(ifelse(DOSE_TAKEN == "Y", TABLETS, NA_character_))), ECDOSU = "tablet", ECDOSFRM = "TABLET", ECDOSFRQ = "QD", ECROUTE = "ORAL", ECSTDTC = suppressWarnings(as.Date(LOGDATE)), ECENDTC = suppressWarnings(as.Date(LOGDATE)), ECADJ = na_if(DOSE_ADJ, "") ) |> arrange(STUDYID, USUBJID, ECSTDTC) |> group_by(STUDYID, USUBJID) |> mutate(ECSEQ = row_number()) |> ungroup() |> select( DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ )dir.create("/app/output", showWarnings = FALSE)write_csv(ec, "/app/output/ec.csv", na = "")solutions/sdtm-ec-collected-exposure/result.py
Edit81 lines# Reference solution for the yamaa benchmark sdtm-ec-collected-exposure (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build the SDTM EC dataset from the DOSING_LOG dataset.One EC record per dosing-log day, per the task instruction."""import csvfrom pathlib import PathINPUT_DIR = Path("/app/input")OUTPUT_DIR = Path("/app/output")COLUMNS = [ "DOMAIN", "STUDYID", "USUBJID", "ECSEQ", "ECTRT", "ECMOOD", "ECOCCUR", "ECREASND", "ECDOSE", "ECDOSU", "ECDOSFRM", "ECDOSFRQ", "ECROUTE", "ECSTDTC", "ECENDTC", "ECADJ",]def main() -> None: with (INPUT_DIR / "dosing_log.csv").open(newline="") as f: log = list(csv.DictReader(f)) # One record per dosing-log day; sequence each subject's days in # date order starting at 1. log.sort(key=lambda r: (r["USUBJID"], r["LOGDATE"])) records = [] counters: dict[str, int] = {} for row in log: subject = row["USUBJID"] counters[subject] = counters.get(subject, 0) + 1 taken = row["DOSE_TAKEN"].strip() == "Y" records.append( { "DOMAIN": "EC", "STUDYID": row["STUDYID"], "USUBJID": subject, "ECSEQ": counters[subject], "ECTRT": "Study Drug", "ECMOOD": "PERFORMED", "ECOCCUR": "Y" if taken else "N", "ECREASND": "" if taken else row["MISSREAS"], "ECDOSE": row["TABLETS"] if taken else "", "ECDOSU": "tablet", "ECDOSFRM": "TABLET", "ECDOSFRQ": "QD", "ECROUTE": "ORAL", "ECSTDTC": row["LOGDATE"], "ECENDTC": row["LOGDATE"], "ECADJ": row["DOSE_ADJ"], } ) OUTPUT_DIR.mkdir(parents=True, exist_ok=True) with (OUTPUT_DIR / "ec.csv").open("w", newline="") as f: writer = csv.DictWriter(f, fieldnames=COLUMNS) writer.writeheader() writer.writerows(records)if __name__ == "__main__": main()solutions/sdtm-ec-collected-exposure/result.sas
Edit24 lines/* opensas reference solution for the yamaa benchmark sdtm-ec-collected-exposure. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dosing_log; length STUDYID USUBJID LOGDATE TABLETS DOSE_TAKEN MISSREAS DOSE_ADJ $1024; infile "/app/input/dosing_log.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. LOGDATE : $char1024. TABLETS : $char1024. DOSE_TAKEN : $char1024. MISSREAS : $char1024. DOSE_ADJ : $char1024.;run;data prepared; set dosing_log; length DOMAIN $2 ECTRT ECMOOD ECOCCUR ECREASND ECDOSU ECDOSFRM ECDOSFRQ ECROUTE ECSTDTC ECENDTC ECADJ $1024;DOMAIN='EC'; ECTRT='Study Drug'; ECMOOD='PERFORMED'; ECOCCUR=DOSE_TAKEN; ECDOSE=.; ECREASND='';if ECOCCUR='Y' then ECDOSE=input(TABLETS,best32.); else ECREASND=MISSREAS;ECDOSU='tablet'; ECDOSFRM='TABLET'; ECDOSFRQ='QD'; ECROUTE='ORAL'; ECSTDTC=LOGDATE; ECENDTC=LOGDATE; ECADJ=DOSE_ADJ; run;proc sort data=prepared; by STUDYID USUBJID ECSTDTC; run;data result; set prepared; by STUDYID USUBJID; retain ECSEQ; if first.USUBJID then ECSEQ=0; ECSEQ+1;run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ from result;quit;proc export data=final outfile="/app/output/ec.csv" dbms=csv replace; run;