yamaa As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
EditUnder reviewWhat this benchmark meansREADME.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: one row per concomitant medication with an analysis end date
(AENDT) completed from the collected end at whatever precision it was
collected, never later than a date of death that falls within the
collected month or year, and a flag (AENDTF) marking how much of the
date was supplied.
Input: concomitant medication records, each identified by its
sequence number (CMSEQ), carrying the reported name (CMTRT) and the
collected end (CMENDTC): a full date, a year and month, a year alone,
or nothing at all, plus the date of death (DTHDT) from the
subject-level analysis dataset (ADSL), empty for a subject who has not
died.
Variables:
AENDTis the analysis end date. A fully collected date is used as it stands. A year and month is completed to the last day of that month, and a year alone to December 31 of that year. A missing end gives no analysis date.AENDTFisDwhen only the day was supplied andMwhen the month and day were supplied. It is empty when the date was collected in full and when no analysis date could be formed.DTHDTis the subject's date of death, empty when the subject has not died.
Note: a supplied end is never placed after a death that falls
inside the collected month or year. When the last day of the collected
month or year falls after DTHDT, the date moves back to the date of
death, which the collected month or year still allows, and the flag
still records what was supplied. When the whole collected month begins
after the date of death, no day it allows could reach it, so the end
stays on the last day of that month rather than moving into a month
nobody recorded. A fully collected date is left exactly as collected
even when it falls after DTHDT.
Inputs
Under review2 source filesinput/adsl.csv
Edit5 rows| STUDYID | USUBJID | DTHDT |
|---|---|---|
| YAMAA-01 | 2024-03-12 | |
| YAMAA-01 | — | |
| YAMAA-01 | 2024-02-10 | |
| YAMAA-01 | 2024-04-20 | |
| YAMAA-01 | 2024-06-30 |
input/cm.csv
Edit11 rows| STUDYID | USUBJID | CMSEQ | CMTRT | CMENDTC |
|---|---|---|---|---|
| YAMAA-01 | 1 | Paracetamol | 2024-03 | |
| YAMAA-01 | 2 | Omeprazole | 2024-03-08 | |
| YAMAA-01 | 3 | Loratadine | 2024-03-20 | |
| YAMAA-01 | 1 | Amoxicillin | 2024-03 | |
| YAMAA-01 | 2 | Cetirizine | 2024-02 | |
| YAMAA-01 | 1 | Atorvastatin | 2024 | |
| YAMAA-01 | 2 | Aspirin | 2023 | |
| YAMAA-01 | 1 | Simvastatin | 2024-05 | |
| YAMAA-01 | 2 | Ramipril | 2024-04 | |
| YAMAA-01 | 1 | Insulin glargine | 2024-06 | |
| YAMAA-01 | 2 | Warfarin | — |
Expected output
Under review11 expected rowsexpected/adcm.csv
Edit11 rows| STUDYID | USUBJID | CMSEQ | CMTRT | CMENDTC | AENDT | AENDTF | DTHDT |
|---|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | Paracetamol | 2024-03 | 2024-03-12 | D | 2024-03-12 | |
| YAMAA-01 | 2 | Omeprazole | 2024-03-08 | 2024-03-08 | — | 2024-03-12 | |
| YAMAA-01 | 3 | Loratadine | 2024-03-20 | 2024-03-20 | — | 2024-03-12 | |
| YAMAA-01 | 1 | Amoxicillin | 2024-03 | 2024-03-31 | D | — | |
| YAMAA-01 | 2 | Cetirizine | 2024-02 | 2024-02-29 | D | — | |
| YAMAA-01 | 1 | Atorvastatin | 2024 | 2024-02-10 | M | 2024-02-10 | |
| YAMAA-01 | 2 | Aspirin | 2023 | 2023-12-31 | M | 2024-02-10 | |
| YAMAA-01 | 1 | Simvastatin | 2024-05 | 2024-05-31 | D | 2024-04-20 | |
| YAMAA-01 | 2 | Ramipril | 2024-04 | 2024-04-20 | D | 2024-04-20 | |
| YAMAA-01 | 1 | Insulin glargine | 2024-06 | 2024-06-30 | D | 2024-06-30 | |
| YAMAA-01 | 2 | Warfarin | — | — | — | 2024-06-30 |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADCM | STUDYID | Study Identifier | character | Copy value from CM.STUDYID. | Predecessor | ||
| ADCM | USUBJID | Unique Subject Identifier | character | Copy value from CM.USUBJID. | Predecessor | ||
| ADCM | CMSEQ | Sequence Number | numeric | Copy value from CM.CMSEQ. | Predecessor | ||
| ADCM | CMTRT | Reported Name of Drug, Medication, or Therapy | character | Copy value from CM.CMTRT. | Predecessor | ||
| ADCM | CMENDTC | End Date/Time of Medication | character | Copy value from CM.CMENDTC. | Predecessor | ||
| ADCM | DTHDT | Date of Death | numeric | ISO 8601 | Copy value from ADSL.DTHDT. | Predecessor | |
| ADCM | AENDT | Analysis End Date | numeric | ISO 8601 | Impute incomplete date from CMENDTC, defaulting month=12, day=last, never after DTHDT; a bound outside the collected period is ignored; invalid text → blank. | Derived | |
| ADCM | AENDTF | Analysis End Date Imputation Flag | character | D, M | Date imputation flag for AENDT: "Y" when the whole date was imputed, "M" when the month and day were, "D" when only the day was; blank when the date was collected in full. | Derived |
| Variable | Permitted values |
|---|---|
| AENDTF | D, M |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adcm-end-date-bound/full.md
Edit25 linesFollowing CDISC ADaM standards, use the provided ADSL and CM datasets to create an ADCM dataset with one record per concomitant medication.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, CMSEQ, CMTRT, CMENDTC, AENDT, AENDTF, DTHDT
AENDT is the analysis end date, formed from the collected end date. A collected end date given in full is used exactly as collected, even when it falls after the subject's date of death. A collected end date that gives a year and month is completed to the last day of that month. A collected end date that gives a year alone is completed to December 31 of that year. A missing collected end date gives AENDT no value. When the completed end date falls after the subject's date of death, DTHDT, but the collected month or year also contains DTHDT, AENDT is DTHDT. When the whole collected month or year begins after DTHDT, AENDT stays on the last day of that month or year. AENDTF is "D" when only the day was supplied and "M" when the month and day were supplied. AENDTF has no value when the collected end date was given in full and when AENDT has no value. AENDTF still records what was supplied when AENDT moves back to the date of death. DTHDT is the subject's date of death from ADSL, and has no value when the subject has not died.
Read the source datasets from /app/input and save the completed dataset as /app/output/adcm.csv.
prompts/adam-adcm-end-date-bound/conventions.md
Edit10 linesFollowing CDISC ADaM standards, use the provided ADSL and CM datasets to create an ADCM dataset with one record per concomitant medication.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, CMSEQ, CMTRT, CMENDTC, AENDT, AENDTF, DTHDT
AENDTF is "D" or "M", or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/adcm.csv.
prompts/adam-adcm-end-date-bound/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided ADSL and CM datasets to create an ADCM dataset with one record per concomitant medication.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, CMSEQ, CMTRT, CMENDTC, AENDT, AENDTF, DTHDT
Read the source datasets from /app/input and save the completed dataset as /app/output/adcm.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-adcm-end-date-bound/result.R
Edit54 lines# Reference solution for the yamaa benchmark adam-adcm-end-date-bound (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate, warn.conflicts = FALSE)library(readr)cm <- read_csv( "/app/input/cm.csv", col_types = cols(CMSEQ = col_integer(), .default = col_character()))adsl <- read_csv("/app/input/adsl.csv", col_types = cols(.default = col_character()))# A full date stands as collected; a year and month runs to its last day,# a year alone to December 31, never past a date of death that falls inside# the collected month or year. When the whole collected month or year lies# after death, the last day stands.adcm <- cm |> left_join(select(adsl, STUDYID, USUBJID, DTHDT), by = c("STUDYID", "USUBJID")) |> mutate( LEN = nchar(CMENDTC), FIRST = case_when( LEN == 7 ~ ymd(paste0(CMENDTC, "-01"), quiet = TRUE), LEN == 4 ~ ymd(paste0(CMENDTC, "-01-01"), quiet = TRUE) ), LAST = case_when( LEN == 7 ~ ceiling_date(FIRST, "month") - days(1), LEN == 4 ~ ymd(paste0(CMENDTC, "-12-31"), quiet = TRUE) ), DEATH = ymd(DTHDT, quiet = TRUE), BOUNDED = case_when( is.na(DEATH) ~ LAST, DEATH < FIRST ~ LAST, TRUE ~ pmin(LAST, DEATH) ), AENDT = case_when( LEN == 10 ~ CMENDTC, LEN %in% c(4, 7) & !is.na(BOUNDED) ~ format(BOUNDED, "%Y-%m-%d"), TRUE ~ NA_character_ ), AENDTF = case_when( is.na(AENDT) ~ NA_character_, LEN == 7 ~ "D", LEN == 4 ~ "M", TRUE ~ NA_character_ ) ) |> select(STUDYID, USUBJID, CMSEQ, CMTRT, CMENDTC, AENDT, AENDTF, DTHDT)dir.create("/app/output", showWarnings = FALSE)write_csv(adcm, "/app/output/adcm.csv", na = "")solutions/adam-adcm-end-date-bound/result.py
Edit98 lines# Reference solution for the yamaa benchmark adam-adcm-end-date-bound (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build ADCM from ADSL and CM, bounding partial end dates at death."""import calendarimport csvfrom datetime import dateINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"def read_csv(path): with open(path, newline="") as handle: return list(csv.DictReader(handle))def parse_full(text): year, month, day = (int(part) for part in text.split("-")) return date(year, month, day)def derive_end(cmendtc, dthdt): """Return (aendt, aendtf) for one collected end date.""" if not cmendtc: return "", "" parts = cmendtc.split("-") if len(parts) == 3: # A full date is used exactly as collected, even after death. return cmendtc, "" if len(parts) == 2: year, month = int(parts[0]), int(parts[1]) last_day = calendar.monthrange(year, month)[1] completed = date(year, month, last_day) flag = "D" period_start = date(year, month, 1) period_end = completed else: year = int(parts[0]) completed = date(year, 12, 31) flag = "M" period_start = date(year, 1, 1) period_end = completed if dthdt and completed > dthdt and period_start <= dthdt <= period_end: # The collected month or year also contains the death date; # the flag still records what was supplied. return dthdt.isoformat(), flag # When the whole collected month or year begins after death, the last # day stands. return completed.isoformat(), flagdef main(): adsl = read_csv(f"{INPUT_DIR}/adsl.csv") death = {} for row in adsl: death[row["USUBJID"]] = parse_full(row["DTHDT"]) if row["DTHDT"] else None cm = read_csv(f"{INPUT_DIR}/cm.csv") out_rows = [] for row in cm: dthdt = death.get(row["USUBJID"]) aendt, aendtf = derive_end(row["CMENDTC"], dthdt) out_rows.append( { "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "CMSEQ": row["CMSEQ"], "CMTRT": row["CMTRT"], "CMENDTC": row["CMENDTC"], "AENDT": aendt, "AENDTF": aendtf, "DTHDT": dthdt.isoformat() if dthdt else "", } ) fieldnames = [ "STUDYID", "USUBJID", "CMSEQ", "CMTRT", "CMENDTC", "AENDT", "AENDTF", "DTHDT", ] with open(f"{OUTPUT_DIR}/adcm.csv", "w", newline="") as handle: writer = csv.DictWriter(handle, fieldnames=fieldnames) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/adam-adcm-end-date-bound/result.sas
Edit59 lines/* opensas reference solution for the yamaa benchmark adam-adcm-end-date-bound. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data cm; length STUDYID USUBJID CMTRT CMENDTC $1024; infile "/app/input/cm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. CMSEQ CMTRT : $char1024. CMENDTC : $char1024.;run;data adsl; length STUDYID USUBJID DTHDT $1024; infile "/app/input/adsl.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. DTHDT : $char1024.;run;proc sql; create table joined as select a.*, b.DTHDT from cm as a left join adsl as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID order by a.STUDYID, a.USUBJID, a.CMSEQ;quit;/* A full date stands as collected; a year and month runs to its last day, a year alone to December 31, never past a date of death that falls inside the collected month or year. When the whole collected month or year lies after death, the last day stands. */data result; length AENDT $10 AENDTF $1; set joined; AENDT=''; AENDTF=''; len=lengthn(CMENDTC); death=.; if not missing(DTHDT) then death=input(DTHDT, yymmdd10.); if len=10 then AENDT=CMENDTC; else if len in (4, 7) then do; if len=7 then do; first=input(cats(CMENDTC, '-01'), yymmdd10.); last=intnx('month', first, 0, 'end'); flag='D'; end; else do; first=mdy(1, 1, input(CMENDTC, 4.)); last=mdy(12, 31, input(CMENDTC, 4.)); flag='M'; end; if not missing(death) then do; if death >= first then last=min(last, death); end; if not missing(last) then do; AENDT=put(last, yymmdd10.); AENDTF=flag; end; end;run;proc sql; create table final as select STUDYID, USUBJID, CMSEQ, CMTRT, CMENDTC, AENDT, AENDTF, DTHDT from result;quit;proc export data=final outfile="/app/output/adcm.csv" dbms=csv replace; run;