As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: one row per adverse event (AE) with an analysis start date
(ASTDT) completed from the collected start at whatever precision
it was collected, a flag (ASTDTF) marking the dates whose day was
supplied, and a treatment emergence flag (TRTEMFL).
Input: adverse event records carrying the reported term
(AETERM) and the collected start (AESTDTC): a full date, a year
and month, a year alone, non-date text, or nothing at all, plus the
first-exposure date (TRTSDT) from the subject-level analysis
dataset (ADSL), empty when the subject has no ADSL record.
Variables:
ASTDTis the analysis start date. A fully collected date is used as it stands. A year and month is completed to the 15th. A year alone, non-date text, and a missing value give no analysis date.ASTDTCis the same analysis date written as text, empty when there is no analysis date.ASTDTFisDwhen the day was supplied to complete the date. It is empty when the date was collected in full and when no analysis date could be formed, so the flag always matches the date shown. In CDISC terms only the day was imputed, which is the level of imputation the flag records.TRTEMFLisYwhen the analysis start falls on or afterTRTSDT. It stays empty when there is no analysis date, when the event started before first exposure, or when the subject's first-exposure date is unknown. A supplied day counts exactly as a collected one would, soASTDTFtells a reader which flagged events rested on a supplied day.
Note: a supplied day is never placed before a first exposure
that falls inside the collected month. When the 15th would fall
before TRTSDT, the date moves forward to the exposure date, which
the collected month still allows. When the whole collected month
ends before the exposure date, no day it allows could reach it, so
the date stays on the 15th rather than moving into a month nobody
recorded, and the event is not treatment emergent. A fully collected
date is left exactly as collected even when it falls before
TRTSDT, because there is nothing about it to choose.
Inputs
Under review2 source filesinput/adsl.csv
Edit3 rows| STUDYID | USUBJID | TRTSDT |
|---|---|---|
| CATH | 2025-01-10 | |
| CATH | 2025-03-20 | |
| CATH | 2025-02-15 |
input/ae.csv
Edit13 rows| STUDYID | USUBJID | AESEQ | AETERM | AESTDTC |
|---|---|---|---|---|
| CATH | 1 | HEADACHE | 2025-01-15 | |
| CATH | 2 | NAUSEA | 2025-01 | |
| CATH | 3 | ECZEMA FLARE | 2025 | |
| CATH | 4 | DIZZINESS | UNKNOWN | |
| CATH | 5 | FATIGUE | — | |
| CATH | 6 | RASH | 2025-02-20 | |
| CATH | 1 | PYREXIA | 2025-03 | |
| CATH | 2 | COUGH | 2025-02 | |
| CATH | 3 | INSOMNIA | 2025-04-05 | |
| CATH | 4 | MIGRAINE | 2025-01-05 | |
| CATH | 7 | FLUSHING | 2025-01-10 | |
| CATH | 1 | BACK PAIN | 2025-04 | |
| CATH | 1 | VOMITING | 2025-02 |
Expected output
Under review13 expected rowsexpected/adae.csv
Edit13 rows| STUDYID | USUBJID | AESEQ | AETERM | AESTDTC | ASTDT | ASTDTC | ASTDTF | TRTSDT | TRTEMFL |
|---|---|---|---|---|---|---|---|---|---|
| CATH | 1 | HEADACHE | 2025-01-15 | 2025-01-15 | 2025-01-15 | — | 2025-01-10 | Y | |
| CATH | 2 | NAUSEA | 2025-01 | 2025-01-15 | 2025-01-15 | D | 2025-01-10 | Y | |
| CATH | 3 | ECZEMA FLARE | 2025 | — | — | — | 2025-01-10 | — | |
| CATH | 4 | DIZZINESS | UNKNOWN | — | — | — | 2025-01-10 | — | |
| CATH | 5 | FATIGUE | — | — | — | — | 2025-01-10 | — | |
| CATH | 6 | RASH | 2025-02-20 | 2025-02-20 | 2025-02-20 | — | 2025-01-10 | Y | |
| CATH | 1 | PYREXIA | 2025-03 | 2025-03-20 | 2025-03-20 | D | 2025-03-20 | Y | |
| CATH | 2 | COUGH | 2025-02 | 2025-02-15 | 2025-02-15 | D | 2025-03-20 | — | |
| CATH | 3 | INSOMNIA | 2025-04-05 | 2025-04-05 | 2025-04-05 | — | 2025-03-20 | Y | |
| CATH | 4 | MIGRAINE | 2025-01-05 | 2025-01-05 | 2025-01-05 | — | 2025-03-20 | — | |
| CATH | 7 | FLUSHING | 2025-01-10 | 2025-01-10 | 2025-01-10 | — | 2025-01-10 | Y | |
| CATH | 1 | BACK PAIN | 2025-04 | 2025-04-15 | 2025-04-15 | D | — | — | |
| CATH | 1 | VOMITING | 2025-02 | 2025-02-15 | 2025-02-15 | D | 2025-02-15 | Y |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADAE | STUDYID | Study Identifier | character | Copy value from AE.STUDYID. | Predecessor | ||
| ADAE | USUBJID | Unique Subject Identifier | character | Copy value from AE.USUBJID. | Predecessor | ||
| ADAE | AESEQ | Sequence Number | numeric | Copy value from AE.AESEQ. | Predecessor | ||
| ADAE | AETERM | Reported Term for the Adverse Event | character | Copy value from AE.AETERM. | Predecessor | ||
| ADAE | AESTDTC | Start Date/Time of Adverse Event | character | Copy value from AE.AESTDTC. | Predecessor | ||
| ADAE | TRTSDT | Date of First Exposure to Treatment | numeric | ISO 8601 | Copy value from ADSL.TRTSDT. | Predecessor | |
| ADAE | ASTDT | Analysis Start Date | numeric | ISO 8601 | Impute incomplete date from AESTDTC, defaulting day=15, never before TRTSDT; a bound outside the collected period is ignored; a year alone gives blank; invalid text → blank. | Derived | |
| ADAE | ASTDTC | Analysis Start Date (C) | character | Copy value from ASTDT. | Derived | ||
| ADAE | ASTDTF | Analysis Start Date Imputation Flag | character | D | Date imputation flag for ASTDT: "Y" when the whole date was imputed, "M" when the month and day were, "D" when only the day was; blank when the date was collected in full. | Derived | |
| ADAE | TRTEMFL | Treatment Emergent Analysis Flag | character | Y | "Y" when ASTDT >= TRTSDT; blank otherwise. | Derived |
| Variable | Permitted values |
|---|---|
| ASTDTF | D |
| TRTEMFL | Y |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adae-partial-dates/full.md
Edit27 linesFollowing CDISC ADaM standards, use the provided ADSL and AE datasets to create an ADAE dataset with one record per adverse event.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, AESTDTC, ASTDT, ASTDTC, ASTDTF, TRTSDT, TRTEMFL
ASTDT is the collected start date as it stands when AESTDTC is a full date. A full date is left exactly as collected even when it falls before TRTSDT. When AESTDTC holds a year and a month only, ASTDT is the 15th of that month. When that 15th falls before TRTSDT and TRTSDT falls in the same month, ASTDT is TRTSDT instead. When the whole of that month falls before TRTSDT, ASTDT stays on the 15th. ASTDT has no value when AESTDTC holds a year only, holds text that is not a date, or has no value. When TRTSDT has no value, a year and a month is still completed to the 15th. ASTDTC is ASTDT written as text in the form "YYYY-MM-DD", and ASTDTC has no value when ASTDT has no value. ASTDTF is "D" when the day of ASTDT was supplied to complete a year and a month, and ASTDTF has no value when AESTDTC is a full date or when ASTDT has no value. TRTEMFL is "Y" when ASTDT falls on or after TRTSDT. A supplied day counts for TRTEMFL exactly as a collected day would. TRTEMFL has no value when ASTDT has no value, when ASTDT falls before TRTSDT, or when TRTSDT has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.
prompts/adam-adae-partial-dates/conventions.md
Edit12 linesFollowing CDISC ADaM standards, use the provided ADSL and AE datasets to create an ADAE dataset with one record per adverse event.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, AESTDTC, ASTDT, ASTDTC, ASTDTF, TRTSDT, TRTEMFL
ASTDTC is ASTDT written as text in the form "YYYY-MM-DD" or has no value. ASTDTF is "D" or has no value. TRTEMFL is "Y" or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.
prompts/adam-adae-partial-dates/brief.md
Edit9 linesFollowing CDISC ADaM standards, use the provided ADSL and AE datasets to create an ADAE dataset with one record per adverse event.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, AESTDTC, ASTDT, ASTDTC, ASTDTF, TRTSDT, TRTEMFL
Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-adae-partial-dates/result.R
Edit64 lines# Reference solution for the yamaa benchmark adam-adae-partial-dates (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate, warn.conflicts = FALSE)library(readr)library(stringr)ae <- read_csv( "/app/input/ae.csv", col_types = cols(AESEQ = col_integer(), .default = col_character()))adsl <- read_csv( "/app/input/adsl.csv", col_types = cols(TRTSDT = col_date(), .default = col_character())) |> select(USUBJID, TRTSDT)adae <- ae |> left_join(adsl, by = "USUBJID") |> mutate( is_full = str_detect(AESTDTC, "^\\d{4}-\\d{2}-\\d{2}$"), is_month = str_detect(AESTDTC, "^\\d{4}-\\d{2}$"), # A year and month is completed to the 15th. completed = if_else( is_month, ymd(paste0(AESTDTC, "-15"), quiet = TRUE), ymd(AESTDTC, quiet = TRUE) ), completed = if_else(is_full | is_month, completed, as.Date(NA)), # A supplied day is never placed before a first exposure that falls # inside the collected month. When the whole collected month ends # before the exposure date, no day it allows could reach it, so the # date stays on the 15th. month_end = if_else( is_month, completed + days(days_in_month(completed) - 15L), completed ), month_before_trt = is_month & !is.na(month_end) & !is.na(TRTSDT) & month_end < TRTSDT, ASTDT = case_when( month_before_trt ~ completed, is_month & !is.na(completed) & !is.na(TRTSDT) & completed < TRTSDT ~ TRTSDT, .default = completed ), ASTDTC = if_else(is.na(ASTDT), NA_character_, as.character(ASTDT)), ASTDTF = if_else(is_month & !is.na(ASTDT), "D", NA_character_), TRTEMFL = if_else( !is.na(ASTDT) & !is.na(TRTSDT) & ASTDT >= TRTSDT, "Y", NA_character_ ) ) |> select( STUDYID, USUBJID, AESEQ, AETERM, AESTDTC, ASTDT, ASTDTC, ASTDTF, TRTSDT, TRTEMFL )dir.create("/app/output", showWarnings = FALSE)write_csv(adae, "/app/output/adae.csv", na = "")solutions/adam-adae-partial-dates/result.py
Edit110 lines# Reference solution for the yamaa benchmark adam-adae-partial-dates (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build ADAE from ADSL and AE, completing partial AE start dates.ASTDT is the adverse event start date derived from the collectedAESTDTC character date:- a full date is kept exactly as collected, even before treatment start;- a year and month only is completed to the 15th of that month, except that when the 15th falls before TRTSDT within the same month ASTDT is TRTSDT; when the whole month falls before TRTSDT the 15th stands;- a year only, non-date text, or a missing value gives no ASTDT.ASTDTF is "D" when a day was supplied to complete a year and month.TRTEMFL is "Y" when ASTDT falls on or after TRTSDT."""import csvimport osimport refrom datetime import dateINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"FULL_DATE = re.compile(r"^(\d{4})-(\d{2})-(\d{2})$")YEAR_MONTH = re.compile(r"^(\d{4})-(\d{2})$")def parse_full_date(text): """Return a date for a strict YYYY-MM-DD string, else None.""" match = FULL_DATE.match(text or "") if not match: return None try: return date(int(match.group(1)), int(match.group(2)), int(match.group(3))) except ValueError: return Nonedef derive_astdt(aestdtc, trtsdt): """Return (ASTDT as a date or None, day-imputed flag).""" aestdtc = (aestdtc or "").strip() full = parse_full_date(aestdtc) if full is not None: return full, False match = YEAR_MONTH.match(aestdtc) if match: year, month = int(match.group(1)), int(match.group(2)) if not 1 <= month <= 12: return None, False candidate = date(year, month, 15) if trtsdt is None or candidate >= trtsdt: return candidate, True if trtsdt.year == year and trtsdt.month == month: return trtsdt, True # The whole month falls before TRTSDT, so the 15th stands. return candidate, True return None, Falsedef main(): with open(os.path.join(INPUT_DIR, "adsl.csv"), newline="") as handle: adsl = list(csv.DictReader(handle)) with open(os.path.join(INPUT_DIR, "ae.csv"), newline="") as handle: ae = list(csv.DictReader(handle)) trtsdt_by_subject = {} for row in adsl: trtsdt_by_subject[row["USUBJID"]] = parse_full_date(row.get("TRTSDT")) columns = [ "STUDYID", "USUBJID", "AESEQ", "AETERM", "AESTDTC", "ASTDT", "ASTDTC", "ASTDTF", "TRTSDT", "TRTEMFL", ] out_rows = [] for row in ae: trtsdt = trtsdt_by_subject.get(row["USUBJID"]) astdt, day_imputed = derive_astdt(row.get("AESTDTC"), trtsdt) astdtc = astdt.isoformat() if astdt is not None else "" astdtf = "D" if (day_imputed and astdt is not None) else "" trtemfl = "" if astdt is not None and trtsdt is not None and astdt >= trtsdt: trtemfl = "Y" out_rows.append({ "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "AESEQ": row["AESEQ"], "AETERM": row["AETERM"], "AESTDTC": row["AESTDTC"], "ASTDT": astdt.isoformat() if astdt is not None else "", "ASTDTC": astdtc, "ASTDTF": astdtf, "TRTSDT": trtsdt.isoformat() if trtsdt is not None else "", "TRTEMFL": trtemfl, }) os.makedirs(OUTPUT_DIR, exist_ok=True) with open(os.path.join(OUTPUT_DIR, "adae.csv"), "w", newline="") as handle: writer = csv.DictWriter(handle, fieldnames=columns) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/adam-adae-partial-dates/result.sas
Edit41 lines/* opensas reference solution for the yamaa benchmark adam-adae-partial-dates. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data adsl; length STUDYID USUBJID TRTSDT $1024; infile "/app/input/adsl.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. TRTSDT : $char1024.;run;data ae; length STUDYID USUBJID AETERM AESTDTC $1024; infile "/app/input/ae.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. AESEQ AETERM : $char1024. AESTDTC : $char1024.;run;proc sql; create table joined as select a.*,b.TRTSDT from ae as a left join adsl as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID; quit;data result; length ASTDT ASTDTC $10 ASTDTF TRTEMFL $1; set joined; treatment=input(TRTSDT,yymmdd10.); analysis=.; if prxmatch('/^[0-9]{4}-[0-9]{2}-[0-9]{2}$/',strip(AESTDTC)) then analysis=input(AESTDTC,yymmdd10.); else if prxmatch('/^[0-9]{4}-[0-9]{2}$/',strip(AESTDTC)) then do; analysis=input(cats(AESTDTC,'-15'),yymmdd10.); if not missing(analysis) then do; /* First exposure moves the 15th forward only when it falls inside the collected month; when it falls after the whole month, the 15th stands. */ if missing(treatment) or treatment<=intnx('month',analysis,0,'end') then analysis=max(analysis,treatment); ASTDTF='D'; end; end; if not missing(analysis) then do; ASTDT=put(analysis,yymmdd10.); ASTDTC=ASTDT; if not missing(treatment) and analysis>=treatment then TRTEMFL='Y'; end;run;proc sql; create table final as select STUDYID, USUBJID, AESEQ, AETERM, AESTDTC, ASTDT, ASTDTC, ASTDTF, TRTSDT, TRTEMFL from result;quit;proc export data=final outfile="/app/output/adae.csv" dbms=csv replace; run;
yamaa