As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: derive date/time of death (DTHDTC) and death flag (DTHFL) for
each demographics subject.
Input: demographics subjects, disposition records with status and start date, and adverse events with outcome and end date.
Variables:
DTHDTCis the latest disposition death date, or the latest fatal adverse event end date when disposition has no death; empty when neither source records a death.DTHFLisYwhen either source supplies a death date; empty otherwise.
Note: when the sources record different death dates, the disposition date wins. An undated death or fatal event cannot displace a dated one.
Inputs
Under review3 source filesinput/ae.csv
Edit6 rows| STUDYID | USUBJID | AESEQ | AETERM | AEOUT | AEENDTC |
|---|---|---|---|---|---|
| CATH | 1 | HEADACHE | RECOVERED | 2025-01-10 | |
| CATH | 1 | CARDIAC ARREST | FATAL | 2025-02-15 | |
| CATH | 1 | RESPIRATORY FAILURE | FATAL | 2025-03-22 | |
| CATH | 1 | NAUSEA | RECOVERED | 2025-04-10 | |
| CATH | 1 | MYOCARDIAL INFARCTION | FATAL | 2025-05-25 | |
| CATH | 2 | CARDIAC ARREST | FATAL | — |
input/dm_raw.csv
Edit5 rows| STUDYID | USUBJID |
|---|---|
| CATH | |
| CATH | |
| CATH | |
| CATH | |
| CATH |
input/ds.csv
Edit6 rows| STUDYID | USUBJID | DSSEQ | DSDECOD | DSCAT | DSSTDTC |
|---|---|---|---|---|---|
| CATH | 1 | DEATH | DISPOSITION EVENT | 2025-01-15 | |
| CATH | 1 | COMPLETED | DISPOSITION EVENT | 2025-02-20 | |
| CATH | 1 | DEATH | DISPOSITION EVENT | 2025-03-20 | |
| CATH | 1 | COMPLETED | DISPOSITION EVENT | 2025-04-20 | |
| CATH | 1 | DEATH | DISPOSITION EVENT | 2025-05-25 | |
| CATH | 2 | DEATH | DISPOSITION EVENT | — |
Expected output
Under review5 expected rowsexpected/dm.csv
Edit5 rows| DOMAIN | STUDYID | USUBJID | DTHDTC | DTHFL |
|---|---|---|---|---|
| DM | CATH | 2025-01-15 | Y | |
| DM | CATH | 2025-02-15 | Y | |
| DM | CATH | 2025-03-20 | Y | |
| DM | CATH | — | — | |
| DM | CATH | 2025-05-25 | Y |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "DM". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from DM_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from DM_RAW.USUBJID. | SDTM | 3 | ||||
| DTHDTC | Date/Time of Death | Char | ISO 8601 | Derived | First non-missing of DSDTHDT and AEDTHDT. | SDTM | 4 | |||
| DTHFL | Subject Death Flag | Char | Y | Derived | "Y" when DSDTHDT IS NOT NULL OR AEDTHDT IS NOT NULL; blank otherwise. | SDTM | 5 |
| Variable | Permitted values |
|---|---|
| DTHFL | Y |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-dm-death/full.md
Edit17 linesFollowing CDISC SDTM standards, use the provided DM_RAW, DS, and AE datasets to create a DM dataset with one record per subject.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DTHDTC, DTHFL
DTHDTC is the latest disposition death date, or the latest fatal adverse event end date when disposition records no death. A disposition death is a record with DSDECOD DEATH; a fatal event is one with AEOUT FATAL. DTHDTC has no value when neither source records a death. DTHFL is Y when either source supplies a death date, and has no value otherwise. When the sources record different death dates, the disposition date wins. An undated death or fatal event cannot displace a dated one.
Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.
prompts/sdtm-dm-death/conventions.md
Edit10 linesFollowing CDISC SDTM standards, use the provided DM_RAW, DS, and AE datasets to create a DM dataset with one record per subject.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DTHDTC, DTHFL
DTHFL is Y or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.
prompts/sdtm-dm-death/brief.md
Edit8 linesFollowing CDISC SDTM standards, use the provided DM_RAW, DS, and AE datasets to create a DM dataset with one record per subject.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DTHDTC, DTHFL
Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.
run.R
Edit4 lineslibrary(yamaa)dm <- yamaa_domain("spec.yaml")$outputdmrun.py
Edit4 linesimport yamaadm = yamaa.yamaa_domain("spec.yaml").outputdmsolutions/sdtm-dm-death/result.R
Edit48 lines# Reference solution for the yamaa benchmark sdtm-dm-death (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)dm_raw <- read_csv( "/app/input/dm_raw.csv", col_types = cols(.default = col_character()))ds <- read_csv( "/app/input/ds.csv", col_types = cols(.default = col_character()))ae <- read_csv( "/app/input/ae.csv", col_types = cols(.default = col_character()))ds_death <- ds |> filter(DSDECOD == "DEATH") |> mutate(DSSTDTC = suppressWarnings(as.Date(DSSTDTC))) |> filter(!is.na(DSSTDTC)) |> group_by(STUDYID, USUBJID) |> summarise(DS_DTH = max(DSSTDTC), .groups = "drop")ae_fatal <- ae |> filter(AEOUT == "FATAL") |> mutate(AEENDTC = suppressWarnings(as.Date(AEENDTC))) |> filter(!is.na(AEENDTC)) |> group_by(STUDYID, USUBJID) |> summarise(AE_DTH = max(AEENDTC), .groups = "drop")dm <- dm_raw |> left_join(ds_death, by = c("STUDYID", "USUBJID")) |> left_join(ae_fatal, by = c("STUDYID", "USUBJID")) |> mutate( DOMAIN = "DM", DTHDTC = coalesce(DS_DTH, AE_DTH), DTHFL = ifelse(!is.na(DTHDTC), "Y", NA_character_) ) |> select(DOMAIN, STUDYID, USUBJID, DTHDTC, DTHFL)dir.create("/app/output", showWarnings = FALSE)write_csv(dm, "/app/output/dm.csv", na = "")solutions/sdtm-dm-death/result.py
Edit47 lines# Reference solution for the yamaa benchmark sdtm-dm-death (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pldm_raw = pl.read_csv("/app/input/dm_raw.csv", infer_schema=False)ds = pl.read_csv("/app/input/ds.csv", infer_schema=False)ae = pl.read_csv("/app/input/ae.csv", infer_schema=False)ds_death = ( ds.filter(pl.col("DSDECOD") == "DEATH") .with_columns(pl.col("DSSTDTC").str.to_date(strict=False)) .filter(pl.col("DSSTDTC").is_not_null()) .group_by("STUDYID", "USUBJID") .agg(pl.col("DSSTDTC").max().alias("DS_DTH")))ae_fatal = ( ae.filter(pl.col("AEOUT") == "FATAL") .with_columns(pl.col("AEENDTC").str.to_date(strict=False)) .filter(pl.col("AEENDTC").is_not_null()) .group_by("STUDYID", "USUBJID") .agg(pl.col("AEENDTC").max().alias("AE_DTH")))dm = ( dm_raw.join(ds_death, on=["STUDYID", "USUBJID"], how="left") .join(ae_fatal, on=["STUDYID", "USUBJID"], how="left") .with_columns( DOMAIN=pl.lit("DM"), DTHDTC=pl.coalesce("DS_DTH", "AE_DTH"), ) .with_columns( DTHFL=pl.when(pl.col("DTHDTC").is_not_null()) .then(pl.lit("Y")) .otherwise(None) ) .select("DOMAIN", "STUDYID", "USUBJID", "DTHDTC", "DTHFL"))Path("/app/output").mkdir(exist_ok=True)dm.write_csv("/app/output/dm.csv")solutions/sdtm-dm-death/result.sas
Edit31 lines/* opensas reference solution for the yamaa benchmark sdtm-dm-death. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ae; length STUDYID USUBJID AETERM AEOUT AEENDTC $1024; infile "/app/input/ae.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. AESEQ AETERM : $char1024. AEOUT : $char1024. AEENDTC : $char1024.;run;data dm_raw; length STUDYID USUBJID $1024; infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024.;run;data ds; length STUDYID USUBJID DSDECOD DSCAT DSSTDTC $1024; infile "/app/input/ds.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. DSSEQ DSDECOD : $char1024. DSCAT : $char1024. DSSTDTC : $char1024.;run;proc sql;create table dsdeath as select STUDYID,USUBJID,max(DSSTDTC) as DSDATE from ds where DSDECOD='DEATH' and not missing(DSSTDTC) group by STUDYID,USUBJID;create table aedeath as select STUDYID,USUBJID,max(AEENDTC) as AEDATE from ae where AEOUT='FATAL' and not missing(AEENDTC) group by STUDYID,USUBJID;create table joined as select a.*,b.DSDATE,c.AEDATE from dm_raw a left join dsdeath b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID left join aedeath c on a.STUDYID=c.STUDYID and a.USUBJID=c.USUBJID; quit;data result; set joined; length DOMAIN $2 DTHDTC $1024 DTHFL $1; DOMAIN='DM'; DTHDTC=coalescec(DSDATE,AEDATE); if not missing(DTHDTC) then DTHFL='Y'; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, DTHDTC, DTHFL from result;quit;proc export data=final outfile="/app/output/dm.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions