As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: derive the combined population flag POPFL from the
safety flag SAFFL and the intent-to-treat flag ITTFL, plus
the dates and age they read: first exposure date TRTSDT,
randomization date RANDDT, collected age AGE, age band
AGEGR1, and age rank AGERNK.
Input: one demographic record per subject carrying age and the randomization date, and exposure records carrying the exposure start date.
Variables:
POPFLholdsYonly when bothSAFFLandITTFLholdY, andNotherwise, marking subjects in both populations.SAFFLholdsYwhen the subject has a first exposure date, andNotherwise, marking subjects who started treatment.ITTFLholdsYwhen the subject was randomized, andNotherwise, marking subjects in the intent-to-treat population.TRTSDTholds the earliest dated exposure start for the subject, and stays empty when no exposure record carries a date.RANDDTholds the collected randomization date, and stays empty when the subject was not randomized.AGEGR1holds<65whenAGEis below 65 and>=65when it is 65 or above; a subject with no age stops the run.AGERNKholds the subject rank byAGEwithin the study, youngest first, withUSUBJIDbreaking ties.
Note: the dates and age are settled before the flags that read them: the safety and intent-to-treat flags come from their dates, and the combined flag comes from both flags.
Inputs
Under review2 source filesinput/dm.csv
Edit7 rows| STUDYID | USUBJID | AGE | RANDDT |
|---|---|---|---|
| YAMAA-01 | 34 | 2025-01-05 | |
| YAMAA-01 | 65 | 2025-02-05 | |
| YAMAA-01 | 52 | — | |
| YAMAA-01 | 65 | 2025-02-20 | |
| YAMAA-01 | 41 | — | |
| YAMAA-01 | 88 | 2025-03-01 | |
| YAMAA-01 | 29 | 2025-04-01 |
input/ex.csv
Edit7 rows| STUDYID | USUBJID | EXSEQ | EXSTDTC |
|---|---|---|---|
| YAMAA-01 | 1 | 2025-01-08 | |
| YAMAA-01 | 2 | 2025-01-15 | |
| YAMAA-01 | 3 | — | |
| YAMAA-01 | 1 | 2025-02-08 | |
| YAMAA-01 | 1 | 2025-02-28 | |
| YAMAA-01 | 1 | 2025-03-05 | |
| YAMAA-01 | 1 | — |
Expected output
Under review7 expected rowsexpected/adsl.csv
Edit7 rows| POPFL | SAFFL | ITTFL | TRTSDT | RANDDT | AGEGR1 | AGERNK | STUDYID | USUBJID | AGE |
|---|---|---|---|---|---|---|---|---|---|
| Y | Y | Y | 2025-01-08 | 2025-01-05 | <65 | 2 | YAMAA-01 | 34 | |
| Y | Y | Y | 2025-02-08 | 2025-02-05 | >=65 | 5 | YAMAA-01 | 65 | |
| N | N | N | — | — | <65 | 4 | YAMAA-01 | 52 | |
| N | N | Y | — | 2025-02-20 | >=65 | 6 | YAMAA-01 | 65 | |
| N | Y | N | 2025-02-28 | — | <65 | 3 | YAMAA-01 | 41 | |
| Y | Y | Y | 2025-03-05 | 2025-03-01 | >=65 | 7 | YAMAA-01 | 88 | |
| N | N | Y | — | 2025-04-01 | <65 | 1 | YAMAA-01 | 29 |
Comments and assessment materials
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADSL | TRTSDT | Date of First Exposure to Treatment | numeric | ISO 8601 | Aggregate MIN(EX.EXSTDTC) over records matching on the shared output keys. | Derived | |
| ADSL | SAFFL | Safety Population Flag | character | "Y" when TRTSDT IS NOT NULL; "N" otherwise; "N" when the condition is unknown. | Derived | ||
| ADSL | RANDDT | Randomization Date | numeric | ISO 8601 | Copy value from DM.RANDDT. | Predecessor | |
| ADSL | ITTFL | Intent-to-Treat Population Flag | character | "Y" when RANDFL = 'Y'; "N" otherwise; "N" when the condition is unknown. | Derived | ||
| ADSL | POPFL | Analysis Population Flag | character | "Y" when SAFFL = 'Y' AND ITTFL = 'Y'; "N" otherwise; "N" when the condition is unknown. | Derived | ||
| ADSL | STUDYID | Study Identifier | character | Copy value from DM.STUDYID. | Predecessor | ||
| ADSL | USUBJID | Unique Subject Identifier | character | Copy value from DM.USUBJID. | Predecessor | ||
| ADSL | AGE | Age | numeric | Copy value from DM.AGE. | Predecessor | ||
| ADSL | AGEGR1 | Pooled Age Group 1 | character | Band AGE at 65 into "<65" and ">=65", intervals closed on the left. | Derived | ||
| ADSL | AGERNK | Pooled Age Group 1 Rank | numeric | Row number within each group of STUDYID, ordered by AGE then USUBJID. | Derived |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adsl-flag-chain/full.md
Edit23 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: POPFL, SAFFL, ITTFL, TRTSDT, RANDDT, AGEGR1, AGERNK, STUDYID, USUBJID, AGE
TRTSDT is the earliest dated exposure start for the subject, and has no value when no exposure record carries a date. RANDDT is the collected randomization date, and has no value when the subject was not randomized.
SAFFL is Y when the subject has a first exposure date and N otherwise. ITTFL is Y when the subject was randomized and N otherwise. POPFL is Y only when both SAFFL and ITTFL hold Y, and N otherwise.
AGEGR1 holds <65 when AGE is below 65 and >=65 when it is 65 or above. AGERNK holds the subject rank by AGE within the study, youngest first, with USUBJID breaking ties.
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
prompts/adam-adsl-flag-chain/conventions.md
Edit14 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: POPFL, SAFFL, ITTFL, TRTSDT, RANDDT, AGEGR1, AGERNK, STUDYID, USUBJID, AGE
SAFFL, ITTFL, and POPFL are each Y or N.
AGEGR1 holds <65 or >=65. AGERNK holds the subject rank by AGE within the study, youngest first, with USUBJID breaking ties.
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
prompts/adam-adsl-flag-chain/brief.md
Edit9 linesFollowing CDISC ADaM standards, use the provided DM and EX datasets to create an ADSL dataset with one record per subject.
The output dataset should contain the following columns in this order: POPFL, SAFFL, ITTFL, TRTSDT, RANDDT, AGEGR1, AGERNK, STUDYID, USUBJID, AGE
Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.
run.R
Edit4 lineslibrary(yamaa)adsl <- yamaa_domain("spec.yaml")$outputadslrun.py
Edit4 linesimport yamaaadsl = yamaa.yamaa_domain("spec.yaml").outputadslsolutions/adam-adsl-flag-chain/result.R
Edit47 lines# Reference solution for the yamaa benchmark adam-adsl-flag-chain (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)dm <- read_csv( "/app/input/dm.csv", col_types = cols(AGE = col_integer(), RANDDT = col_date(), .default = col_character()))ex <- read_csv( "/app/input/ex.csv", col_types = cols(EXSEQ = col_integer(), EXSTDTC = col_date(), .default = col_character()))first_exposure <- ex |> group_by(STUDYID, USUBJID) |> summarise( TRTSDT = if (all(is.na(EXSTDTC))) as.Date(NA) else min(EXSTDTC, na.rm = TRUE), .groups = "drop" )adsl <- dm |> left_join(first_exposure, by = c("STUDYID", "USUBJID")) |> mutate( SAFFL = if_else(!is.na(TRTSDT), "Y", "N"), RANDFL = if_else(!is.na(RANDDT), "Y", "N"), ITTFL = if_else(RANDFL == "Y", "Y", "N"), POPFL = if_else(SAFFL == "Y" & ITTFL == "Y", "Y", "N"), AGEGR1 = if_else(AGE < 65, "<65", ">=65") ) |> mutate(.idx = row_number()) |> arrange(STUDYID, AGE, USUBJID) |> group_by(STUDYID) |> mutate(AGERNK = row_number()) |> ungroup() |> arrange(.idx) |> select( POPFL, SAFFL, ITTFL, TRTSDT, RANDDT, AGEGR1, AGERNK, STUDYID, USUBJID, AGE )dir.create("/app/output", showWarnings = FALSE)write_csv(adsl, "/app/output/adsl.csv", na = "")solutions/adam-adsl-flag-chain/result.py
Edit72 lines# Reference solution for the yamaa benchmark adam-adsl-flag-chain (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pldm = pl.read_csv("/app/input/dm.csv", infer_schema=False).with_columns( pl.col("RANDDT").str.to_date(strict=False), pl.col("AGE").cast(pl.Int64, strict=False),)ex = pl.read_csv("/app/input/ex.csv", infer_schema=False).with_columns( pl.col("EXSTDTC").str.to_date(strict=False),)first_exposure = ex.group_by(["STUDYID", "USUBJID"]).agg( TRTSDT=pl.col("EXSTDTC").min())adsl = ( dm.join(first_exposure, on=["STUDYID", "USUBJID"], how="left", maintain_order="left") .with_columns( SAFFL=pl.when(pl.col("TRTSDT").is_not_null()) .then(pl.lit("Y")) .otherwise(pl.lit("N")), RANDFL=pl.when(pl.col("RANDDT").is_not_null()) .then(pl.lit("Y")) .otherwise(pl.lit("N")), ) .with_columns( ITTFL=pl.when(pl.col("RANDFL") == "Y") .then(pl.lit("Y")) .otherwise(pl.lit("N")), ) .with_columns( POPFL=pl.when((pl.col("SAFFL") == "Y") & (pl.col("ITTFL") == "Y")) .then(pl.lit("Y")) .otherwise(pl.lit("N")), AGEGR1=pl.when(pl.col("AGE") < 65) .then(pl.lit("<65")) .when(pl.col("AGE") >= 65) .then(pl.lit(">=65")) .otherwise(None), ))# Rank by age within the study, youngest first, USUBJID breaking ties.ranked = ( adsl.with_columns(__idx=pl.int_range(pl.len())) .sort(["STUDYID", "AGE", "USUBJID"], nulls_last=True) .with_columns(AGERNK=pl.int_range(1, pl.len() + 1).over("STUDYID")) .sort("__idx") .drop("__idx") .select( "POPFL", "SAFFL", "ITTFL", "TRTSDT", "RANDDT", "AGEGR1", "AGERNK", "STUDYID", "USUBJID", "AGE", ))Path("/app/output").mkdir(exist_ok=True)ranked.write_csv("/app/output/adsl.csv")solutions/adam-adsl-flag-chain/result.sas
Edit39 lines/* opensas reference solution for the yamaa benchmark adam-adsl-flag-chain. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dm; length STUDYID USUBJID RANDDT $1024; infile "/app/input/dm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. AGE RANDDT : $char1024.;run;data ex; length STUDYID USUBJID EXSTDTC $1024; infile "/app/input/ex.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. EXSEQ EXSTDTC : $char1024.;run;proc sql; create table firstex as select STUDYID,USUBJID,min(EXSTDTC) as TRTSDT from ex where not missing(EXSTDTC) group by STUDYID,USUBJID; create table joined as select a.*,b.TRTSDT from dm as a left join firstex as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID;quit;data ranked; set joined; NOAGE=missing(AGE); run;proc sort data=ranked; by STUDYID NOAGE AGE USUBJID; run;data result; length POPFL SAFFL ITTFL $1 AGEGR1 $4; set ranked; by STUDYID; if first.STUDYID then AGERNK=0; AGERNK+1; SAFFL='N'; ITTFL='N'; POPFL='N'; if not missing(TRTSDT) then SAFFL='Y'; if not missing(RANDDT) then ITTFL='Y'; if SAFFL='Y' and ITTFL='Y' then POPFL='Y'; if not missing(AGE) then do; if AGE<65 then AGEGR1='<65'; else AGEGR1='>=65'; end;run;proc sql; create table final as select POPFL, SAFFL, ITTFL, TRTSDT, RANDDT, AGEGR1, AGERNK, STUDYID, USUBJID, AGE from result;quit;proc export data=final outfile="/app/output/adsl.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions