As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: derive the confirmation flag CONFIRMED for tumor
response assessments.
Input: response assessments within a subject, each with a
sequence number RSSEQ, a collection date RSDTC, and a
standardized result RSSTRESC.
Variables:
CONFIRMED:Ywhen the analysis resultAVALCis progressive disease (PD), which needs no confirmation, or when a partial (PR) or complete (CR) response is followed at least 28 days later by another partial or complete response;Nwhen the current assessment has no result, or when the next response is too early, is not a response, or does not exist. Each assessment is compared with the next one in analysis dateADTorder within a subject.
Note: assessments sharing a date are ordered by sequence
number RSSEQ: the higher-numbered one is the next assessment for
the lower-numbered one, zero days later, so it never confirms it.
Inputs
Under review1 source fileinput/rs.csv
Edit14 rows| STUDYID | USUBJID | RSSEQ | RSDTC | RSSTRESC |
|---|---|---|---|---|
| PILOT7 | 1 | 2025-01-01 | PR | |
| PILOT7 | 2 | 2025-02-12 | PR | |
| PILOT7 | 1 | 2025-01-01 | PR | |
| PILOT7 | 2 | 2025-02-12 | SD | |
| PILOT7 | 1 | 2025-01-01 | PR | |
| PILOT7 | 2 | 2025-01-15 | PR | |
| PILOT7 | 1 | 2025-01-01 | PR | |
| PILOT7 | 1 | 2025-01-01 | PD | |
| PILOT7 | 2 | 2025-02-12 | NE | |
| PILOT7 | 1 | 2025-04-01 | PR | |
| PILOT7 | 2 | 2025-04-01 | PR | |
| PILOT7 | 1 | 2025-01-01 | PR | |
| PILOT7 | 2 | 2025-01-29 | CR | |
| PILOT7 | 1 | 2025-05-06 | — |
Expected output
Under review14 expected rowsexpected/adrs.csv
Edit14 rows| STUDYID | USUBJID | PARAMCD | RSSEQ | ADT | AVALC | CONFIRMED |
|---|---|---|---|---|---|---|
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PR | Y | |
| PILOT7 | CONFRESP | 2 | 2025-02-12 | PR | N | |
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PR | N | |
| PILOT7 | CONFRESP | 2 | 2025-02-12 | SD | N | |
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PR | N | |
| PILOT7 | CONFRESP | 2 | 2025-01-15 | PR | N | |
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PR | N | |
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PD | Y | |
| PILOT7 | CONFRESP | 2 | 2025-02-12 | NE | N | |
| PILOT7 | CONFRESP | 1 | 2025-04-01 | PR | N | |
| PILOT7 | CONFRESP | 2 | 2025-04-01 | PR | N | |
| PILOT7 | CONFRESP | 1 | 2025-01-01 | PR | Y | |
| PILOT7 | CONFRESP | 2 | 2025-01-29 | CR | N | |
| PILOT7 | CONFRESP | 1 | 2025-05-06 | — | N |
Comments and assessment materials
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADRS | STUDYID | Study Identifier | character | Copy value from RS.STUDYID. | Predecessor | ||
| ADRS | USUBJID | Unique Subject Identifier | character | Copy value from RS.USUBJID. | Predecessor | ||
| ADRS | PARAMCD | Parameter Code | character | Set constant value "CONFRESP". | Assigned | ||
| ADRS | RSSEQ | Sequence Number | numeric | Copy value from RS.RSSEQ. | Predecessor | ||
| ADRS | ADT | Analysis Date | numeric | ISO 8601 | Copy value from RS.RSDTC. | Predecessor | |
| ADRS | AVALC | Analysis Value (C) | character | Copy value from RS.RSSTRESC. | Predecessor | ||
| ADRS | CONFIRMED | Confirmation Flag | character | If AVALC = 'PD' then "Y"; If AVALC IN ('PR', 'CR') AND NEXTRESP IN ('PR', 'CR') AND DAYS_TO_NEXT >= 28 then "Y"; otherwise "N". | Derived |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adrs-confirmed-response/full.md
Edit23 linesFollowing CDISC ADaM standards, use the provided RS dataset to create an ADRS dataset with one record per response assessment.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, RSSEQ, ADT, AVALC, CONFIRMED
Use PARAMCD "CONFRESP", where ADT is the collection date and AVALC carries the standardized result.
CONFIRMED is Y when the result is progressive disease (PD), which needs no confirmation, or when a partial (PR) or complete (CR) response is followed at least 28 days later by another partial or complete response. It is N when the current assessment has no result, or when the next response is too early, is not a response, or does not exist. Each assessment is compared with the next one in analysis date order within a subject.
Assessments sharing a date are ordered by sequence number: the higher-numbered one is the next assessment for the lower-numbered one, zero days later, so it never confirms it.
Read the source datasets from /app/input and save the completed dataset as /app/output/adrs.csv.
prompts/adam-adrs-confirmed-response/conventions.md
Edit10 linesFollowing CDISC ADaM standards, use the provided RS dataset to create an ADRS dataset with one record per response assessment.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, RSSEQ, ADT, AVALC, CONFIRMED
Use PARAMCD "CONFRESP". CONFIRMED is Y or N.
Read the source datasets from /app/input and save the completed dataset as /app/output/adrs.csv.
prompts/adam-adrs-confirmed-response/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided RS dataset to create an ADRS dataset with one record per response assessment.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, RSSEQ, ADT, AVALC, CONFIRMED
Read the source datasets from /app/input and save the completed dataset as /app/output/adrs.csv.
run.R
Edit4 lineslibrary(yamaa)adrs <- yamaa_domain("spec.yaml")$outputadrsrun.py
Edit4 linesimport yamaaadrs = yamaa.yamaa_domain("spec.yaml").outputadrssolutions/adam-adrs-confirmed-response/result.R
Edit43 lines# Reference solution for the yamaa benchmark adam-adrs-confirmed-response (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)rs <- read_csv( "/app/input/rs.csv", col_types = cols( RSSEQ = col_integer(), RSDTC = col_date(), .default = col_character() ))# Each assessment is compared with the next one in analysis date order;# assessments sharing a date are ordered by sequence number, so the# higher-numbered one is zero days later and never confirms the lower.ordered <- rs |> mutate(PARAMCD = "CONFRESP", ADT = RSDTC, AVALC = RSSTRESC) |> arrange(STUDYID, USUBJID, ADT, RSSEQ) |> group_by(STUDYID, USUBJID) |> mutate( NEXT_AVALC = lead(AVALC), NEXT_ADT = lead(ADT), DAYS = as.integer(NEXT_ADT - ADT), CONFIRMED = case_when( AVALC %in% "PD" ~ "Y", AVALC %in% c("PR", "CR") & NEXT_AVALC %in% c("PR", "CR") & !is.na(DAYS) & DAYS >= 28L ~ "Y", .default = "N" ) ) |> ungroup() |> select(STUDYID, USUBJID, PARAMCD, RSSEQ, ADT, AVALC, CONFIRMED) |> arrange(USUBJID, ADT, RSSEQ)dir.create("/app/output", showWarnings = FALSE)write_csv(ordered, "/app/output/adrs.csv", na = "")solutions/adam-adrs-confirmed-response/result.py
Edit45 lines# Reference solution for the yamaa benchmark adam-adrs-confirmed-response (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plrs = pl.read_csv("/app/input/rs.csv", infer_schema=False).with_columns( pl.col("RSSEQ").cast(pl.Int64), pl.col("RSDTC").str.to_date(),)# Each assessment is compared with the next one in analysis date order;# assessments sharing a date are ordered by sequence number, so the# higher-numbered one is zero days later and never confirms the lower.ordered = ( rs.with_columns( PARAMCD=pl.lit("CONFRESP"), ADT=pl.col("RSDTC"), AVALC=pl.col("RSSTRESC") ) .sort(["STUDYID", "USUBJID", "ADT", "RSSEQ"]) .with_columns( NEXT_AVALC=pl.col("AVALC").shift(-1).over(["STUDYID", "USUBJID"]), NEXT_ADT=pl.col("ADT").shift(-1).over(["STUDYID", "USUBJID"]), ) .with_columns(DAYS=(pl.col("NEXT_ADT") - pl.col("ADT")).dt.total_days()) .with_columns( CONFIRMED=pl.when(pl.col("AVALC") == "PD") .then(pl.lit("Y")) .when( pl.col("AVALC").is_in(["PR", "CR"]) & pl.col("NEXT_AVALC").is_in(["PR", "CR"]) & (pl.col("DAYS") >= 28) ) .then(pl.lit("Y")) .otherwise(pl.lit("N")) ) .select("STUDYID", "USUBJID", "PARAMCD", "RSSEQ", "ADT", "AVALC", "CONFIRMED") .sort(["USUBJID", "ADT", "RSSEQ"]))Path("/app/output").mkdir(exist_ok=True)ordered.write_csv("/app/output/adrs.csv")solutions/adam-adrs-confirmed-response/result.sas
Edit26 lines/* opensas reference solution for the yamaa benchmark adam-adrs-confirmed-response. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data rs; length STUDYID USUBJID RSDTC RSSTRESC $1024; infile "/app/input/rs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. RSSEQ RSDTC : $char1024. RSSTRESC : $char1024.;run;proc sort data=rs out=ordered; by STUDYID USUBJID descending RSDTC descending RSSEQ; run;data result; length PARAMCD $16 ADT $10 AVALC $1024 CONFIRMED $1 nextresult $1024 nextdate $10; retain nextresult nextdate; set ordered; by STUDYID USUBJID; if first.USUBJID then do; nextresult=''; nextdate=''; end; PARAMCD='CONFRESP'; ADT=RSDTC; AVALC=RSSTRESC; CONFIRMED='N'; if AVALC='PD' then CONFIRMED='Y'; else if AVALC in ('PR','CR') and nextresult in ('PR','CR') and not missing(ADT) and not missing(nextdate) and input(nextdate,yymmdd10.)-input(ADT,yymmdd10.)>=28 then CONFIRMED='Y'; nextresult=AVALC; nextdate=ADT;run;proc sql; create table final as select STUDYID, USUBJID, PARAMCD, RSSEQ, ADT, AVALC, CONFIRMED from result;quit;proc export data=final outfile="/app/output/adrs.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions