Time to Deterioration for a PRO Endpoint

Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Goal: flag each visit-level patient-reported outcome (PRO) score as a deterioration or not, then build one time-to-deterioration record per subject with parameter code TTDGHS (Time to Deterioration in Global Health Status), carrying STARTDT, ADT, AVAL, CNSR, EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, and SRCSEQ.

Input: subject-level randomization and death dates (RANDDT, DTHDT), visit-level PRO scores on a 0 to 100 scale, each with a sequence number (QSSEQ), an assessment date (ADT), and a score (AVAL), and disposition records naming disease progression, study discontinuation, or withdrawal of consent (DSDECOD, DSDTC).

Variables:

DETERFL is Y on an assessment dated after the subject's earliest assessment whose score has fallen at least 10 points below that earliest (baseline) score; blank otherwise, including on the baseline assessment itself. It is computed only to find the earliest flagged assessment per subject and is not an output column.
STARTDT is the randomization date, the origin from which the time to deterioration is counted.
ADT is the analysis date: the deterioration date when a flagged assessment exists and is not dated after the first censoring reason, else the death date when the subject died and the death is not dated after the first censoring reason. A deterioration and a death on the same date count as deterioration. With neither, ADT is the last assessment dated on or before the first censoring reason, or randomization when no assessment qualifies.
AVAL is the number of whole calendar months from STARTDT to ADT.
CNSR is 0 when the subject has a deterioration or death event and 1 otherwise.
EVNTDESC is PRO DETERIORATION for a deterioration event, DEATH for a death event, and CENSORED otherwise.
CNSDTDSC is blank for an event, holds the censoring reason (DISEASE PROGRESSION, STUDY DISCONTINUATION, or WITHDRAWAL OF CONSENT) when one is available, and is STUDY COMPLETION otherwise.
SRCDOM is QS for a deterioration or a censored record with a supporting assessment, and ADSL for a death or a record censored at randomization.
SRCVAR is ADT for a deterioration or a censored record with a supporting assessment, DTHDT for a death, and RANDDT for a record censored at randomization.
SRCSEQ is the sequence number from the deterioration or censoring assessment, and blank when the analysis date comes from a subject-level date.

Note: progression, discontinuation, and withdrawal are censoring reasons, never events: a subject who progresses before any deterioration is censored at the last assessment on or before the progression date, and a deterioration dated the same day as the progression still counts as the event.

YAML specification

1 spec file

spec.yaml

Edit264 lines

1schema_version: "1.0"2domain: ADTTE3keys: [STUDYID, USUBJID, PARAMCD]45# Single-spec benchmark: the deterioration flag is derived in the6# intermediates below (BASELINE/FIRSTDETER), not in a separate producer7# spec. The flag is a derivation step of the ADTTE build, the same way8# adam-adtte-pfs derives its progression picks in intermediates.9input:10  ADSL: {path: input/adsl.csv, types: {RANDDT: date, DTHDT: date}}11  QS:12    path: input/qs.csv13    types: {QSSEQ: int, ADT: date, AVAL: int}14  DS:15    path: input/ds.csv16    types: {DSDTC: date, DSSEQ: int}1718base: ADSL1920output:21  path: adtte.csv22  columns: [STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR,23            EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ]2425intermediates:26  # The baseline assessment: the earliest dated assessment per subject.27  # The deterioration search below runs strictly after it, so the baseline28  # record itself can never flag even though its own change from baseline29  # is zero.30  - id: BASELINE31    dataset: QS32    key: [STUDYID, USUBJID]33    filter: "QS.ADT IS NOT NULL"34    order_by: [QS.ADT, QS.QSSEQ]35    keep: first36    no_match: null3738  # The first deteriorated assessment: dated after the baseline assessment39  # with a score at least 10 points below the baseline score. The cutoff is40  # read from BASELINE per donor record, so the flag logic lives here41  # instead of in a separate flagged dataset. Ordered by date, then42  # sequence, so one record wins whatever the input order.43  - id: FIRSTDETER44    dataset: QS45    key: [STUDYID, USUBJID]46    filter: "QS.ADT IS NOT NULL AND QS.ADT > QS.BASELINEDT47      AND QS.AVAL <= QS.CUTOFF"48    order_by: [QS.ADT, QS.QSSEQ]49    keep: first50    no_match: null51    derivations:52      BASELINEDT: BASELINE.ADT53      CUTOFF: {compute: {expr: "BASELINE.AVAL - 10"}}5455  # The first censoring reason: progression, discontinuation, or56  # withdrawal. Only the earliest counts; later ones do not move the57  # censor date. A DS record naming anything else is not a censoring58  # reason and is ignored here.59  - id: FIRSTDS60    dataset: DS61    key: [STUDYID, USUBJID]62    filter: "DS.DSDECOD IN ('DISEASE PROGRESSION',63      'STUDY DISCONTINUATION', 'WITHDRAWAL OF CONSENT')"64    order_by: [DS.DSDTC, DS.DSSEQ]65    keep: first66    no_match: null6768columns:69  - name: STUDYID70    type: str71    label: Study Identifier72    derivation: ADSL.STUDYID7374  - name: USUBJID75    type: str76    label: Unique Subject Identifier77    derivation: ADSL.USUBJID7879  - name: PARAMCD80    type: str81    label: Parameter Code82    derivation: {literal: TTDGHS}8384  - name: PARAM85    type: str86    label: Parameter87    derivation: {literal: Time to Deterioration in Global Health Status}8889  - name: STARTDT90    type: date91    label: Time-to-Event Origin Date92    derivation: ADSL.RANDDT9394  - name: DETERDT95    type: date96    label: Deterioration Date97    derivation: FIRSTDETER.ADT9899  - name: DEATHDT100    type: date101    label: Death Date102    derivation: ADSL.DTHDT103104  # The censoring reason's date and description, when a reason was105  # recorded; empty for subjects with no reason on file. Declared before106  # EVENTDT because the event logic below reads the reason date.107  - name: CENSORRSNDT108    type: date109    label: Censoring Reason Date110    derivation: FIRSTDS.DSDTC111112  - name: CENSORRSN113    type: str114    label: Censoring Reason115    derivation: FIRSTDS.DSDECOD116117  # The event is the earlier of deterioration and death; a deterioration118  # and a death on the same date count as deterioration. Neither counts119  # when it falls after the first censoring reason: progression,120  # discontinuation, and withdrawal are censoring reasons, never events.121  # A deterioration or death dated the same day as the censoring reason122  # still counts as the event.123  - name: EVENTDT124    type: date125    label: Event Date126    derivation:127      case:128        - when: "DETERDT IS NOT NULL AND (DEATHDT IS NULL129            OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL130            OR DETERDT <= CENSORRSNDT)"131          then: DETERDT132        - when: "DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL133            OR DEATHDT <= CENSORRSNDT)"134          then: DEATHDT135136  # The last assessment dated on or before the censoring reason. A137  # missing reason date admits no assessment, so the match is empty then138  # and the subject falls back to randomization below.139  - name: LASTQSLE140    type: date141    label: Last Assessment On Or Before Censoring142    derivation:143      aggregate:144        filter: "QS.ADT IS NOT NULL"145        group_by: [QS.STUDYID, QS.USUBJID]146        between: {value: CENSORRSNDT, lower: ADT}147        expr: "MAX(QS.ADT)"148149  # The sequence of that assessment: the latest sequence among the150  # records sharing the censoring date.151  - name: LASTQSSEQ152    type: int153    label: Sequence of Last Assessment On Or Before Censoring154    derivation:155      aggregate:156        filter: "QS.ADT IS NOT NULL"157        group_by: [QS.STUDYID, QS.USUBJID]158        between: {value: LASTQSLE, lower: ADT, upper: ADT}159        expr: "MAX(QS.QSSEQ)"160161  # With no usable assessment the subject is censored at randomization.162  - name: CENSORDT163    type: date164    label: Censoring Date165    derivation:166      first_available:167        sources: [LASTQSLE, STARTDT]168169  - name: ADT170    type: date171    label: Analysis Date172    derivation:173      first_available:174        sources: [EVENTDT, CENSORDT]175176  - name: AVAL177    type: int178    label: Analysis Value179    derivation:180      date_diff:181        start: STARTDT182        end: ADT183        unit: month184185  - name: CNSR186    type: int187    label: Censor188    derivation:189      case:190        - when: "EVENTDT IS NOT NULL"191          then: {literal: 0}192        - otherwise: {literal: 1}193194  # A deterioration and a censoring reason on the same date count as a195  # deterioration: the event always wins over censoring, but only when the196  # event is not dated after the censoring reason.197  - name: EVNTDESC198    type: str199    label: Event or Censoring Description200    derivation:201      case:202        - when: "DETERDT IS NOT NULL AND (DEATHDT IS NULL203            OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL204            OR DETERDT <= CENSORRSNDT)"205          then: {literal: PRO DETERIORATION}206        - when: "DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL207            OR DEATHDT <= CENSORRSNDT)"208          then: {literal: DEATH}209        - otherwise: {literal: CENSORED}210211  - name: CNSDTDSC212    type: str213    label: Censoring Date Description214    derivation:215      case:216        - when: "EVENTDT IS NOT NULL"217          then: {literal: null}218        - when: "CENSORRSN IS NOT NULL"219          then: CENSORRSN220        - otherwise: {literal: STUDY COMPLETION}221222  - name: SRCDOM223    type: str224    label: Source Data225    derivation:226      case:227        - when: "DETERDT IS NOT NULL AND (DEATHDT IS NULL228            OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL229            OR DETERDT <= CENSORRSNDT)"230          then: {literal: QS}231        - when: "DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL232            OR DEATHDT <= CENSORRSNDT)"233          then: {literal: ADSL}234        - when: "LASTQSLE IS NULL"235          then: {literal: ADSL}236        - otherwise: {literal: QS}237238  - name: SRCVAR239    type: str240    label: Source Variable241    derivation:242      case:243        - when: "DETERDT IS NOT NULL AND (DEATHDT IS NULL244            OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL245            OR DETERDT <= CENSORRSNDT)"246          then: {literal: ADT}247        - when: "DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL248            OR DEATHDT <= CENSORRSNDT)"249          then: {literal: DTHDT}250        - when: "LASTQSLE IS NULL"251          then: {literal: RANDDT}252        - otherwise: {literal: ADT}253254  - name: SRCSEQ255    type: int256    label: Source Sequence Number257    derivation:258      case:259        - when: "DETERDT IS NOT NULL AND (DEATHDT IS NULL260            OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL261            OR DETERDT <= CENSORRSNDT)"262          then: FIRSTDETER.QSSEQ263        - when: "EVNTDESC = 'CENSORED' AND LASTQSLE IS NOT NULL"264          then: LASTQSSEQ

Inputs

Under review3 source files

input/adsl.csv

Edit6 rows

input/adsl.csv
STUDYID	RANDDT	DTHDT
YAMAA-01	2026-01-15	2026-08-20
YAMAA-01	2026-01-15	2026-06-10
YAMAA-01	2026-01-15	—
YAMAA-01	2026-01-15	—
YAMAA-01	2026-01-15	—
YAMAA-01	2026-01-15	—

input/ds.csv

Edit4 rows

input/ds.csv
STUDYID	DSDECOD	DSDTC	DSSEQ
YAMAA-01	DISEASE PROGRESSION	2026-04-01	1
YAMAA-01	WITHDRAWAL OF CONSENT	2026-06-01	1
YAMAA-01	STUDY DISCONTINUATION	2026-01-20	1
YAMAA-01	DISEASE PROGRESSION	2026-03-12	1

input/qs.csv

Edit14 rows

input/qs.csv
STUDYID	QSSEQ	AVISIT	ADT	AVAL
YAMAA-01	1	BASELINE	2026-01-15	80
YAMAA-01	2	WEEK 8	2026-03-12	74
YAMAA-01	3	WEEK 16	2026-05-07	60
YAMAA-01	1	BASELINE	2026-01-15	70
YAMAA-01	2	WEEK 8	2026-03-12	68
YAMAA-01	3	WEEK 16	2026-05-07	65
YAMAA-01	1	BASELINE	2026-01-15	75
YAMAA-01	2	WEEK 8	2026-03-12	72
YAMAA-01	1	BASELINE	2026-01-15	80
YAMAA-01	2	WEEK 8	2026-03-12	78
YAMAA-01	3	WEEK 16	2026-05-07	76
YAMAA-01	1	BASELINE	2026-01-15	70
YAMAA-01	1	BASELINE	2026-01-15	85
YAMAA-01	2	WEEK 8	2026-03-12	70

Expected output

Under review6 expected rows

expected/adtte.csv

Edit6 rows

expected/adtte.csv
STUDYID	PARAMCD	PARAM	STARTDT	ADT	AVAL	CNSR	EVNTDESC	CNSDTDSC	SRCDOM	SRCVAR	SRCSEQ
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-05-07	3	0	PRO DETERIORATION	—	QS	ADT	3
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-06-10	4	0	DEATH	—	ADSL	DTHDT	—
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-03-12	1	1	CENSORED	DISEASE PROGRESSION	QS	ADT	2
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-05-07	3	1	CENSORED	WITHDRAWAL OF CONSENT	QS	ADT	3
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-01-15	0	1	CENSORED	STUDY DISCONTINUATION	QS	ADT	1
YAMAA-01	TTDGHS	Time to Deterioration in Global Health Status	2026-01-15	2026-03-12	1	0	PRO DETERIORATION	—	QS	ADT	2

Shaded columns are derived— = missing value

Comments and assessment materials

Dataset Name	Variable Name	Variable Label	Type	Codelist/Controlled Terms	Computational Method	Origin
ADTTE	STUDYID	Study Identifier	character		Copy value from ADSL.STUDYID.	Predecessor
ADTTE	USUBJID	Unique Subject Identifier	character		Copy value from ADSL.USUBJID.	Predecessor
ADTTE	PARAMCD	Parameter Code	character		Set constant value "TTDGHS".	Assigned
ADTTE	PARAM	Parameter	character		Set constant value "Time to Deterioration in Global Health Status".	Assigned
ADTTE	STARTDT	Time-to-Event Origin Date	numeric	ISO 8601	Copy value from ADSL.RANDDT.	Predecessor
ADTTE	ADT	Analysis Date	numeric	ISO 8601	First non-missing of EVENTDT and CENSORDT.	Derived
ADTTE	AVAL	Analysis Value	numeric		Whole months from STARTDT to ADT, counting the end date but not the start.	Derived
ADTTE	CNSR	Censor	numeric		If EVENTDT IS NOT NULL then "0"; otherwise "1".	Derived
ADTTE	EVNTDESC	Event or Censoring Description	character		If DETERDT IS NOT NULL AND (DEATHDT IS NULL OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL OR DETERDT <= CENSORRSNDT) then "PRO DETERIORATION"; If DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL OR DEATHDT <= CENSORRSNDT) then "DEATH"; otherwise "CENSORED".	Derived
ADTTE	CNSDTDSC	Censoring Date Description	character		If EVENTDT IS NOT NULL then "None"; If CENSORRSN IS NOT NULL then "CENSORRSN"; otherwise "STUDY COMPLETION".	Derived
ADTTE	SRCDOM	Source Data	character		If DETERDT IS NOT NULL AND (DEATHDT IS NULL OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL OR DETERDT <= CENSORRSNDT) then "QS"; If DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL OR DEATHDT <= CENSORRSNDT) then "ADSL"; If LASTQSLE IS NULL then "ADSL"; otherwise "QS".	Derived
ADTTE	SRCVAR	Source Variable	character		If DETERDT IS NOT NULL AND (DEATHDT IS NULL OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL OR DETERDT <= CENSORRSNDT) then "ADT"; If DEATHDT IS NOT NULL AND (CENSORRSNDT IS NULL OR DEATHDT <= CENSORRSNDT) then "DTHDT"; If LASTQSLE IS NULL then "RANDDT"; otherwise "ADT".	Derived
ADTTE	SRCSEQ	Source Sequence Number	numeric		If DETERDT IS NOT NULL AND (DEATHDT IS NULL OR DETERDT <= DEATHDT) AND (CENSORRSNDT IS NULL OR DETERDT <= CENSORRSNDT) then "FIRSTDETER.QSSEQ"; If EVNTDESC = 'CENSORED' AND LASTQSLE IS NOT NULL then "LASTQSSEQ"; otherwise blank.	Derived

Version	Date	Author	Description
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/adam-adtte-pro-deterioration/full.md

Edit44 lines

Following CDISC ADaM standards, use the provided ADSL, QS, and DS datasets to create an ADTTE dataset for time to PRO deterioration, one record per subject.

The ADTTE dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR, EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ.

Use PARAMCD "TTDGHS" and PARAM "Time to Deterioration in Global Health Status". STARTDT is the randomization date, the origin from which the time to deterioration is counted. QSSEQ is the input sequence number, as given.

The baseline score is the score from the earliest assessment. An assessment is a deterioration when it is dated after the baseline assessment and its score is at least 10 points below the baseline score; the baseline assessment itself is never a deterioration.

The event is the first deterioration, or death when the subject dies without a prior deterioration; a deterioration and a death on the same date count as deterioration. Neither counts when it falls after the first censoring reason. ADT is the deterioration date for a deterioration event and the death date for a death event.

Disease progression, study discontinuation, and withdrawal of consent are censoring reasons, never events; only the earliest such record counts. A censored subject's ADT is the latest assessment dated on or before the censoring reason, or the randomization date when no assessment qualifies. A deterioration dated the same day as the censoring reason still counts as the event.

AVAL is the number of whole calendar months from STARTDT to ADT. CNSR is 0 for a deterioration or death and 1 otherwise. EVNTDESC is PRO DETERIORATION for a deterioration, DEATH for a death, and CENSORED otherwise. CNSDTDSC has no value for an event and holds the censoring reason for a censored record, or STUDY COMPLETION when no DS record names a reason.

Trace ADT with SRCDOM QS, SRCVAR ADT, and the assessment sequence for a deterioration; with SRCDOM ADSL, SRCVAR DTHDT, and no sequence for a death; with SRCDOM QS, SRCVAR ADT, and the assessment sequence when censoring at an assessment; and with SRCDOM ADSL, SRCVAR RANDDT, and no sequence when censoring at randomization.

Read the source datasets from /app/input and save the completed dataset as /app/output/adtte.csv.

prompts/adam-adtte-pro-deterioration/conventions.md

Edit20 lines

Following CDISC ADaM standards, use the provided ADSL, QS, and DS datasets to create an ADTTE dataset for time to PRO deterioration, one record per subject.

The ADTTE dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR, EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ.

Use PARAMCD "TTDGHS" and PARAM "Time to Deterioration in Global Health Status". A deterioration is dated after the earliest assessment and scores at least 10 points below it. The event is the first deterioration, or death without a prior one; neither counts when it falls after the first censoring reason. Progression, discontinuation, and withdrawal are censoring reasons, never events. AVAL counts whole calendar months from STARTDT to ADT; CNSR is 0 for an event and 1 otherwise. EVNTDESC is PRO DETERIORATION, DEATH, or CENSORED; CNSDTDSC holds the censoring reason (STUDY COMPLETION when no reason is recorded). Trace ADT as QS/ADT/sequence, ADSL/DTHDT, or ADSL/RANDDT.

Read the source datasets from /app/input and save the completed dataset as /app/output/adtte.csv.

prompts/adam-adtte-pro-deterioration/brief.md

Edit10 lines

Following CDISC ADaM standards, use the provided ADSL, QS, and DS datasets to create an ADTTE dataset for time to PRO deterioration, one record per subject.

The ADTTE dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR, EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ.

Read the source datasets from /app/input and save the completed dataset as /app/output/adtte.csv.

run.R

Edit4 lines

1library(yamaa)23adtte <- yamaa_domain("spec.yaml")$output4adtte

run.py

Edit4 lines

1import yamaa23adtte = yamaa.yamaa_domain("spec.yaml").output4adtte

solutions/adam-adtte-pro-deterioration/result.R

Edit147 lines

1# Reference solution for the yamaa benchmark adam-adtte-pro-deterioration2# (R track).3#4# Written from the benchmark's full prompt and its inputs alone, as an agent5# would write it. Harbor's oracle agent runs it to check that the prompt can6# be solved and that the grader scores a correct answer 1.78library(dplyr, warn.conflicts = FALSE)9library(readr)10library(lubridate, warn.conflicts = FALSE)1112whole_months <- function(d1, d2) {13  m <- (year(d2) - year(d1)) * 12 + (month(d2) - month(d1))14  last <- days_in_month(d2)15  m - as.integer(day(d2) < day(d1) & day(d2) != last)16}1718adsl <- read_csv(19  "/app/input/adsl.csv",20  col_types = cols(21    RANDDT = col_date(), DTHDT = col_date(), .default = col_character()22  )23)24qs <- read_csv(25  "/app/input/qs.csv",26  col_types = cols(27    QSSEQ = col_integer(), ADT = col_date(), AVAL = col_integer(),28    .default = col_character()29  )30)31ds <- read_csv(32  "/app/input/ds.csv",33  col_types = cols(34    DSSEQ = col_integer(), DSDTC = col_date(), .default = col_character()35  )36)3738# Flag deteriorations as an internal step: the baseline is the earliest39# assessment, and an assessment deteriorates when it is dated after the40# baseline and its score is at least 10 points below the baseline score.41qs_flagged <- qs |>42  arrange(STUDYID, USUBJID, ADT, QSSEQ) |>43  group_by(STUDYID, USUBJID) |>44  mutate(45    BASEDT = first(ADT),46    BASEVAL = first(AVAL),47    DETERFL = if_else(48      !is.na(ADT) & !is.na(BASEDT) & ADT > BASEDT & AVAL <= BASEVAL - 10L,49      "Y",50      NA_character_51    )52  ) |>53  ungroup() |>54  select(STUDYID, USUBJID, QSSEQ, AVISIT, ADT, AVAL, DETERFL) |>55  arrange(USUBJID, QSSEQ)5657# The first deterioration per subject, earliest date then lowest sequence.58deter <- qs_flagged |>59  filter(DETERFL == "Y") |>60  arrange(STUDYID, USUBJID, ADT, QSSEQ) |>61  distinct(STUDYID, USUBJID, .keep_all = TRUE) |>62  select(STUDYID, USUBJID, DETERDT = ADT, DETERSEQ = QSSEQ)6364# The first censoring reason per subject. Only progression,65# discontinuation, and withdrawal count; any other DS record is ignored.66reason <- ds |>67  filter(68    DSDECOD %in% c(69      "DISEASE PROGRESSION", "STUDY DISCONTINUATION", "WITHDRAWAL OF CONSENT"70    )71  ) |>72  arrange(STUDYID, USUBJID, DSDTC, DSSEQ) |>73  distinct(STUDYID, USUBJID, .keep_all = TRUE) |>74  select(STUDYID, USUBJID, CENSORRSNDT = DSDTC, CENSORRSN = DSDECOD)7576# The latest assessment dated on or before the censoring reason, with the77# latest sequence breaking a tied date.78censor_qs <- qs_flagged |>79  inner_join(reason, by = c("STUDYID", "USUBJID")) |>80  filter(!is.na(ADT), ADT <= CENSORRSNDT) |>81  group_by(STUDYID, USUBJID) |>82  summarize(83    LASTQSLE = max(ADT),84    LASTQSSEQ = max(QSSEQ[ADT == max(ADT)]),85    .groups = "drop"86  )8788adtte <- adsl |>89  left_join(deter, by = c("STUDYID", "USUBJID")) |>90  left_join(reason, by = c("STUDYID", "USUBJID")) |>91  left_join(censor_qs, by = c("STUDYID", "USUBJID")) |>92  mutate(93    STARTDT = RANDDT,94    # Deterioration wins over death on the same date. Neither counts when95    # it falls after the first censoring reason: progression,96    # discontinuation, and withdrawal are censoring reasons, never events,97    # though a same-day deterioration or death still counts.98    DETER_EVENT = !is.na(DETERDT) & (is.na(DTHDT) | DETERDT <= DTHDT) &99      (is.na(CENSORRSNDT) | DETERDT <= CENSORRSNDT),100    DEATH_EVENT = !is.na(DTHDT) &101      (is.na(CENSORRSNDT) | DTHDT <= CENSORRSNDT),102    EVENTDT = case_when(103      DETER_EVENT ~ DETERDT,104      DEATH_EVENT ~ DTHDT,105      .default = as.Date(NA)106    ),107    CENSORDT = coalesce(LASTQSLE, STARTDT),108    ADT = coalesce(EVENTDT, CENSORDT),109    AVAL = whole_months(STARTDT, ADT),110    CNSR = if_else(!is.na(EVENTDT), 0L, 1L),111    EVNTDESC = case_when(112      DETER_EVENT ~ "PRO DETERIORATION",113      DEATH_EVENT ~ "DEATH",114      .default = "CENSORED"115    ),116    CNSDTDSC = case_when(117      !is.na(EVENTDT) ~ NA_character_,118      .default = coalesce(CENSORRSN, "STUDY COMPLETION")119    ),120    SRCDOM = case_when(121      DETER_EVENT ~ "QS",122      DEATH_EVENT ~ "ADSL",123      is.na(LASTQSLE) ~ "ADSL",124      .default = "QS"125    ),126    SRCVAR = case_when(127      DETER_EVENT ~ "ADT",128      DEATH_EVENT ~ "DTHDT",129      is.na(LASTQSLE) ~ "RANDDT",130      .default = "ADT"131    ),132    SRCSEQ = case_when(133      DETER_EVENT ~ DETERSEQ,134      is.na(EVENTDT) & !is.na(LASTQSLE) ~ LASTQSSEQ,135      .default = NA_integer_136    ),137    PARAMCD = "TTDGHS",138    PARAM = "Time to Deterioration in Global Health Status"139  ) |>140  select(141    STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR,142    EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ143  ) |>144  arrange(USUBJID)145146dir.create("/app/output", showWarnings = FALSE)147write_csv(adtte, "/app/output/adtte.csv", na = "")

solutions/adam-adtte-pro-deterioration/result.py

Edit178 lines

1# Reference solution for the yamaa benchmark adam-adtte-pro-deterioration2# (Python track).3#4# Written from the benchmark's full prompt and its inputs alone, as an agent5# would write it. Harbor's oracle agent runs it to check that the prompt can6# be solved and that the grader scores a correct answer 1.78import calendar9from pathlib import Path1011import polars as pl121314def whole_months(d1, d2):15    """Whole calendar months from d1 to d2."""16    m = (d2.year - d1.year) * 12 + (d2.month - d1.month)17    last = calendar.monthrange(d2.year, d2.month)[1]18    if d2.day < d1.day and d2.day != last:19        m -= 120    return m212223adsl = pl.read_csv(24    "/app/input/adsl.csv", infer_schema=False, null_values=""25).with_columns(26    pl.col("RANDDT").str.to_date(strict=False),27    pl.col("DTHDT").str.to_date(strict=False),28)29qs = pl.read_csv("/app/input/qs.csv", infer_schema=False, null_values="").with_columns(30    pl.col("QSSEQ").cast(pl.Int64, strict=False),31    pl.col("ADT").str.to_date(strict=False),32    pl.col("AVAL").cast(pl.Int64, strict=False),33)34ds = pl.read_csv("/app/input/ds.csv", infer_schema=False, null_values="").with_columns(35    pl.col("DSSEQ").cast(pl.Int64, strict=False),36    pl.col("DSDTC").str.to_date(strict=False),37)3839# Flag deteriorations as an internal step: the baseline is the earliest40# assessment, and an assessment deteriorates when it is dated after the41# baseline and its score is at least 10 points below the baseline score.42qs_flagged = (43    qs.sort(["STUDYID", "USUBJID", "ADT", "QSSEQ"], nulls_last=True)44    .with_columns(45        BASEDT=pl.col("ADT").first().over(["STUDYID", "USUBJID"]),46        BASEVAL=pl.col("AVAL").first().over(["STUDYID", "USUBJID"]),47    )48    .with_columns(49        DETERFL=pl.when(50            pl.col("ADT").is_not_null()51            & (pl.col("ADT") > pl.col("BASEDT"))52            & (pl.col("AVAL") <= pl.col("BASEVAL") - 10)53        )54        .then(pl.lit("Y"))55        .otherwise(None)56    )57    .select("STUDYID", "USUBJID", "QSSEQ", "AVISIT", "ADT", "AVAL", "DETERFL")58    .sort(["USUBJID", "QSSEQ"])59)6061# The first deterioration per subject, earliest date then lowest sequence.62deter = (63    qs_flagged.filter(pl.col("DETERFL") == "Y")64    .sort(["STUDYID", "USUBJID", "ADT", "QSSEQ"])65    .unique(["STUDYID", "USUBJID"], keep="first", maintain_order=True)66    .select("STUDYID", "USUBJID", DETERDT="ADT", DETERSEQ="QSSEQ")67)6869# The first censoring reason per subject. Only progression,70# discontinuation, and withdrawal count; any other DS record is ignored.71reason = (72    ds.filter(73        pl.col("DSDECOD").is_in(74            ["DISEASE PROGRESSION", "STUDY DISCONTINUATION", "WITHDRAWAL OF CONSENT"]75        )76    )77    .sort(["STUDYID", "USUBJID", "DSDTC", "DSSEQ"], nulls_last=True)78    .unique(["STUDYID", "USUBJID"], keep="first", maintain_order=True)79    .select("STUDYID", "USUBJID", CENSORRSNDT="DSDTC", CENSORRSN="DSDECOD")80)8182# The latest assessment dated on or before the censoring reason, with the83# latest sequence breaking a tied date.84censor_qs = (85    qs_flagged.join(reason, on=["STUDYID", "USUBJID"], how="inner")86    .filter(pl.col("ADT").is_not_null() & (pl.col("ADT") <= pl.col("CENSORRSNDT")))87    .group_by(["STUDYID", "USUBJID"])88    .agg(89        LASTQSLE=pl.col("ADT").max(),90        LASTQSSEQ=pl.col("QSSEQ").filter(pl.col("ADT") == pl.col("ADT").max()).max(),91    )92)9394adtte = (95    adsl.join(deter, on=["STUDYID", "USUBJID"], how="left", maintain_order="left")96    .join(reason, on=["STUDYID", "USUBJID"], how="left", maintain_order="left")97    .join(censor_qs, on=["STUDYID", "USUBJID"], how="left", maintain_order="left")98    .with_columns(STARTDT=pl.col("RANDDT"))99    .with_columns(100        # Deterioration wins over death on the same date. Neither counts101        # when it falls after the first censoring reason: progression,102        # discontinuation, and withdrawal are censoring reasons, never103        # events, though a same-day deterioration or death still counts.104        DETER_EVENT=pl.col("DETERDT").is_not_null()105        & (pl.col("DTHDT").is_null() | (pl.col("DETERDT") <= pl.col("DTHDT")))106        & (107            pl.col("CENSORRSNDT").is_null()108            | (pl.col("DETERDT") <= pl.col("CENSORRSNDT"))109        ),110        DEATH_EVENT=pl.col("DTHDT").is_not_null()111        & (112            pl.col("CENSORRSNDT").is_null() | (pl.col("DTHDT") <= pl.col("CENSORRSNDT"))113        ),114    )115    .with_columns(116        EVENTDT=pl.when(pl.col("DETER_EVENT"))117        .then(pl.col("DETERDT"))118        .when(pl.col("DEATH_EVENT"))119        .then(pl.col("DTHDT"))120        .otherwise(None),121        CENSORDT=pl.col("LASTQSLE").fill_null(pl.col("STARTDT")),122    )123    .with_columns(ADT=pl.col("EVENTDT").fill_null(pl.col("CENSORDT")))124    .with_columns(125        AVAL=pl.struct(["STARTDT", "ADT"]).map_elements(126            lambda s: whole_months(s["STARTDT"], s["ADT"]), return_dtype=pl.Int64127        ),128        CNSR=pl.when(pl.col("EVENTDT").is_not_null()).then(0).otherwise(1),129        EVNTDESC=pl.when(pl.col("DETER_EVENT"))130        .then(pl.lit("PRO DETERIORATION"))131        .when(pl.col("DEATH_EVENT"))132        .then(pl.lit("DEATH"))133        .otherwise(pl.lit("CENSORED")),134        CNSDTDSC=pl.when(pl.col("EVENTDT").is_not_null())135        .then(None)136        .otherwise(pl.col("CENSORRSN").fill_null(pl.lit("STUDY COMPLETION"))),137        SRCDOM=pl.when(pl.col("DETER_EVENT"))138        .then(pl.lit("QS"))139        .when(pl.col("DEATH_EVENT"))140        .then(pl.lit("ADSL"))141        .when(pl.col("LASTQSLE").is_null())142        .then(pl.lit("ADSL"))143        .otherwise(pl.lit("QS")),144        SRCVAR=pl.when(pl.col("DETER_EVENT"))145        .then(pl.lit("ADT"))146        .when(pl.col("DEATH_EVENT"))147        .then(pl.lit("DTHDT"))148        .when(pl.col("LASTQSLE").is_null())149        .then(pl.lit("RANDDT"))150        .otherwise(pl.lit("ADT")),151        SRCSEQ=pl.when(pl.col("DETER_EVENT"))152        .then(pl.col("DETERSEQ"))153        .when(pl.col("EVENTDT").is_null() & pl.col("LASTQSLE").is_not_null())154        .then(pl.col("LASTQSSEQ"))155        .otherwise(None),156        PARAMCD=pl.lit("TTDGHS"),157        PARAM=pl.lit("Time to Deterioration in Global Health Status"),158    )159    .select(160        "STUDYID",161        "USUBJID",162        "PARAMCD",163        "PARAM",164        "STARTDT",165        "ADT",166        "AVAL",167        "CNSR",168        "EVNTDESC",169        "CNSDTDSC",170        "SRCDOM",171        "SRCVAR",172        "SRCSEQ",173    )174    .sort("USUBJID")175)176177Path("/app/output").mkdir(parents=True, exist_ok=True)178adtte.write_csv("/app/output/adtte.csv", null_value="")

solutions/adam-adtte-pro-deterioration/result.sas

Edit57 lines

1/* opensas reference solution for the yamaa benchmark adam-adtte-pro-deterioration.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data adsl;6  length STUDYID USUBJID RANDDT DTHDT $1024;7  infile "/app/input/adsl.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. RANDDT : $char1024. DTHDT : $char1024.;9run;1011data ds;12  length STUDYID USUBJID DSDECOD DSDTC $1024;13  infile "/app/input/ds.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;14  input STUDYID : $char1024. USUBJID : $char1024. DSDECOD : $char1024. DSDTC : $char1024. DSSEQ;15run;1617data qs;18  length STUDYID USUBJID AVISIT ADT $1024;19  infile "/app/input/qs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;20  input STUDYID : $char1024. USUBJID : $char1024. QSSEQ AVISIT : $char1024. ADT : $char1024. AVAL;21run;2223proc sort data=qs(where=(ADT ne '')) out=ordered; by STUDYID USUBJID ADT QSSEQ; run;24data deteriorations;25  length BASEDT $10;26  retain BASEDT BASEVAL;27  set ordered; by STUDYID USUBJID;28  if first.USUBJID then do; BASEDT=ADT; BASEVAL=AVAL; end;29  if ADT>BASEDT and not missing(AVAL) and not missing(BASEVAL) and AVAL<=BASEVAL-10;30run;31data deterioration; set deteriorations; by STUDYID USUBJID; if first.USUBJID; keep STUDYID USUBJID ADT QSSEQ; rename ADT=DETERDT QSSEQ=DETERSEQ; run;32data ds_order; set ds; NODATE=missing(DSDTC); run;33proc sort data=ds_order(where=(DSDECOD in ('DISEASE PROGRESSION','STUDY DISCONTINUATION','WITHDRAWAL OF CONSENT'))) out=reasons; by STUDYID USUBJID NODATE DSDTC DSSEQ; run;34data reason; set reasons; by STUDYID USUBJID; if first.USUBJID; keep STUDYID USUBJID DSDTC DSDECOD; rename DSDTC=REASONDT DSDECOD=REASON; run;35proc sql; create table assessment_reasons as select a.*,b.REASONDT from ordered as a left join reason as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID; quit;36data candidates; set assessment_reasons; if missing(REASONDT) or ADT<=REASONDT; run;37proc sort data=candidates; by STUDYID USUBJID descending ADT descending QSSEQ; run;38data lastassessment; set candidates; by STUDYID USUBJID; if first.USUBJID; keep STUDYID USUBJID ADT QSSEQ; rename ADT=LASTDT QSSEQ=LASTSEQ; run;39proc sql; create table joined as select a.*,b.DETERDT,b.DETERSEQ,c.REASONDT,c.REASON,d.LASTDT,d.LASTSEQ from adsl as a40  left join deterioration as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID left join reason as c on a.STUDYID=c.STUDYID and a.USUBJID=c.USUBJID left join lastassessment as d on a.STUDYID=d.STUDYID and a.USUBJID=d.USUBJID; quit;41data result;42  length PARAMCD $8 PARAM $60 STARTDT ADT $10 EVNTDESC CNSDTDSC $40 SRCDOM $4 SRCVAR $16;43  set joined; PARAMCD='TTDGHS'; PARAM='Time to Deterioration in Global Health Status'; STARTDT=RANDDT;44  if not missing(DETERDT) and (missing(DTHDT) or DETERDT<=DTHDT) and (missing(REASONDT) or DETERDT<=REASONDT) then do; ADT=DETERDT; CNSR=0; EVNTDESC='PRO DETERIORATION'; SRCDOM='QS'; SRCVAR='ADT'; SRCSEQ=DETERSEQ; end;45  else if not missing(DTHDT) and (missing(REASONDT) or DTHDT<=REASONDT) then do; ADT=DTHDT; CNSR=0; EVNTDESC='DEATH'; SRCDOM='ADSL'; SRCVAR='DTHDT'; end;46  else do;47    CNSR=1; EVNTDESC='CENSORED'; CNSDTDSC=coalescec(REASON,'STUDY COMPLETION');48    if not missing(LASTDT) then do; ADT=LASTDT; SRCDOM='QS'; SRCVAR='ADT'; SRCSEQ=LASTSEQ; end;49    else do; ADT=RANDDT; SRCDOM='ADSL'; SRCVAR='RANDDT'; end;50  end;51  if not missing(STARTDT) and not missing(ADT) then AVAL=intck('month',input(STARTDT,yymmdd10.),input(ADT,yymmdd10.),'continuous');52run;5354proc sql;55  create table final as select STUDYID, USUBJID, PARAMCD, PARAM, STARTDT, ADT, AVAL, CNSR, EVNTDESC, CNSDTDSC, SRCDOM, SRCVAR, SRCSEQ from result;56quit;57proc export data=final outfile="/app/output/adtte.csv" dbms=csv replace; run;

input/adsl.csv

input/ds.csv

input/qs.csv

expected/adtte.csv

Comments

Mapping spec

Agent prompt

prompts/adam-adtte-pro-deterioration/full.md

prompts/adam-adtte-pro-deterioration/conventions.md

prompts/adam-adtte-pro-deterioration/brief.md

yamaa code

run.R

run.py

Solution without yamaa

solutions/adam-adtte-pro-deterioration/result.R

solutions/adam-adtte-pro-deterioration/result.py

solutions/adam-adtte-pro-deterioration/result.sas