Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Goal: build one EC (Exposure as Collected) record per dosing-log day, carrying ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, and ECADJ.

Input: a daily dosing log, one row per subject per day, carrying the date, tablets taken, whether the dose was taken, the reason a dose was missed, and any dose adjustment.

Variables:

  • ECSEQ numbers the subject's dosing days in date order. With STUDYID and USUBJID it identifies the record.
  • ECMOOD is PERFORMED on every record: these are administrations as collected.
  • ECOCCUR is Y when the subject took the day's dose, N when the dose was not taken.
  • ECREASND is the reason the dose was not taken, for example SUBJECT FORGOT DOSE; blank when the dose was taken.
  • ECDOSE is the tablets taken that day; blank on a day the dose was not taken.
  • ECSTDTC and ECENDTC are the dosing day, since one record covers one day.
  • ECADJ is the adjustment reason the log records for that day, for example DOSE REDUCED; blank when the day records none.

Note: a dose reduced after an adverse event carries the adjustment reason (DOSE REDUCED), not the event. A missed day keeps the treatment's unit, form, frequency, and route with no dose and the missed reason.

Inputs

Under review
1 source file

input/dosing_log.csv

Edit
10 rows
input/dosing_log.csv
STUDYIDUSUBJIDLOGDATETABLETSDOSE_TAKENMISSREASDOSE_ADJ
YAMAA-012025-01-022Y——
YAMAA-012025-01-032Y——
YAMAA-012025-01-042Y——
YAMAA-012025-01-05—NSUBJECT FORGOT DOSE—
YAMAA-012025-01-062Y——
YAMAA-012025-01-022Y——
YAMAA-012025-01-032Y——
YAMAA-012025-01-041Y—DOSE REDUCED
YAMAA-012025-01-051Y—DOSE REDUCED
YAMAA-012025-01-061Y—DOSE REDUCED

Expected output

Under review
10 expected rows

expected/ec.csv

Edit
10 rows
expected/ec.csv
DOMAINSTUDYIDUSUBJIDECSEQECTRTECMOODECOCCURECREASNDECDOSEECDOSUECDOSFRMECDOSFRQECROUTEECSTDTCECENDTCECADJ
ECYAMAA-011Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-022025-01-02—
ECYAMAA-012Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-032025-01-03—
ECYAMAA-013Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-042025-01-04—
ECYAMAA-014Study DrugPERFORMEDNSUBJECT FORGOT DOSE—tabletTABLETQDORAL2025-01-052025-01-05—
ECYAMAA-015Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-062025-01-06—
ECYAMAA-011Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-022025-01-02—
ECYAMAA-012Study DrugPERFORMEDY—2tabletTABLETQDORAL2025-01-032025-01-03—
ECYAMAA-013Study DrugPERFORMEDY—1tabletTABLETQDORAL2025-01-042025-01-04DOSE REDUCED
ECYAMAA-014Study DrugPERFORMEDY—1tabletTABLETQDORAL2025-01-052025-01-05DOSE REDUCED
ECYAMAA-015Study DrugPERFORMEDY—1tabletTABLETQDORAL2025-01-062025-01-06DOSE REDUCED
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Variable NameVariable LabelTypeLengthControlled Terms or FormatOriginCoreConversion DefinitionVariable TypeVariable OrderComments for Define
DOMAINDomain AbbreviationCharAssignedSet constant value "EC".SDTM1
STUDYIDStudy IdentifierCharCollectedCopy value from LOG.STUDYID.SDTM2
USUBJIDUnique Subject IdentifierCharCollectedCopy value from LOG.USUBJID.SDTM3
ECTRTName of TreatmentCharAssignedSet constant value "Study Drug".SDTM4
ECMOODMoodCharAssignedSet constant value "PERFORMED".SDTM5
ECOCCUREC OccurrenceCharCollectedCopy value from LOG.DOSE_TAKEN.SDTM6
ECREASNDReason Not DosedCharCollectedCopy value from LOG.MISSREAS.SDTM7
ECDOSEDoseNumDerivedIf ECOCCUR = 'Y' then "LOG.TABLETS"; otherwise blank.SDTM8
ECDOSUDose UnitsCharAssignedSet constant value "tablet".SDTM9
ECDOSFRMDose FormCharAssignedSet constant value "TABLET".SDTM10
ECDOSFRQDosing Frequency per IntervalCharAssignedSet constant value "QD".SDTM11
ECROUTERoute of AdministrationCharAssignedSet constant value "ORAL".SDTM12
ECSTDTCStart Date/Time of TreatmentCharISO 8601CollectedCopy value from LOG.LOGDATE.SDTM13
ECENDTCEnd Date/Time of TreatmentCharISO 8601CollectedCopy value from LOG.LOGDATE.SDTM14
ECADJReason for Dose AdjustmentCharCollectedCopy value from LOG.DOSE_ADJ.SDTM15
ECSEQSequence NumberNumDerivedRow number within each group of STUDYID, USUBJID, ordered by LOG.LOGDATE.SDTM16
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/sdtm-ec-collected-exposure/full.md

Edit
21 lines

Following CDISC SDTM standards, use the provided LOG dataset to create an EC dataset with one record per dosing-log day.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ

ECSEQ numbers the subject's dosing days in date order. ECTRT is Study Drug and ECMOOD is PERFORMED on every record. ECOCCUR is Y when the subject took the day's dose and N when the dose was not taken. ECREASND is the reason the dose was not taken, as the log records it, and has no value when the dose was taken. ECDOSE is the tablets taken that day, with no value on a day the dose was not taken. ECDOSU is tablet, ECDOSFRM is TABLET, ECDOSFRQ is QD, and ECROUTE is ORAL. ECSTDTC and ECENDTC are both the dosing day, since one record covers one day. ECADJ is the adjustment reason the log records for that day, and has no value when the day records none. A missed day keeps the unit, form, frequency, and route with no dose and the missed reason.

Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.

prompts/sdtm-ec-collected-exposure/conventions.md

Edit
12 lines

Following CDISC SDTM standards, use the provided LOG dataset to create an EC dataset with one record per dosing-log day.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ

ECTRT is Study Drug and ECMOOD is PERFORMED on every record. ECOCCUR is Y or N. ECDOSFRM is TABLET, ECDOSFRQ is QD, and ECROUTE is ORAL.

Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.

prompts/sdtm-ec-collected-exposure/brief.md

Edit
9 lines

Following CDISC SDTM standards, use the provided LOG dataset to create an EC dataset with one record per dosing-log day.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ

Read the source datasets from /app/input and save the completed dataset as /app/output/ec.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)ec <- yamaa_domain("spec.yaml")$outputec

run.py

Edit
4 lines
import yamaaec = yamaa.yamaa_domain("spec.yaml").outputec

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/sdtm-ec-collected-exposure/result.R

Edit
41 lines
# Reference solution for the yamaa benchmark sdtm-ec-collected-exposure (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)log <- read_csv(  "/app/input/dosing_log.csv",  col_types = cols(.default = col_character()))ec <- log |>  mutate(    DOMAIN = "EC",    ECTRT = "Study Drug",    ECMOOD = "PERFORMED",    ECOCCUR = DOSE_TAKEN,    ECREASND = ifelse(DOSE_TAKEN == "Y", NA_character_, na_if(MISSREAS, "")),    ECDOSE = suppressWarnings(as.integer(ifelse(DOSE_TAKEN == "Y", TABLETS, NA_character_))),    ECDOSU = "tablet",    ECDOSFRM = "TABLET",    ECDOSFRQ = "QD",    ECROUTE = "ORAL",    ECSTDTC = suppressWarnings(as.Date(LOGDATE)),    ECENDTC = suppressWarnings(as.Date(LOGDATE)),    ECADJ = na_if(DOSE_ADJ, "")  ) |>  arrange(STUDYID, USUBJID, ECSTDTC) |>  group_by(STUDYID, USUBJID) |>  mutate(ECSEQ = row_number()) |>  ungroup() |>  select(    DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND,    ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ  )dir.create("/app/output", showWarnings = FALSE)write_csv(ec, "/app/output/ec.csv", na = "")

solutions/sdtm-ec-collected-exposure/result.py

Edit
61 lines
# Reference solution for the yamaa benchmark sdtm-ec-collected-exposure (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pllog = pl.read_csv("/app/input/dosing_log.csv", infer_schema=False)ec = (    log.with_columns(        DOMAIN=pl.lit("EC"),        ECTRT=pl.lit("Study Drug"),        ECMOOD=pl.lit("PERFORMED"),        ECOCCUR=pl.col("DOSE_TAKEN"),        ECREASND=pl.when(pl.col("DOSE_TAKEN") == "Y")        .then(None)        .otherwise(pl.col("MISSREAS"))        .map_elements(lambda v: None if v == "" else v, return_dtype=pl.String),        ECDOSE=pl.when(pl.col("DOSE_TAKEN") == "Y")        .then(pl.col("TABLETS").cast(pl.Int64, strict=False))        .otherwise(None),        ECDOSU=pl.lit("tablet"),        ECDOSFRM=pl.lit("TABLET"),        ECDOSFRQ=pl.lit("QD"),        ECROUTE=pl.lit("ORAL"),        ECSTDTC=pl.col("LOGDATE"),        ECENDTC=pl.col("LOGDATE"),        ECADJ=pl.col("DOSE_ADJ").map_elements(            lambda v: None if v == "" or v is None else v, return_dtype=pl.String        ),    )    .with_columns(LOGDATE_DT=pl.col("LOGDATE").str.to_date(strict=False))    .sort(["STUDYID", "USUBJID", "LOGDATE_DT"])    .with_columns(ECSEQ=pl.col("USUBJID").cum_count().over(["STUDYID", "USUBJID"]))    .with_columns(ECSEQ=pl.col("ECSEQ").cast(pl.Int64))    .select(        "DOMAIN",        "STUDYID",        "USUBJID",        "ECSEQ",        "ECTRT",        "ECMOOD",        "ECOCCUR",        "ECREASND",        "ECDOSE",        "ECDOSU",        "ECDOSFRM",        "ECDOSFRQ",        "ECROUTE",        "ECSTDTC",        "ECENDTC",        "ECADJ",    ))Path("/app/output").mkdir(exist_ok=True)ec.write_csv("/app/output/ec.csv")

solutions/sdtm-ec-collected-exposure/result.sas

Edit
24 lines
/* opensas reference solution for the yamaa benchmark sdtm-ec-collected-exposure.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dosing_log;  length STUDYID USUBJID LOGDATE TABLETS DOSE_TAKEN MISSREAS DOSE_ADJ $1024;  infile "/app/input/dosing_log.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. LOGDATE : $char1024. TABLETS : $char1024. DOSE_TAKEN : $char1024. MISSREAS : $char1024. DOSE_ADJ : $char1024.;run;data prepared; set dosing_log; length DOMAIN $2 ECTRT ECMOOD ECOCCUR ECREASND ECDOSU ECDOSFRM ECDOSFRQ ECROUTE ECSTDTC ECENDTC ECADJ $1024;DOMAIN='EC'; ECTRT='Study Drug'; ECMOOD='PERFORMED'; ECOCCUR=DOSE_TAKEN; ECDOSE=.; ECREASND='';if ECOCCUR='Y' then ECDOSE=input(TABLETS,best32.); else ECREASND=MISSREAS;ECDOSU='tablet'; ECDOSFRM='TABLET'; ECDOSFRQ='QD'; ECROUTE='ORAL'; ECSTDTC=LOGDATE; ECENDTC=LOGDATE; ECADJ=DOSE_ADJ; run;proc sort data=prepared; by STUDYID USUBJID ECSTDTC; run;data result; set prepared; by STUDYID USUBJID; retain ECSEQ;  if first.USUBJID then ECSEQ=0;  ECSEQ+1;run;proc sql;  create table final as select DOMAIN, STUDYID, USUBJID, ECSEQ, ECTRT, ECMOOD, ECOCCUR, ECREASND, ECDOSE, ECDOSU, ECDOSFRM, ECDOSFRQ, ECROUTE, ECSTDTC, ECENDTC, ECADJ from result;quit;proc export data=final outfile="/app/output/ec.csv" dbms=csv replace; run;