Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: place each pharmacokinetic (PK) concentration sample on two actual time scales in hours: the time elapsed since the first dose of the drug the analyte measures (AFRLT) and since the sample's reference dose (ARRLT), with the sample, first-dose and reference-dose date/times (ADTM, FANLDTM, PCRFTDTM).

Input: PK concentration (PC) records carrying the analyte (PCTESTCD), the planned time point (PCTPT) and the collection date/time (PCDTC), and exposure (EX) records carrying the treatment (EXTRT) and the dose start date/time (EXSTDTC). Each concentration record gives one output record, identified by its sequence number (PCSEQ); the records of a subject who was never dosed stay.

Variables:

  • PARAMCD and ATPT: the analyte and the planned time point, as collected.
  • ADTM: the collection date and time; empty when the sample was recorded with a date but no time, which is never filled in.
  • FANLDTM: start of the subject's earliest dose of the analyte's drug, DRUG X for the parent drug (DRGX) and its metabolite (MET1) and DRUG Y for DRGY; empty when the subject has no dose of that drug.
  • PCRFTDTM: start of the reference dose, the latest dose of that drug starting at or before the sample, so a sample taken at a dose time refers to that dose; for a sample taken before every dose, the first dose. Empty when ADTM or FANLDTM is empty.
  • AFRLT: hours from FANLDTM to ADTM, not rounded; negative before the first dose; empty when either is empty.
  • ARRLT: hours from PCRFTDTM to ADTM, not rounded. It starts again at each dose while AFRLT keeps growing, and is negative only before the first dose; empty when either is empty.

Note: a dose of another drug is never a first or reference dose, and date/times are clock readings taken as written, with no time zone.

Inputs

Under review
2 source files

input/ex.csv

Edit
6 rows
input/ex.csv
STUDYIDUSUBJIDEXSEQEXTRTEXDOSEEXDOSUEXSTDTC
YAMAA-011DRUG X100mg2025-03-10T08:00
YAMAA-012DRUG X100mg2025-03-11T08:00
YAMAA-013DRUG X100mg2025-03-12T08:30
YAMAA-011DRUG X100mg2025-03-30T20:00
YAMAA-012DRUG Y50mg2025-03-31T08:00
YAMAA-013DRUG X100mg2025-03-31T20:00

input/pc.csv

Edit
14 rows
input/pc.csv
STUDYIDUSUBJIDPCSEQPCTESTCDPCTESTPCTPTPCDTC
YAMAA-011DRGXDrug XPre-dose2025-03-10T07:40
YAMAA-012DRGXDrug X0.5 h2025-03-10T08:30
YAMAA-013DRGXDrug X2 h2025-03-10T10:10
YAMAA-014DRGXDrug XPre-dose2025-03-11T08:00
YAMAA-015DRGXDrug X4 h2025-03-11T12:00
YAMAA-016MET1Drug X Metabolite M14 h2025-03-11T12:00
YAMAA-017DRGXDrug X8 h2025-03-11
YAMAA-018DRGXDrug XPre-dose2025-03-12T08:15
YAMAA-019DRGXDrug X1 h2025-03-12T09:30:36
YAMAA-011DRGYDrug YPre-dose2025-03-31T07:30
YAMAA-012DRGYDrug Y2 h2025-03-31T10:00
YAMAA-013DRGXDrug X6 h2025-04-01T02:00
YAMAA-014DRGYDrug Y18 h2025-04-01T02:00
YAMAA-011DRGXDrug XPre-dose2025-03-17T07:45

Expected output

Under review
14 expected rows

expected/adpc.csv

Edit
14 rows
expected/adpc.csv
STUDYIDUSUBJIDPCSEQPARAMCDATPTADTMFANLDTMPCRFTDTMAFRLTARRLT
YAMAA-011DRGXPre-dose2025-03-10T07:40:002025-03-10T08:00:002025-03-10T08:00:00-0.3333333333333333-0.3333333333333333
YAMAA-012DRGX0.5 h2025-03-10T08:30:002025-03-10T08:00:002025-03-10T08:00:000.50.5
YAMAA-013DRGX2 h2025-03-10T10:10:002025-03-10T08:00:002025-03-10T08:00:002.16666666666666652.1666666666666665
YAMAA-014DRGXPre-dose2025-03-11T08:00:002025-03-10T08:00:002025-03-11T08:00:00240
YAMAA-015DRGX4 h2025-03-11T12:00:002025-03-10T08:00:002025-03-11T08:00:00284
YAMAA-016MET14 h2025-03-11T12:00:002025-03-10T08:00:002025-03-11T08:00:00284
YAMAA-017DRGX8 h—2025-03-10T08:00:00———
YAMAA-018DRGXPre-dose2025-03-12T08:15:002025-03-10T08:00:002025-03-11T08:00:0048.2524.25
YAMAA-019DRGX1 h2025-03-12T09:30:362025-03-10T08:00:002025-03-12T08:30:0049.511.01
YAMAA-011DRGYPre-dose2025-03-31T07:30:002025-03-31T08:00:002025-03-31T08:00:00-0.5-0.5
YAMAA-012DRGY2 h2025-03-31T10:00:002025-03-31T08:00:002025-03-31T08:00:0022
YAMAA-013DRGX6 h2025-04-01T02:00:002025-03-30T20:00:002025-03-31T20:00:00306
YAMAA-014DRGY18 h2025-04-01T02:00:002025-03-31T08:00:002025-03-31T08:00:001818
YAMAA-011DRGXPre-dose2025-03-17T07:45:00————
Shaded columns are derived— = missing value

Human review and assessment

Your review

3 questions and your comments · View benchmark discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADPCSTUDYIDStudy IdentifiercharacterCopy value from PC.STUDYID.Predecessor
ADPCUSUBJIDUnique Subject IdentifiercharacterCopy value from PC.USUBJID.Predecessor
ADPCPCSEQSequence NumbernumericCopy value from PC.PCSEQ.Predecessor
ADPCPARAMCDParameter CodecharacterDRGX, MET1, DRGYCopy value from PC.PCTESTCD.Predecessor
ADPCATPTAnalysis Time PointcharacterCopy value from PC.PCTPT.Predecessor
ADPCADTMAnalysis Date/TimenumericISO 8601If PCDTPR IN ('minute', 'second') then "PC.PCDTC"; otherwise blank.Derived
ADPCFANLDTMFirst Dose Date/Time of the Analyte's DrugnumericISO 8601Take EXSTDTC from the first EX record whose STUDYID equals STUDYID, USUBJID equals USUBJID and EXTRT equals DRUG, ordered by EX.EXSTDTC then EX.EXSEQ, and no match yields blank.Derived
ADPCPCRFTDTMDate/Time of Reference DosenumericISO 8601If ADTM IS NULL then "None"; If EXSTDTC IS NOT NULL, with EXSTDTC taken from the last EX record whose STUDYID equals STUDYID, USUBJID equals USUBJID and EXTRT equals DRUG, where ADTM is EXSTDTC or more, ordered by EX.EXSTDTC then EX.EXSEQ, and no match yields blank then "PRIORDOSE.EXSTDTC"; otherwise "FANLDTM".Derived
ADPCAFRLTActual Relative Time from First Dose (h)numericElapsed hours from FANLDTM to ADTM, negative when ADTM is earlier, not rounded.Derived
ADPCARRLTActual Relative Time from Reference Dose (h)numericElapsed hours from PCRFTDTM to ADTM, negative when ADTM is earlier, not rounded.Derived
VariablePermitted values
PARAMCDDRGX, MET1, DRGY
VersionDateAuthorDescriptionReviewerSign-off
1.72026-10-20yamaa docs buildRead a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.62026-10-17yamaa docs buildDescribe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.52026-10-17yamaa docs buildName collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.42026-10-15yamaa docs buildState datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-adpc-relative-times/full.md

Edit
32 lines

Following CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT

Each concentration record gives one output record, identified by its sequence number PCSEQ, and the records of a subject who was never dosed stay. PARAMCD is the analyte as collected in PCTESTCD, and ATPT is the planned time point as collected in PCTPT. ADTM is the collection date and time in PCDTC. ADTM has no value when the sample was recorded with a date but no time, and that time is never filled in. The analyte's drug is "DRUG X" for the parent drug DRGX and its metabolite MET1, and "DRUG Y" for DRGY. FANLDTM is the start date and time of the subject's earliest dose of the analyte's drug. FANLDTM has no value when the subject has no dose of that drug. A dose of another drug is never a first dose or a reference dose. PCRFTDTM is the start date and time of the reference dose, the latest dose of the analyte's drug starting at or before the sample, so a sample taken at a dose time refers to that dose. For a sample taken before every dose, the reference dose is the first dose. PCRFTDTM has no value when ADTM or FANLDTM has no value. AFRLT is the number of hours from FANLDTM to ADTM, not rounded. AFRLT is negative for a sample taken before the first dose, and it has no value when FANLDTM or ADTM has no value. ARRLT is the number of hours from PCRFTDTM to ADTM, not rounded. ARRLT starts again at each dose while AFRLT keeps growing, and ARRLT is negative only for a sample taken before the first dose. ARRLT has no value when PCRFTDTM or ADTM has no value. Date/times are clock readings taken as written, with no time zone.

Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.

prompts/adam-adpc-relative-times/conventions.md

Edit
13 lines

Following CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT

The analyte's drug is "DRUG X" for the parent drug DRGX and its metabolite MET1, and "DRUG Y" for DRGY. AFRLT is not rounded. ARRLT is not rounded. Date/times are clock readings taken as written, with no time zone.

Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.

prompts/adam-adpc-relative-times/brief.md

Edit
8 lines

Following CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT

Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.

yamaa code

Public calls generated from benchmark.yaml; results must match the expected output

R

Edit
3 lines
library(yamaa)yamaa_spec("spec.yaml")$run()$output

Python

Edit
3 lines
import yamaayamaa.spec("spec.yaml").run().output

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-adpc-relative-times/result.R

Edit
68 lines
# Reference solution for the yamaa benchmark adam-adpc-relative-times (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate, warn.conflicts = FALSE)library(readr)pc <- read_csv(  "/app/input/pc.csv",  col_types = cols(PCSEQ = col_integer(), .default = col_character()))ex <- read_csv("/app/input/ex.csv", col_types = cols(.default = col_character()))# A date without a time is not a moment; a time without seconds counts as# second zero. Clock readings, no time zone.moment <- function(x) {  full <- if_else(nchar(x) == 16, paste0(x, ":00"), x)  if_else(!is.na(x) & nchar(x) > 10, ymd_hms(full, tz = "UTC", quiet = TRUE), as.POSIXct(NA, tz = "UTC"))}stamp <- function(x) if_else(is.na(x), NA_character_, format(x, "%Y-%m-%dT%H:%M:%S"))doses <- ex |>  transmute(USUBJID, DRUG = EXTRT, START = moment(EXSTDTC)) |>  filter(!is.na(START))samples <- pc |>  mutate(    ADTMM = moment(PCDTC),    DRUG = case_when(      PCTESTCD %in% c("DRGX", "MET1") ~ "DRUG X",      PCTESTCD == "DRGY" ~ "DRUG Y"    )  )first <- doses |>  group_by(USUBJID, DRUG) |>  summarise(FIRSTM = min(START), .groups = "drop")# The latest dose of the drug at or before the sample.latest <- samples |>  filter(!is.na(ADTMM)) |>  inner_join(doses, by = c("USUBJID", "DRUG"), relationship = "many-to-many") |>  filter(START <= ADTMM) |>  group_by(USUBJID, PCSEQ) |>  summarise(LATESTM = max(START), .groups = "drop")adpc <- samples |>  left_join(first, by = c("USUBJID", "DRUG")) |>  left_join(latest, by = c("USUBJID", "PCSEQ")) |>  mutate(    # A sample taken before every dose refers to the first dose.    REFM = if_else(is.na(ADTMM) | is.na(FIRSTM), as.POSIXct(NA, tz = "UTC"),                   coalesce(LATESTM, FIRSTM)),    PARAMCD = PCTESTCD,    ATPT = PCTPT,    ADTM = stamp(ADTMM),    FANLDTM = stamp(FIRSTM),    PCRFTDTM = stamp(REFM),    AFRLT = as.numeric(difftime(ADTMM, FIRSTM, units = "hours")),    ARRLT = as.numeric(difftime(ADTMM, REFM, units = "hours"))  ) |>  select(STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT)dir.create("/app/output", showWarnings = FALSE)write_csv(adpc, "/app/output/adpc.csv", na = "")

solutions/adam-adpc-relative-times/result.py

Edit
115 lines
# Reference solution for the yamaa benchmark adam-adpc-relative-times (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.import csvfrom datetime import datetimeINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"# The drug each analyte measures: the parent drug DRGX and its# metabolite MET1 both belong to "DRUG X"; DRGY belongs to "DRUG Y".ANALYTE_DRUG = {"DRGX": "DRUG X", "MET1": "DRUG X", "DRGY": "DRUG Y"}def parse_datetime(value):    """Parse an ISO 8601 date-time as written.    Returns None when the value carries a date but no time; such a    time is never filled in.    """    if value is None:        return None    value = value.strip()    if "T" not in value:        return None    return datetime.fromisoformat(value)def main():    with open(f"{INPUT_DIR}/ex.csv", newline="") as f:        ex_rows = list(csv.DictReader(f))    with open(f"{INPUT_DIR}/pc.csv", newline="") as f:        pc_rows = list(csv.DictReader(f))    # Dose start date-times per subject and treatment, earliest first.    # Each entry keeps the collected text alongside the parsed value.    doses = {}    for row in ex_rows:        dtm = parse_datetime(row["EXSTDTC"])        if dtm is None:            continue        key = (row["USUBJID"], row["EXTRT"])        doses.setdefault(key, []).append((dtm, row["EXSTDTC"].strip()))    for subject_doses in doses.values():        subject_doses.sort(key=lambda item: item[0])    out_rows = []    for row in pc_rows:        drug = ANALYTE_DRUG[row["PCTESTCD"]]        subject_doses = doses.get((row["USUBJID"], drug), [])        adtm = parse_datetime(row["PCDTC"])        adtm_text = row["PCDTC"].strip() if adtm is not None else ""        fanldtm = subject_doses[0][0] if subject_doses else None        fanldtm_text = subject_doses[0][1] if subject_doses else ""        # Reference dose: the latest dose of the analyte's drug starting        # at or before the sample; a sample before every dose refers to        # the first dose. No value when ADTM or FANLDTM has no value.        pcrftdtm = None        pcrftdtm_text = ""        if adtm is not None and fanldtm is not None:            ref = subject_doses[0]            for dose in subject_doses:                if dose[0] <= adtm:                    ref = dose                else:                    break            pcrftdtm, pcrftdtm_text = ref        afrlt = ""        if adtm is not None and fanldtm is not None:            afrlt = (adtm - fanldtm).total_seconds() / 3600        arrlt = ""        if adtm is not None and pcrftdtm is not None:            arrlt = (adtm - pcrftdtm).total_seconds() / 3600        out_rows.append(            {                "STUDYID": row["STUDYID"],                "USUBJID": row["USUBJID"],                "PCSEQ": row["PCSEQ"],                "PARAMCD": row["PCTESTCD"],                "ATPT": row["PCTPT"],                "ADTM": adtm_text,                "FANLDTM": fanldtm_text,                "PCRFTDTM": pcrftdtm_text,                "AFRLT": afrlt,                "ARRLT": arrlt,            }        )    fieldnames = [        "STUDYID",        "USUBJID",        "PCSEQ",        "PARAMCD",        "ATPT",        "ADTM",        "FANLDTM",        "PCRFTDTM",        "AFRLT",        "ARRLT",    ]    with open(f"{OUTPUT_DIR}/adpc.csv", "w", newline="") as f:        writer = csv.DictWriter(f, fieldnames=fieldnames)        writer.writeheader()        writer.writerows(out_rows)if __name__ == "__main__":    main()

solutions/adam-adpc-relative-times/result.sas

Edit
73 lines
/* opensas reference solution for the yamaa benchmark adam-adpc-relative-times.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data pc;  length STUDYID USUBJID PCTESTCD PCTEST PCTPT PCDTC $1024;  infile "/app/input/pc.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. PCSEQ PCTESTCD : $char1024. PCTEST : $char1024.        PCTPT : $char1024. PCDTC : $char1024.;run;data ex;  length STUDYID USUBJID EXTRT EXDOSU EXSTDTC $1024;  infile "/app/input/ex.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. EXSEQ EXTRT : $char1024. EXDOSE EXDOSU : $char1024.        EXSTDTC : $char1024.;run;/* A date without a time is not a moment; a time without seconds counts as   second zero. Clock readings, no time zone. */data doses;  set ex;  length DRUG $1024;  DRUG=EXTRT;  if lengthn(EXSTDTC)>10 then START=input(cats(EXSTDTC, ifc(lengthn(EXSTDTC)=16, ':00', '')), e8601dt.);  if not missing(START);  keep USUBJID DRUG START;run;data samples;  set pc;  length DRUG $1024;  if PCTESTCD in ('DRGX', 'MET1') then DRUG='DRUG X';  else if PCTESTCD='DRGY' then DRUG='DRUG Y';  if lengthn(PCDTC)>10 then ADTMN=input(cats(PCDTC, ifc(lengthn(PCDTC)=16, ':00', '')), e8601dt.);run;proc sql;  create table first as select USUBJID, DRUG, min(START) as FIRSTN    from doses group by USUBJID, DRUG;  create table latest as select s.USUBJID, s.PCSEQ, max(d.START) as LATESTN    from samples as s inner join doses as d on s.USUBJID=d.USUBJID and s.DRUG=d.DRUG    where not missing(s.ADTMN) and d.START <= s.ADTMN    group by s.USUBJID, s.PCSEQ;  create table joined as select s.*, f.FIRSTN, l.LATESTN    from samples as s    left join first as f on s.USUBJID=f.USUBJID and s.DRUG=f.DRUG    left join latest as l on s.USUBJID=l.USUBJID and s.PCSEQ=l.PCSEQ    order by s.USUBJID, s.PCSEQ;quit;data result;  length PARAMCD ATPT $1024 ADTM FANLDTM PCRFTDTM $19;  set joined;  PARAMCD=PCTESTCD; ATPT=PCTPT;  ADTM=''; FANLDTM=''; PCRFTDTM='';  /* A sample taken before every dose refers to the first dose. */  REFN=.;  if not missing(ADTMN) and not missing(FIRSTN) then REFN=coalesce(LATESTN, FIRSTN);  if not missing(ADTMN) then ADTM=put(ADTMN, e8601dt19.);  if not missing(FIRSTN) then FANLDTM=put(FIRSTN, e8601dt19.);  if not missing(REFN) then PCRFTDTM=put(REFN, e8601dt19.);  AFRLT=.; ARRLT=.;  format AFRLT ARRLT best32.;  if not missing(ADTMN) and not missing(FIRSTN) then AFRLT=(ADTMN-FIRSTN)/3600;  if not missing(ADTMN) and not missing(REFN) then ARRLT=(ADTMN-REFN)/3600;run;proc sql;  create table final as select STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT    from result;quit;proc export data=final outfile="/app/output/adpc.csv" dbms=csv replace; run;