As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: place each pharmacokinetic (PK) concentration sample on two
actual time scales in hours: the time elapsed since the first dose of the
drug the analyte measures (AFRLT) and since the sample's reference dose
(ARRLT), with the sample, first-dose and reference-dose date/times
(ADTM, FANLDTM, PCRFTDTM).
Input: PK concentration (PC) records carrying the analyte
(PCTESTCD), the planned time point (PCTPT) and the collection
date/time (PCDTC), and exposure (EX) records carrying the treatment
(EXTRT) and the dose start date/time (EXSTDTC). Each concentration
record gives one output record, identified by its sequence number
(PCSEQ); the records of a subject who was never dosed stay.
Variables:
PARAMCDandATPT: the analyte and the planned time point, as collected.ADTM: the collection date and time; empty when the sample was recorded with a date but no time, which is never filled in.FANLDTM: start of the subject's earliest dose of the analyte's drug,DRUG Xfor the parent drug (DRGX) and its metabolite (MET1) andDRUG YforDRGY; empty when the subject has no dose of that drug.PCRFTDTM: start of the reference dose, the latest dose of that drug starting at or before the sample, so a sample taken at a dose time refers to that dose; for a sample taken before every dose, the first dose. Empty whenADTMorFANLDTMis empty.AFRLT: hours fromFANLDTMtoADTM, not rounded; negative before the first dose; empty when either is empty.ARRLT: hours fromPCRFTDTMtoADTM, not rounded. It starts again at each dose whileAFRLTkeeps growing, and is negative only before the first dose; empty when either is empty.
Note: a dose of another drug is never a first or reference dose, and date/times are clock readings taken as written, with no time zone.
Inputs
Under review2 source filesinput/ex.csv
Edit6 rows| STUDYID | USUBJID | EXSEQ | EXTRT | EXDOSE | EXDOSU | EXSTDTC |
|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | DRUG X | 100 | mg | 2025-03-10T08:00 | |
| YAMAA-01 | 2 | DRUG X | 100 | mg | 2025-03-11T08:00 | |
| YAMAA-01 | 3 | DRUG X | 100 | mg | 2025-03-12T08:30 | |
| YAMAA-01 | 1 | DRUG X | 100 | mg | 2025-03-30T20:00 | |
| YAMAA-01 | 2 | DRUG Y | 50 | mg | 2025-03-31T08:00 | |
| YAMAA-01 | 3 | DRUG X | 100 | mg | 2025-03-31T20:00 |
input/pc.csv
Edit14 rows| STUDYID | USUBJID | PCSEQ | PCTESTCD | PCTEST | PCTPT | PCDTC |
|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | DRGX | Drug X | Pre-dose | 2025-03-10T07:40 | |
| YAMAA-01 | 2 | DRGX | Drug X | 0.5 h | 2025-03-10T08:30 | |
| YAMAA-01 | 3 | DRGX | Drug X | 2 h | 2025-03-10T10:10 | |
| YAMAA-01 | 4 | DRGX | Drug X | Pre-dose | 2025-03-11T08:00 | |
| YAMAA-01 | 5 | DRGX | Drug X | 4 h | 2025-03-11T12:00 | |
| YAMAA-01 | 6 | MET1 | Drug X Metabolite M1 | 4 h | 2025-03-11T12:00 | |
| YAMAA-01 | 7 | DRGX | Drug X | 8 h | 2025-03-11 | |
| YAMAA-01 | 8 | DRGX | Drug X | Pre-dose | 2025-03-12T08:15 | |
| YAMAA-01 | 9 | DRGX | Drug X | 1 h | 2025-03-12T09:30:36 | |
| YAMAA-01 | 1 | DRGY | Drug Y | Pre-dose | 2025-03-31T07:30 | |
| YAMAA-01 | 2 | DRGY | Drug Y | 2 h | 2025-03-31T10:00 | |
| YAMAA-01 | 3 | DRGX | Drug X | 6 h | 2025-04-01T02:00 | |
| YAMAA-01 | 4 | DRGY | Drug Y | 18 h | 2025-04-01T02:00 | |
| YAMAA-01 | 1 | DRGX | Drug X | Pre-dose | 2025-03-17T07:45 |
Expected output
Under review14 expected rowsexpected/adpc.csv
Edit14 rows| STUDYID | USUBJID | PCSEQ | PARAMCD | ATPT | ADTM | FANLDTM | PCRFTDTM | AFRLT | ARRLT |
|---|---|---|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | DRGX | Pre-dose | 2025-03-10T07:40:00 | 2025-03-10T08:00:00 | 2025-03-10T08:00:00 | -0.3333333333333333 | -0.3333333333333333 | |
| YAMAA-01 | 2 | DRGX | 0.5 h | 2025-03-10T08:30:00 | 2025-03-10T08:00:00 | 2025-03-10T08:00:00 | 0.5 | 0.5 | |
| YAMAA-01 | 3 | DRGX | 2 h | 2025-03-10T10:10:00 | 2025-03-10T08:00:00 | 2025-03-10T08:00:00 | 2.1666666666666665 | 2.1666666666666665 | |
| YAMAA-01 | 4 | DRGX | Pre-dose | 2025-03-11T08:00:00 | 2025-03-10T08:00:00 | 2025-03-11T08:00:00 | 24 | 0 | |
| YAMAA-01 | 5 | DRGX | 4 h | 2025-03-11T12:00:00 | 2025-03-10T08:00:00 | 2025-03-11T08:00:00 | 28 | 4 | |
| YAMAA-01 | 6 | MET1 | 4 h | 2025-03-11T12:00:00 | 2025-03-10T08:00:00 | 2025-03-11T08:00:00 | 28 | 4 | |
| YAMAA-01 | 7 | DRGX | 8 h | — | 2025-03-10T08:00:00 | — | — | — | |
| YAMAA-01 | 8 | DRGX | Pre-dose | 2025-03-12T08:15:00 | 2025-03-10T08:00:00 | 2025-03-11T08:00:00 | 48.25 | 24.25 | |
| YAMAA-01 | 9 | DRGX | 1 h | 2025-03-12T09:30:36 | 2025-03-10T08:00:00 | 2025-03-12T08:30:00 | 49.51 | 1.01 | |
| YAMAA-01 | 1 | DRGY | Pre-dose | 2025-03-31T07:30:00 | 2025-03-31T08:00:00 | 2025-03-31T08:00:00 | -0.5 | -0.5 | |
| YAMAA-01 | 2 | DRGY | 2 h | 2025-03-31T10:00:00 | 2025-03-31T08:00:00 | 2025-03-31T08:00:00 | 2 | 2 | |
| YAMAA-01 | 3 | DRGX | 6 h | 2025-04-01T02:00:00 | 2025-03-30T20:00:00 | 2025-03-31T20:00:00 | 30 | 6 | |
| YAMAA-01 | 4 | DRGY | 18 h | 2025-04-01T02:00:00 | 2025-03-31T08:00:00 | 2025-03-31T08:00:00 | 18 | 18 | |
| YAMAA-01 | 1 | DRGX | Pre-dose | 2025-03-17T07:45:00 | — | — | — | — |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADPC | STUDYID | Study Identifier | character | Copy value from PC.STUDYID. | Predecessor | ||
| ADPC | USUBJID | Unique Subject Identifier | character | Copy value from PC.USUBJID. | Predecessor | ||
| ADPC | PCSEQ | Sequence Number | numeric | Copy value from PC.PCSEQ. | Predecessor | ||
| ADPC | PARAMCD | Parameter Code | character | DRGX, MET1, DRGY | Copy value from PC.PCTESTCD. | Predecessor | |
| ADPC | ATPT | Analysis Time Point | character | Copy value from PC.PCTPT. | Predecessor | ||
| ADPC | ADTM | Analysis Date/Time | numeric | ISO 8601 | If PCDTPR IN ('minute', 'second') then "PC.PCDTC"; otherwise blank. | Derived | |
| ADPC | FANLDTM | First Dose Date/Time of the Analyte's Drug | numeric | ISO 8601 | Take EXSTDTC from the first EX record whose STUDYID equals STUDYID, USUBJID equals USUBJID and EXTRT equals DRUG, ordered by EX.EXSTDTC then EX.EXSEQ, and no match yields blank. | Derived | |
| ADPC | PCRFTDTM | Date/Time of Reference Dose | numeric | ISO 8601 | If ADTM IS NULL then "None"; If EXSTDTC IS NOT NULL, with EXSTDTC taken from the last EX record whose STUDYID equals STUDYID, USUBJID equals USUBJID and EXTRT equals DRUG, where ADTM is EXSTDTC or more, ordered by EX.EXSTDTC then EX.EXSEQ, and no match yields blank then "PRIORDOSE.EXSTDTC"; otherwise "FANLDTM". | Derived | |
| ADPC | AFRLT | Actual Relative Time from First Dose (h) | numeric | Elapsed hours from FANLDTM to ADTM, negative when ADTM is earlier, not rounded. | Derived | ||
| ADPC | ARRLT | Actual Relative Time from Reference Dose (h) | numeric | Elapsed hours from PCRFTDTM to ADTM, negative when ADTM is earlier, not rounded. | Derived |
| Variable | Permitted values |
|---|---|
| PARAMCD | DRGX, MET1, DRGY |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adpc-relative-times/full.md
Edit32 linesFollowing CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT
Each concentration record gives one output record, identified by its sequence number PCSEQ, and the records of a subject who was never dosed stay. PARAMCD is the analyte as collected in PCTESTCD, and ATPT is the planned time point as collected in PCTPT. ADTM is the collection date and time in PCDTC. ADTM has no value when the sample was recorded with a date but no time, and that time is never filled in. The analyte's drug is "DRUG X" for the parent drug DRGX and its metabolite MET1, and "DRUG Y" for DRGY. FANLDTM is the start date and time of the subject's earliest dose of the analyte's drug. FANLDTM has no value when the subject has no dose of that drug. A dose of another drug is never a first dose or a reference dose. PCRFTDTM is the start date and time of the reference dose, the latest dose of the analyte's drug starting at or before the sample, so a sample taken at a dose time refers to that dose. For a sample taken before every dose, the reference dose is the first dose. PCRFTDTM has no value when ADTM or FANLDTM has no value. AFRLT is the number of hours from FANLDTM to ADTM, not rounded. AFRLT is negative for a sample taken before the first dose, and it has no value when FANLDTM or ADTM has no value. ARRLT is the number of hours from PCRFTDTM to ADTM, not rounded. ARRLT starts again at each dose while AFRLT keeps growing, and ARRLT is negative only for a sample taken before the first dose. ARRLT has no value when PCRFTDTM or ADTM has no value. Date/times are clock readings taken as written, with no time zone.
Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.
prompts/adam-adpc-relative-times/conventions.md
Edit13 linesFollowing CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT
The analyte's drug is "DRUG X" for the parent drug DRGX and its metabolite MET1, and "DRUG Y" for DRGY. AFRLT is not rounded. ARRLT is not rounded. Date/times are clock readings taken as written, with no time zone.
Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.
prompts/adam-adpc-relative-times/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided EX and PC datasets to create an ADPC dataset with one record per concentration record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT
Read the source datasets from /app/input and save the completed dataset as /app/output/adpc.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-adpc-relative-times/result.R
Edit68 lines# Reference solution for the yamaa benchmark adam-adpc-relative-times (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(lubridate, warn.conflicts = FALSE)library(readr)pc <- read_csv( "/app/input/pc.csv", col_types = cols(PCSEQ = col_integer(), .default = col_character()))ex <- read_csv("/app/input/ex.csv", col_types = cols(.default = col_character()))# A date without a time is not a moment; a time without seconds counts as# second zero. Clock readings, no time zone.moment <- function(x) { full <- if_else(nchar(x) == 16, paste0(x, ":00"), x) if_else(!is.na(x) & nchar(x) > 10, ymd_hms(full, tz = "UTC", quiet = TRUE), as.POSIXct(NA, tz = "UTC"))}stamp <- function(x) if_else(is.na(x), NA_character_, format(x, "%Y-%m-%dT%H:%M:%S"))doses <- ex |> transmute(USUBJID, DRUG = EXTRT, START = moment(EXSTDTC)) |> filter(!is.na(START))samples <- pc |> mutate( ADTMM = moment(PCDTC), DRUG = case_when( PCTESTCD %in% c("DRGX", "MET1") ~ "DRUG X", PCTESTCD == "DRGY" ~ "DRUG Y" ) )first <- doses |> group_by(USUBJID, DRUG) |> summarise(FIRSTM = min(START), .groups = "drop")# The latest dose of the drug at or before the sample.latest <- samples |> filter(!is.na(ADTMM)) |> inner_join(doses, by = c("USUBJID", "DRUG"), relationship = "many-to-many") |> filter(START <= ADTMM) |> group_by(USUBJID, PCSEQ) |> summarise(LATESTM = max(START), .groups = "drop")adpc <- samples |> left_join(first, by = c("USUBJID", "DRUG")) |> left_join(latest, by = c("USUBJID", "PCSEQ")) |> mutate( # A sample taken before every dose refers to the first dose. REFM = if_else(is.na(ADTMM) | is.na(FIRSTM), as.POSIXct(NA, tz = "UTC"), coalesce(LATESTM, FIRSTM)), PARAMCD = PCTESTCD, ATPT = PCTPT, ADTM = stamp(ADTMM), FANLDTM = stamp(FIRSTM), PCRFTDTM = stamp(REFM), AFRLT = as.numeric(difftime(ADTMM, FIRSTM, units = "hours")), ARRLT = as.numeric(difftime(ADTMM, REFM, units = "hours")) ) |> select(STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT)dir.create("/app/output", showWarnings = FALSE)write_csv(adpc, "/app/output/adpc.csv", na = "")solutions/adam-adpc-relative-times/result.py
Edit115 lines# Reference solution for the yamaa benchmark adam-adpc-relative-times (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.import csvfrom datetime import datetimeINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"# The drug each analyte measures: the parent drug DRGX and its# metabolite MET1 both belong to "DRUG X"; DRGY belongs to "DRUG Y".ANALYTE_DRUG = {"DRGX": "DRUG X", "MET1": "DRUG X", "DRGY": "DRUG Y"}def parse_datetime(value): """Parse an ISO 8601 date-time as written. Returns None when the value carries a date but no time; such a time is never filled in. """ if value is None: return None value = value.strip() if "T" not in value: return None return datetime.fromisoformat(value)def main(): with open(f"{INPUT_DIR}/ex.csv", newline="") as f: ex_rows = list(csv.DictReader(f)) with open(f"{INPUT_DIR}/pc.csv", newline="") as f: pc_rows = list(csv.DictReader(f)) # Dose start date-times per subject and treatment, earliest first. # Each entry keeps the collected text alongside the parsed value. doses = {} for row in ex_rows: dtm = parse_datetime(row["EXSTDTC"]) if dtm is None: continue key = (row["USUBJID"], row["EXTRT"]) doses.setdefault(key, []).append((dtm, row["EXSTDTC"].strip())) for subject_doses in doses.values(): subject_doses.sort(key=lambda item: item[0]) out_rows = [] for row in pc_rows: drug = ANALYTE_DRUG[row["PCTESTCD"]] subject_doses = doses.get((row["USUBJID"], drug), []) adtm = parse_datetime(row["PCDTC"]) adtm_text = row["PCDTC"].strip() if adtm is not None else "" fanldtm = subject_doses[0][0] if subject_doses else None fanldtm_text = subject_doses[0][1] if subject_doses else "" # Reference dose: the latest dose of the analyte's drug starting # at or before the sample; a sample before every dose refers to # the first dose. No value when ADTM or FANLDTM has no value. pcrftdtm = None pcrftdtm_text = "" if adtm is not None and fanldtm is not None: ref = subject_doses[0] for dose in subject_doses: if dose[0] <= adtm: ref = dose else: break pcrftdtm, pcrftdtm_text = ref afrlt = "" if adtm is not None and fanldtm is not None: afrlt = (adtm - fanldtm).total_seconds() / 3600 arrlt = "" if adtm is not None and pcrftdtm is not None: arrlt = (adtm - pcrftdtm).total_seconds() / 3600 out_rows.append( { "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "PCSEQ": row["PCSEQ"], "PARAMCD": row["PCTESTCD"], "ATPT": row["PCTPT"], "ADTM": adtm_text, "FANLDTM": fanldtm_text, "PCRFTDTM": pcrftdtm_text, "AFRLT": afrlt, "ARRLT": arrlt, } ) fieldnames = [ "STUDYID", "USUBJID", "PCSEQ", "PARAMCD", "ATPT", "ADTM", "FANLDTM", "PCRFTDTM", "AFRLT", "ARRLT", ] with open(f"{OUTPUT_DIR}/adpc.csv", "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=fieldnames) writer.writeheader() writer.writerows(out_rows)if __name__ == "__main__": main()solutions/adam-adpc-relative-times/result.sas
Edit73 lines/* opensas reference solution for the yamaa benchmark adam-adpc-relative-times. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data pc; length STUDYID USUBJID PCTESTCD PCTEST PCTPT PCDTC $1024; infile "/app/input/pc.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. PCSEQ PCTESTCD : $char1024. PCTEST : $char1024. PCTPT : $char1024. PCDTC : $char1024.;run;data ex; length STUDYID USUBJID EXTRT EXDOSU EXSTDTC $1024; infile "/app/input/ex.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. EXSEQ EXTRT : $char1024. EXDOSE EXDOSU : $char1024. EXSTDTC : $char1024.;run;/* A date without a time is not a moment; a time without seconds counts as second zero. Clock readings, no time zone. */data doses; set ex; length DRUG $1024; DRUG=EXTRT; if lengthn(EXSTDTC)>10 then START=input(cats(EXSTDTC, ifc(lengthn(EXSTDTC)=16, ':00', '')), e8601dt.); if not missing(START); keep USUBJID DRUG START;run;data samples; set pc; length DRUG $1024; if PCTESTCD in ('DRGX', 'MET1') then DRUG='DRUG X'; else if PCTESTCD='DRGY' then DRUG='DRUG Y'; if lengthn(PCDTC)>10 then ADTMN=input(cats(PCDTC, ifc(lengthn(PCDTC)=16, ':00', '')), e8601dt.);run;proc sql; create table first as select USUBJID, DRUG, min(START) as FIRSTN from doses group by USUBJID, DRUG; create table latest as select s.USUBJID, s.PCSEQ, max(d.START) as LATESTN from samples as s inner join doses as d on s.USUBJID=d.USUBJID and s.DRUG=d.DRUG where not missing(s.ADTMN) and d.START <= s.ADTMN group by s.USUBJID, s.PCSEQ; create table joined as select s.*, f.FIRSTN, l.LATESTN from samples as s left join first as f on s.USUBJID=f.USUBJID and s.DRUG=f.DRUG left join latest as l on s.USUBJID=l.USUBJID and s.PCSEQ=l.PCSEQ order by s.USUBJID, s.PCSEQ;quit;data result; length PARAMCD ATPT $1024 ADTM FANLDTM PCRFTDTM $19; set joined; PARAMCD=PCTESTCD; ATPT=PCTPT; ADTM=''; FANLDTM=''; PCRFTDTM=''; /* A sample taken before every dose refers to the first dose. */ REFN=.; if not missing(ADTMN) and not missing(FIRSTN) then REFN=coalesce(LATESTN, FIRSTN); if not missing(ADTMN) then ADTM=put(ADTMN, e8601dt19.); if not missing(FIRSTN) then FANLDTM=put(FIRSTN, e8601dt19.); if not missing(REFN) then PCRFTDTM=put(REFN, e8601dt19.); AFRLT=.; ARRLT=.; format AFRLT ARRLT best32.; if not missing(ADTMN) and not missing(FIRSTN) then AFRLT=(ADTMN-FIRSTN)/3600; if not missing(ADTMN) and not missing(REFN) then ARRLT=(ADTMN-REFN)/3600;run;proc sql; create table final as select STUDYID, USUBJID, PCSEQ, PARAMCD, ATPT, ADTM, FANLDTM, PCRFTDTM, AFRLT, ARRLT from result;quit;proc export data=final outfile="/app/output/adpc.csv" dbms=csv replace; run;
yamaa