yamaa As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
EditUnder reviewWhat this benchmark meansREADME.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: produce pharmacokinetic concentration (PC) records, retaining
reported results and turning the vendor's nominal sample times into
numeric hours (PCTPTNUM).
Input: a bioanalytical laboratory's electronic transfer, with sample identifiers, subject identifiers, nominal time labels, analyte, reported plasma concentration, units and actual collection date and time.
Variables:
PCSEQnumbers samples within each subject by actual collection time, then sample identifier;PCREFIDretains that identifier.PCTESTCDandPCTESTidentify the measured drug,ZX201andZX-201.PCORRESandPCSTRESCretain the reported result as text, includingBLQfor below the limit of quantification.PCORRESUandPCSTRESUretain the supplied unit independently of the result. They are empty when no unit is supplied; no unit conversion is applied.PCSTRESNholds numeric results and is empty forBLQ; no limit-based numeric replacement is made.PCSPECisPLASMA, andPCDTCretains the actual collection time.PCTPTretains the vendor's nominal label.PCTPTNUMis zero forPRE, the number beforeHfor planned post-dose samples, and empty forUNSCHED.
Note: planned hours come from the nominal label, regardless of the actual collection time. An unrecognized label stops the run.
Inputs
Under review1 source fileinput/lab_transfer.csv
Edit5 rows| Protocol | Subject | Sample | Nominal | Analyte | Result | Unit | Collected |
|---|---|---|---|---|---|---|---|
| PK201 | PK201-001 | S001 | PRE | ZX-201 | BLQ | ng/mL | 2026-03-02T07:55:00 |
| PK201 | PK201-001 | S002 | 0.5H | ZX-201 | 10.5 | ng/mL | 2026-03-02T08:31:00 |
| PK201 | PK201-001 | S003 | 2H | ZX-201 | 6.25 | ng/mL | 2026-03-02T10:03:00 |
| PK201 | PK201-001 | S004 | UNSCHED | ZX-201 | 4 | ng/mL | 2026-03-02T11:17:00 |
| PK201 | PK201-002 | S005 | 24H | ZX-201 | 0.75 | ng/mL | 2026-03-03T08:06:00 |
Expected output
Under review5 expected rowsexpected/pc.csv
Edit5 rows| DOMAIN | STUDYID | USUBJID | PCSEQ | PCREFID | PCTESTCD | PCTEST | PCORRES | PCORRESU | PCSTRESC | PCSTRESN | PCSTRESU | PCSPEC | PCDTC | PCTPT | PCTPTNUM |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PC | PK201 | 1 | S001 | ZX201 | ZX-201 | BLQ | ng/mL | BLQ | — | ng/mL | PLASMA | 2026-03-02T07:55:00 | PRE | 0 | |
| PC | PK201 | 2 | S002 | ZX201 | ZX-201 | 10.5 | ng/mL | 10.5 | 10.5 | ng/mL | PLASMA | 2026-03-02T08:31:00 | 0.5H | 0.5 | |
| PC | PK201 | 3 | S003 | ZX201 | ZX-201 | 6.25 | ng/mL | 6.25 | 6.25 | ng/mL | PLASMA | 2026-03-02T10:03:00 | 2H | 2 | |
| PC | PK201 | 4 | S004 | ZX201 | ZX-201 | 4 | ng/mL | 4 | 4 | ng/mL | PLASMA | 2026-03-02T11:17:00 | UNSCHED | — | |
| PC | PK201 | 1 | S005 | ZX201 | ZX-201 | 0.75 | ng/mL | 0.75 | 0.75 | ng/mL | PLASMA | 2026-03-03T08:06:00 | 24H | 24 |
Human review and assessment
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "PC". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from LAB.Protocol. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from LAB.Subject. | SDTM | 3 | ||||
| PCREFID | Reference ID | Char | Collected | Copy value from LAB.Sample. | SDTM | 4 | ||||
| PCTESTCD | PK Concentration Test Short Name | Char | Collected | Recode LAB.Analyte: "ZX-201" → "ZX201"; missing values stay missing; unlisted values are errors. | SDTM | 5 | ||||
| PCTEST | PK Concentration Test Name | Char | Collected | Copy value from LAB.Analyte. | SDTM | 6 | ||||
| PCORRES | Result or Finding in Original Units | Char | Collected | Copy value from LAB.Result. | SDTM | 7 | ||||
| PCORRESU | Original Units | Char | Collected | Copy value from LAB.Unit. | SDTM | 8 | ||||
| PCSTRESC | Character Result/Finding in Std Format | Char | Derived | Copy value from PCORRES. | SDTM | 9 | ||||
| PCSTRESN | Numeric Result/Finding in Standard Units | Num | Derived | If PCORRES = 'BLQ' then "None"; otherwise "PCORRES". | SDTM | 10 | ||||
| PCSTRESU | Standard Units | Char | Derived | Copy value from PCORRESU. | SDTM | 11 | ||||
| PCSPEC | Specimen Material Type | Char | Assigned | Set constant value "PLASMA". | SDTM | 12 | ||||
| PCDTC | Date/Time of Specimen Collection | Char | ISO 8601 | Collected | Copy value from LAB.Collected. | SDTM | 13 | |||
| PCTPT | Planned Time Point Name | Char | Collected | Copy value from LAB.Nominal. | SDTM | 14 | ||||
| PCTPTNUM | Planned Time Point Number | Num | Derived | If PCTPT = 'PRE' then "0"; If PCTPT = 'UNSCHED' then "None"; otherwise extract capture group 1 matching ^([0-9]+(?:\.[0-9]+)?)H$ from PCTPT. | SDTM | 15 | ||||
| PCSEQ | Sequence Number | Num | Derived | Row number within each group of LAB.Protocol, LAB.Subject, ordered by LAB.Collected then LAB.Sample. | SDTM | 16 |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-pc-pk-concentrations/full.md
Edit23 linesFollowing CDISC SDTM standards, use the provided LAB_TRANSFER dataset to create a PC dataset with one record per sample.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, PCSEQ, PCREFID, PCTESTCD, PCTEST, PCORRES, PCORRESU, PCSTRESC, PCSTRESN, PCSTRESU, PCSPEC, PCDTC, PCTPT, PCTPTNUM
DOMAIN is "PC". STUDYID and USUBJID are the Protocol and Subject identifiers as supplied, respectively. PCSEQ numbers samples from 1 within each study and subject by ascending actual collection datetime, then ascending Sample identifier (PCREFID). PCTESTCD is "ZX201", PCTEST is "ZX-201", and PCSPEC is "PLASMA". PCORRES and PCSTRESC hold the reported concentration text unchanged, including "BLQ" and the numeric text with its supplied decimal places. PCORRESU and PCSTRESU both hold the supplied unit independently of the reported result, including for "BLQ", with no unit conversion. PCSTRESN holds the numeric reported concentration without rounding and has no value for "BLQ"; no limit-based numeric replacement is made. PCTPTNUM is the planned time in hours represented by the vendor's nominal label in PCTPT: 0 for "PRE", the numeric value before "H" for planned post-dose labels, and no value for "UNSCHED", regardless of the actual collection time.
Read the source datasets from /app/input and save the completed dataset as /app/output/pc.csv.
prompts/sdtm-pc-pk-concentrations/conventions.md
Edit14 linesFollowing CDISC SDTM standards, use the provided LAB_TRANSFER dataset to create a PC dataset with one record per sample.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, PCSEQ, PCREFID, PCTESTCD, PCTEST, PCORRES, PCORRESU, PCSTRESC, PCSTRESN, PCSTRESU, PCSPEC, PCDTC, PCTPT, PCTPTNUM
DOMAIN is "PC". PCTESTCD is "ZX201", PCTEST is "ZX-201", and PCSPEC is "PLASMA". PCORRES and PCSTRESC include "BLQ" and numeric text with its supplied decimal places. PCSTRESN is given without rounding. PCTPT includes "PRE", "UNSCHED", and planned post-dose labels with the numeric value before "H".
Read the source datasets from /app/input and save the completed dataset as /app/output/pc.csv.
prompts/sdtm-pc-pk-concentrations/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided LAB_TRANSFER dataset to create a PC dataset with one record per sample.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, PCSEQ, PCREFID, PCTESTCD, PCTEST, PCORRES, PCORRESU, PCSTRESC, PCSTRESN, PCSTRESU, PCSPEC, PCDTC, PCTPT, PCTPTNUM
Read the source datasets from /app/input and save the completed dataset as /app/output/pc.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/sdtm-pc-pk-concentrations/result.R
Edit32 lineslab <- read.csv("/app/input/lab_transfer.csv", colClasses = "character", check.names = FALSE, stringsAsFactors = FALSE, na.strings = NULL)collection_time <- as.POSIXct(lab$Collected, format = "%Y-%m-%dT%H:%M:%OS", tz = "UTC")lab <- lab[order(lab$Protocol, lab$Subject, collection_time, lab$Sample), , drop = FALSE]seq_group <- interaction(lab$Protocol, lab$Subject, drop = TRUE)pcseq <- ave(seq_len(nrow(lab)), seq_group, FUN = seq_along)planned_hours <- rep(NA_real_, nrow(lab))planned_hours[lab$Nominal == "PRE"] <- 0postdose <- grepl("^[0-9]+(\\.[0-9]+)?H$", lab$Nominal)planned_hours[postdose] <- as.numeric(sub("H$", "", lab$Nominal[postdose]))pc <- data.frame( DOMAIN = "PC", STUDYID = lab$Protocol, USUBJID = lab$Subject, PCSEQ = as.integer(pcseq), PCREFID = lab$Sample, PCTESTCD = "ZX201", PCTEST = "ZX-201", PCORRES = lab$Result, PCORRESU = lab$Unit, PCSTRESC = lab$Result, PCSTRESN = suppressWarnings(as.numeric(lab$Result)), PCSTRESU = lab$Unit, PCSPEC = "PLASMA", PCDTC = lab$Collected, PCTPT = lab$Nominal, PCTPTNUM = planned_hours, stringsAsFactors = FALSE, check.names = FALSE)stopifnot(nrow(pc) == nrow(lab), identical(pc$PCORRES, pc$PCSTRESC), identical(pc$PCORRESU, pc$PCSTRESU))write.csv(pc, "/app/output/pc.csv", row.names = FALSE, na = "")solutions/sdtm-pc-pk-concentrations/result.py
Edit51 linesfrom pathlib import Pathimport pandas as pddef main(): source = pd.read_csv( "/app/input/lab_transfer.csv", dtype=str, keep_default_na=False ) source["_collection_time"] = pd.to_datetime( source["Collected"], errors="raise", utc=True ) source = source.sort_values( ["Protocol", "Subject", "_collection_time", "Sample"], kind="stable" ) result = pd.DataFrame(index=source.index) result["DOMAIN"] = "PC" result["STUDYID"] = source["Protocol"] result["USUBJID"] = source["Subject"] result["PCSEQ"] = source.groupby(["Protocol", "Subject"], sort=False).cumcount() + 1 result["PCREFID"] = source["Sample"] result["PCTESTCD"] = "ZX201" result["PCTEST"] = "ZX-201" result["PCORRES"] = source["Result"] result["PCORRESU"] = source["Unit"] result["PCSTRESC"] = source["Result"] # Preserve the reported numeric precision in the CSV, with BLQ left empty. result["PCSTRESN"] = source["Result"].where(source["Result"].ne("BLQ"), "") result["PCSTRESU"] = source["Unit"] result["PCSPEC"] = "PLASMA" result["PCDTC"] = source["Collected"] result["PCTPT"] = source["Nominal"] def planned_hours(label): if label == "PRE": return "0" if label == "UNSCHED": return "" if label.endswith("H"): hours = label[:-1] float(hours) return hours raise ValueError(f"Unexpected nominal label: {label}") result["PCTPTNUM"] = source["Nominal"].map(planned_hours) Path("/app/output").mkdir(exist_ok=True) result.to_csv("/app/output/pc.csv", index=False)if __name__ == "__main__": main()solutions/sdtm-pc-pk-concentrations/result.sas
Edit42 linesproc import datafile='/app/input/lab_transfer.csv' out=lab_transfer dbms=csv replace; guessingrows=max;run;proc sort data=lab_transfer out=lab_sorted; by Protocol Subject Collected Sample;run;data pc; length DOMAIN $2 STUDYID $200 USUBJID $200 PCSEQ 8 PCREFID $200 PCTESTCD $8 PCTEST $40 PCORRES $200 PCORRESU $40 PCSTRESC $200 PCSTRESN 8 PCSTRESU $40 PCSPEC $40 PCDTC $40 PCTPT $40 PCTPTNUM 8; set lab_sorted; by Protocol Subject; retain PCSEQ; if first.Subject then PCSEQ=0; PCSEQ=PCSEQ+1; DOMAIN='PC'; STUDYID=Protocol; USUBJID=Subject; PCREFID=Sample; PCTESTCD='ZX201'; PCTEST='ZX-201'; PCORRES=Result; PCORRESU=Unit; PCSTRESC=PCORRES; PCSTRESN=.; if Result ne 'BLQ' then PCSTRESN=input(Result,best32.); PCSTRESU=PCORRESU; PCSPEC='PLASMA'; PCDTC=Collected; PCTPT=Nominal; PCTPTNUM=.; if Nominal='PRE' then PCTPTNUM=0; else if Nominal ne 'UNSCHED' then PCTPTNUM=input(compress(Nominal,'H'),best32.); keep DOMAIN STUDYID USUBJID PCSEQ PCREFID PCTESTCD PCTEST PCORRES PCORRESU PCSTRESC PCSTRESN PCSTRESU PCSPEC PCDTC PCTPT PCTPTNUM;run;proc export data=pc outfile='/app/output/pc.csv' dbms=csv replace;run;