As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: build one laboratory record per collected neutrophil or
hemoglobin result, carrying LBTESTCD, LBSTRESN, and LBTOXGR
from the Common Terminology Criteria for Adverse Events
(CTCAE) v5.0 bands for that test.
Input: collected laboratory results with test, sex, standardized numeric result, and an independently recorded grade confirming the assigned grade.
Variables:
LBTESTCDis the test short name as collected:ANC(absolute neutrophil count) orHGB(hemoglobin).LBSTRESNis the standardized numeric result as collected.LBTOXGRis the standard toxicity grade for the result, one of4,3,2,1, or0, from the band set for that test and sex; each band includes its lower limit and excludes its upper one.
Note: neutrophil counts use one band set whatever the sex, and
the example lab's lower limit of normal is 1.8 x 10^9/L. The ANC
bands are in 10^9/L: below 0.5 is grade 4, 0.5 to below 1.0 is
grade 3, 1.0 to below 1.5 is grade 2, and 1.5 to below 1.8 is
grade 1. Hemoglobin uses one band set per sex. A result for any other
test, or a hemoglobin result with a sex other than M or F, gets no record.
A missing result, or an assigned grade that differs from the
independently recorded one, stops the run.
Inputs
Under review1 source fileinput/lb_raw.csv
Edit10 rows| STUDYID | USUBJID | LBSEQ | LBTESTCD | SEX | LBSTRESN | REPROGRADE |
|---|---|---|---|---|---|---|
| PILOT7 | 1 | ANC | M | 1.6 | 1 | |
| PILOT7 | 2 | ANC | M | 0.75 | 3 | |
| PILOT7 | 1 | ANC | F | 0.08 | 4 | |
| PILOT7 | 1 | HGB | M | 12.5 | 1 | |
| PILOT7 | 1 | HGB | F | 12.5 | 0 | |
| PILOT7 | 2 | HGB | F | 9 | 2 | |
| PILOT7 | 1 | ANC | F | 0.5 | 3 | |
| PILOT7 | 2 | ANC | F | 1.0 | 2 | |
| PILOT7 | 3 | ANC | F | 1.5 | 1 | |
| PILOT7 | 4 | ANC | F | 1.8 | 0 |
Expected output
Under review10 expected rowsexpected/lb.csv
Edit10 rows| DOMAIN | STUDYID | USUBJID | LBSEQ | LBTESTCD | LBSTRESN | LBTOXGR |
|---|---|---|---|---|---|---|
| LB | PILOT7 | 1 | ANC | 1.6 | 1 | |
| LB | PILOT7 | 2 | ANC | 0.75 | 3 | |
| LB | PILOT7 | 1 | ANC | 0.08 | 4 | |
| LB | PILOT7 | 1 | ANC | 0.5 | 3 | |
| LB | PILOT7 | 2 | ANC | 1 | 2 | |
| LB | PILOT7 | 3 | ANC | 1.5 | 1 | |
| LB | PILOT7 | 4 | ANC | 1.8 | 0 | |
| LB | PILOT7 | 1 | HGB | 12.5 | 1 | |
| LB | PILOT7 | 1 | HGB | 12.5 | 0 | |
| LB | PILOT7 | 2 | HGB | 9 | 2 |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "LB". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from LB_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from LB_RAW.USUBJID. | SDTM | 3 | ||||
| LBSEQ | Sequence Number | Num | Collected | Copy value from LB_RAW.LBSEQ. | SDTM | 4 | ||||
| LBTESTCD | Lab Test or Examination Short Name | Char | Collected | Copy value from LB_RAW.LBTESTCD. | SDTM | 5 | ||||
| LBSTRESN | Numeric Result/Finding in Standard Units | Num | Collected | Copy value from LB_RAW.LBSTRESN. | SDTM | 6 | ||||
| LBTOXGR | Standard Toxicity Grade | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 7 |
| Template | Include when | LBTOXGR |
|---|---|---|
| neutrophils | LB_RAW.LBTESTCD = 'ANC' | Band LB_RAW.LBSTRESN at 0.5, 1.0, 1.5, 1.8 into "4", "3", "2", "1" and "0", intervals closed on the left |
| haemoglobin-male | LB_RAW.LBTESTCD = 'HGB' AND LB_RAW.SEX = 'M' | Band LB_RAW.LBSTRESN at 8.0, 10.0, 13.5 into "3", "2", "1" and "0", intervals closed on the left |
| haemoglobin-female | LB_RAW.LBTESTCD = 'HGB' AND LB_RAW.SEX = 'F' | Band LB_RAW.LBSTRESN at 8.0, 10.0, 12.0 into "3", "2", "1" and "0", intervals closed on the left |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-lb-grading/full.md
Edit25 linesFollowing CDISC SDTM standards, use the provided LB_RAW dataset to create an LB dataset with one laboratory record per collected neutrophil or hemoglobin result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBTOXGR
LBSEQ is the sequence number as collected. LBTESTCD is the test short name as collected: ANC for absolute neutrophil count or HGB for hemoglobin. LBSTRESN is the standardized numeric result as collected. LBTOXGR is the standard toxicity grade for the result: 4, 3, 2, 1, or 0. Each band includes its lower limit and excludes its upper one.
Neutrophil counts use one band set whatever the sex, with the example lab's lower limit of normal at 1.8: below 0.5 is grade 4, 0.5 to below 1.0 is grade 3, 1.0 to below 1.5 is grade 2, 1.5 to below 1.8 is grade 1, and 1.8 or above is grade 0. Hemoglobin uses one band set per sex: below 8.0 is grade 3, below 10.0 is grade 2, and below 13.5 for males (below 12.0 for females) is grade 1; higher results are grade 0.
A result for any other test, or a hemoglobin result with a sex other than M or F, gets no record.
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
prompts/sdtm-lb-grading/conventions.md
Edit12 linesFollowing CDISC SDTM standards, use the provided LB_RAW dataset to create an LB dataset with one laboratory record per collected neutrophil or hemoglobin result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBTOXGR
LBTESTCD is ANC for absolute neutrophil count or HGB for hemoglobin. LBTOXGR is 4, 3, 2, 1, or 0.
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
prompts/sdtm-lb-grading/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided LB_RAW dataset to create an LB dataset with one laboratory record per collected neutrophil or hemoglobin result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBTOXGR
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
run.R
Edit4 lineslibrary(yamaa)lb <- yamaa_domain("spec.yaml")$outputlbrun.py
Edit4 linesimport yamaalb = yamaa.yamaa_domain("spec.yaml").outputlbsolutions/sdtm-lb-grading/result.R
Edit49 lines# Reference solution for the yamaa benchmark sdtm-lb-grading (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)raw <- read_csv( "/app/input/lb_raw.csv", col_types = cols( LBSEQ = col_integer(), LBSTRESN = col_double(), .default = col_character() ))toxgrade <- function(test, sex, value) { if (is.na(value)) return(NA_character_) if (test == "ANC") { if (value < 0.5) return("4") if (value < 1.0) return("3") if (value < 1.5) return("2") if (value < 1.8) return("1") return("0") } if (test == "HGB") { if (!sex %in% c("M", "F")) return(NA_character_) if (value < 8.0) return("3") if (value < 10.0) return("2") limit <- if (sex == "M") 13.5 else 12.0 if (value < limit) return("1") return("0") } NA_character_}lb <- raw |> filter(LBTESTCD %in% c("ANC", "HGB"), !(LBTESTCD == "HGB" & !SEX %in% c("M", "F"))) |> rowwise() |> mutate(LBTOXGR = toxgrade(LBTESTCD, SEX, LBSTRESN)) |> ungroup() |> mutate(DOMAIN = "LB") |> select(DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBTOXGR) |> arrange(STUDYID, USUBJID, LBSEQ)dir.create("/app/output", showWarnings = FALSE)write_csv(lb, "/app/output/lb.csv", na = "")solutions/sdtm-lb-grading/result.py
Edit77 lines# Reference solution for the yamaa benchmark sdtm-lb-grading (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plraw = pl.read_csv("/app/input/lb_raw.csv", infer_schema=False).with_columns( pl.col("LBSEQ").cast(pl.Int64), pl.col("LBSTRESN").cast(pl.Float64, strict=False),)def toxgrade(test, sex, value): if value is None: return None if test == "ANC": if value < 0.5: return "4" if value < 1.0: return "3" if value < 1.5: return "2" if value < 1.8: return "1" return "0" if test == "HGB": if sex not in ("M", "F"): return None if value < 8.0: return "3" if value < 10.0: return "2" limit = 13.5 if sex == "M" else 12.0 if value < limit: return "1" return "0" return Nonerows = []for r in raw.to_dicts(): grade = toxgrade(r["LBTESTCD"], r["SEX"], r["LBSTRESN"]) if grade is None and r["LBTESTCD"] not in ("ANC", "HGB"): continue if r["LBTESTCD"] == "HGB" and r["SEX"] not in ("M", "F"): continue rows.append( { "DOMAIN": "LB", "STUDYID": r["STUDYID"], "USUBJID": r["USUBJID"], "LBSEQ": r["LBSEQ"], "LBTESTCD": r["LBTESTCD"], "LBSTRESN": r["LBSTRESN"], "LBTOXGR": grade, } )lb = pl.DataFrame( rows, schema={ "DOMAIN": pl.String, "STUDYID": pl.String, "USUBJID": pl.String, "LBSEQ": pl.Int64, "LBTESTCD": pl.String, "LBSTRESN": pl.Float64, "LBTOXGR": pl.String, },).sort(["STUDYID", "USUBJID", "LBSEQ"])Path("/app/output").mkdir(parents=True, exist_ok=True)lb.write_csv("/app/output/lb.csv")solutions/sdtm-lb-grading/result.sas
Edit17 lines/* opensas reference solution for the yamaa benchmark sdtm-lb-grading. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data lb_raw; length STUDYID USUBJID LBTESTCD SEX REPROGRADE $1024; infile "/app/input/lb_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. LBSEQ LBTESTCD : $char1024. SEX : $char1024. LBSTRESN REPROGRADE : $char1024.;run;data result; set lb_raw; length DOMAIN $2; DOMAIN='LB'; LBTOXGR=.;if LBTESTCD='ANC' then do; if missing(LBSTRESN) then LBTOXGR=.; else if LBSTRESN<0.5 then LBTOXGR=4; else if LBSTRESN<1 then LBTOXGR=3; else if LBSTRESN<1.5 then LBTOXGR=2; else if LBSTRESN<1.8 then LBTOXGR=1; else LBTOXGR=0; end;else if LBTESTCD='HGB' and SEX in ('M','F') then do; if missing(LBSTRESN) then LBTOXGR=.; else if LBSTRESN<8 then LBTOXGR=3; else if LBSTRESN<10 then LBTOXGR=2; else if (SEX='M' and LBSTRESN<13.5) or (SEX='F' and LBSTRESN<12) then LBTOXGR=1; else LBTOXGR=0; end; else delete; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBTOXGR from result;quit;proc export data=final outfile="/app/output/lb.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions