Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Goal: add absolute differential records (LYMPH, NEUT, MONO, EOS, BASO) whose analysis value (AVAL) is the white blood cell count times the matching differential fraction, and flag them with the record-type flag (DTYPE).

Input: SDTM laboratory records (LB) carrying the subject identifier (USUBJID), visit (VISIT), test code (LBTESTCD), test name (LBTEST), and numeric result (LBSTRESN): white blood cell counts (WBC), differential fractions (LYMLE, NEUTLE, MONOLE, EOSLE, BASOLE), a few lab-reported absolute differentials, and unrelated parameters.

Variables:

  • AVAL: the collected result on a collected record; on an added absolute record, the WBC count times the matching differential fraction for the same subject and visit.
  • DTYPE: CALCULATION on an added absolute record; empty on a collected record.

Note: one record is added per subject, visit, and differential: only when both a WBC count and that differential's fraction are present for the subject and visit, and no record of the new absolute parameter is already there. A differential with a fraction but no WBC count gains nothing, and an existing absolute record blocks only its own differential. Each contributing parameter may appear at most once within a subject and visit; a subject and visit with repeated WBC or fraction results stops the run instead of using a value picked by amount or position.

Inputs

Under review
1 source file

input/lb.csv

Edit
20 rows
input/lb.csv
USUBJIDLBTESTCDLBSTRESNLBTESTVISIT
WBC12Leukocyte Count (10^9/L)CYCLE 1 DAY 1
LYMLE0.5Lymphocytes (fraction of 1)CYCLE 1 DAY 1
WBC10Leukocyte Count (10^9/L)CYCLE 2 DAY 1
LYMLE0.75Lymphocytes (fraction of 1)CYCLE 2 DAY 1
WBC8Leukocyte Count (10^9/L)CYCLE 1 DAY 1
LYMLE1.0Lymphocytes (fraction of 1)CYCLE 1 DAY 1
WBC9Leukocyte Count (10^9/L)CYCLE 1 DAY 1
WBC7Leukocyte Count (10^9/L)CYCLE 1 DAY 1
LYMLE0.25Lymphocytes (fraction of 1)CYCLE 1 DAY 1
LYMPH1.75Lymphocytes Abs (10^9/L)CYCLE 1 DAY 1
NEUTLE0.25Neutrophils (fraction of 1)CYCLE 1 DAY 1
MONOLE0.125Monocytes (fraction of 1)CYCLE 1 DAY 1
NEUTLE0.75Neutrophils (fraction of 1)CYCLE 1 DAY 1
NEUTLE0.5Neutrophils (fraction of 1)CYCLE 1 DAY 1
EOSLE0.05Eosinophils (fraction of 1)CYCLE 1 DAY 1
MONOLE0.2Monocytes (fraction of 1)CYCLE 1 DAY 1
MONO1.4Monocytes Abs (10^9/L)CYCLE 1 DAY 1
WBC6Leukocyte Count (10^9/L)CYCLE 1 DAY 1
EOSLE0.25Eosinophils (fraction of 1)CYCLE 1 DAY 1
BASOLE0.125Basophils (fraction of 1)CYCLE 1 DAY 1

Expected output

Under review
29 expected rows

expected/adlb.csv

Edit
29 rows
expected/adlb.csv
USUBJIDPARAMCDAVALPARAMVISITDTYPE
WBC12Leukocyte Count (10^9/L)CYCLE 1 DAY 1—
LYMLE0.5Lymphocytes (fraction of 1)CYCLE 1 DAY 1—
WBC10Leukocyte Count (10^9/L)CYCLE 2 DAY 1—
LYMLE0.75Lymphocytes (fraction of 1)CYCLE 2 DAY 1—
WBC8Leukocyte Count (10^9/L)CYCLE 1 DAY 1—
LYMLE1Lymphocytes (fraction of 1)CYCLE 1 DAY 1—
WBC9Leukocyte Count (10^9/L)CYCLE 1 DAY 1—
WBC7Leukocyte Count (10^9/L)CYCLE 1 DAY 1—
LYMLE0.25Lymphocytes (fraction of 1)CYCLE 1 DAY 1—
LYMPH1.75Lymphocytes Abs (10^9/L)CYCLE 1 DAY 1—
NEUTLE0.25Neutrophils (fraction of 1)CYCLE 1 DAY 1—
MONOLE0.125Monocytes (fraction of 1)CYCLE 1 DAY 1—
NEUTLE0.75Neutrophils (fraction of 1)CYCLE 1 DAY 1—
NEUTLE0.5Neutrophils (fraction of 1)CYCLE 1 DAY 1—
EOSLE0.05Eosinophils (fraction of 1)CYCLE 1 DAY 1—
MONOLE0.2Monocytes (fraction of 1)CYCLE 1 DAY 1—
MONO1.4Monocytes Abs (10^9/L)CYCLE 1 DAY 1—
WBC6Leukocyte Count (10^9/L)CYCLE 1 DAY 1—
EOSLE0.25Eosinophils (fraction of 1)CYCLE 1 DAY 1—
BASOLE0.125Basophils (fraction of 1)CYCLE 1 DAY 1—
LYMPH6Lymphocytes Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
LYMPH7.5Lymphocytes Abs (10^9/L)CYCLE 2 DAY 1CALCULATION
LYMPH8Lymphocytes Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
NEUT3Neutrophils Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
NEUT6Neutrophils Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
NEUT4.5Neutrophils Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
MONO1.5Monocytes Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
EOS1.5Eosinophils Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
BASO0.75Basophils Abs (10^9/L)CYCLE 1 DAY 1CALCULATION
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADLBUSUBJIDUnique Subject IdentifiercharacterCopy value from LB.USUBJID.Predecessor
ADLBVISITVisit NamecharacterCopy value from LB.VISIT.Predecessor
ADLBPARAMCDParameter CodecharacterSet per constructed row; see the Row construction sheet.Derived
ADLBPARAMParametercharacterSet per constructed row; see the Row construction sheet.Derived
ADLBAVALAnalysis ValuenumericSet per constructed row; see the Row construction sheet.Derived
ADLBDTYPEDerivation TypecharacterSet per constructed row; see the Row construction sheet.Derived
TemplateInclude whenPARAMCDPARAMWBCVALFRACVALEXISTING_ABS_COUNTAVALDTYPE
collectedCopy LB.LBTESTCDCopy LB.LBTESTConstant "None"Constant "None"Constant "None"Copy LB.LBSTRESNConstant "None"
derived_lymphWBCVAL IS NOT NULL AND FRACVAL IS NOT NULL AND EXISTING_ABS_COUNT IS NULLConstant "LYMPH"Constant "Lymphocytes Abs (10^9/L)"Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'WBC'Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'LYMLE'Aggregate COUNT(LB.*) over records matching on the shared output keys, where LB.LBTESTCD = 'LYMPH'Compute WBCVAL * FRACVALConstant "CALCULATION"
derived_neutWBCVAL IS NOT NULL AND FRACVAL IS NOT NULL AND EXISTING_ABS_COUNT IS NULLConstant "NEUT"Constant "Neutrophils Abs (10^9/L)"Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'WBC'Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'NEUTLE'Aggregate COUNT(LB.*) over records matching on the shared output keys, where LB.LBTESTCD = 'NEUT'Compute WBCVAL * FRACVALConstant "CALCULATION"
derived_monoWBCVAL IS NOT NULL AND FRACVAL IS NOT NULL AND EXISTING_ABS_COUNT IS NULLConstant "MONO"Constant "Monocytes Abs (10^9/L)"Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'WBC'Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'MONOLE'Aggregate COUNT(LB.*) over records matching on the shared output keys, where LB.LBTESTCD = 'MONO'Compute WBCVAL * FRACVALConstant "CALCULATION"
derived_eosWBCVAL IS NOT NULL AND FRACVAL IS NOT NULL AND EXISTING_ABS_COUNT IS NULLConstant "EOS"Constant "Eosinophils Abs (10^9/L)"Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'WBC'Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'EOSLE'Aggregate COUNT(LB.*) over records matching on the shared output keys, where LB.LBTESTCD = 'EOS'Compute WBCVAL * FRACVALConstant "CALCULATION"
derived_basoWBCVAL IS NOT NULL AND FRACVAL IS NOT NULL AND EXISTING_ABS_COUNT IS NULLConstant "BASO"Constant "Basophils Abs (10^9/L)"Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'WBC'Aggregate ONLY(LB.LBSTRESN) over records matching on the shared output keys, where LB.LBTESTCD = 'BASOLE'Aggregate COUNT(LB.*) over records matching on the shared output keys, where LB.LBTESTCD = 'BASO'Compute WBCVAL * FRACVALConstant "CALCULATION"
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-adlb-lymphocytes/full.md

Edit
26 lines

Following CDISC ADaM standards, use the provided LB dataset (SDTM laboratory records) to create an ADLB dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: USUBJID, PARAMCD, AVAL, PARAM, VISIT, DTYPE

Keep every collected record unchanged, with DTYPE empty.

Add one absolute differential record per subject and visit for each differential fraction present: "LYMLE" becomes "LYMPH" ("Lymphocytes Abs (10^9/L)"), "NEUTLE" becomes "NEUT" ("Neutrophils Abs (10^9/L)"), "MONOLE" becomes "MONO" ("Monocytes Abs (10^9/L)"), "EOSLE" becomes "EOS" ("Eosinophils Abs (10^9/L)"), and "BASOLE" becomes "BASO" ("Basophils Abs (10^9/L)"). An absolute record is added only when the visit also has a white blood cell count (WBC) and no record of the new absolute parameter yet. Its value is the WBC count times the fraction for the same subject and visit, and DTYPE is CALCULATION.

A differential with a fraction but no WBC count gains nothing, and a visit that already holds the absolute record gains none for that differential.

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

prompts/adam-adlb-lymphocytes/conventions.md

Edit
16 lines

Following CDISC ADaM standards, use the provided LB dataset (SDTM laboratory records) to create an ADLB dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: USUBJID, PARAMCD, AVAL, PARAM, VISIT, DTYPE

The differential fraction codes are "LYMLE", "NEUTLE", "MONOLE", "EOSLE", and "BASOLE"; the absolute parameter codes and names are "LYMPH" / "Lymphocytes Abs (10^9/L)", "NEUT" / "Neutrophils Abs (10^9/L)", "MONO" / "Monocytes Abs (10^9/L)", "EOS" / "Eosinophils Abs (10^9/L)", and "BASO" / "Basophils Abs (10^9/L)". DTYPE is CALCULATION or empty.

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

prompts/adam-adlb-lymphocytes/brief.md

Edit
9 lines

Following CDISC ADaM standards, use the provided LB dataset (SDTM laboratory records) to create an ADLB dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: USUBJID, PARAMCD, AVAL, PARAM, VISIT, DTYPE

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)adlb <- yamaa_domain("spec.yaml")$outputadlb

run.py

Edit
4 lines
import yamaaadlb = yamaa.yamaa_domain("spec.yaml").outputadlb

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-adlb-lymphocytes/result.R

Edit
62 lines
# Reference solution for the yamaa benchmark adam-adlb-lymphocytes (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)raw <- read_csv(  "/app/input/lb.csv",  col_types = cols(LBSTRESN = col_double(), .default = col_character())) |>  rename(PARAMCD = LBTESTCD, AVAL = LBSTRESN, PARAM = LBTEST)# Keep every collected record unchanged, with DTYPE empty.collected <- raw |>  select(USUBJID, PARAMCD, AVAL, PARAM, VISIT) |>  mutate(DTYPE = NA_character_)# Differential fraction code -> absolute code and parameter name.differentials <- list(  LYMLE = c("LYMPH", "Lymphocytes Abs (10^9/L)"),  NEUTLE = c("NEUT", "Neutrophils Abs (10^9/L)"),  MONOLE = c("MONO", "Monocytes Abs (10^9/L)"),  EOSLE = c("EOS", "Eosinophils Abs (10^9/L)"),  BASOLE = c("BASO", "Basophils Abs (10^9/L)"))wbc <- raw |>  filter(PARAMCD == "WBC") |>  select(USUBJID, VISIT, WBCVAL = AVAL)# One absolute record per subject and visit for each differential# fraction present with a WBC count and no absolute record yet.calc <- bind_rows(lapply(names(differentials), function(frac_code) {  abs_code <- differentials[[frac_code]][1]  abs_param <- differentials[[frac_code]][2]  frac <- raw |>    filter(PARAMCD == frac_code) |>    select(USUBJID, VISIT, FRACVAL = AVAL)  existing <- raw |>    filter(PARAMCD == abs_code) |>    distinct(USUBJID, VISIT) |>    mutate(HAS_ABS = TRUE)  wbc |>    inner_join(frac, by = c("USUBJID", "VISIT")) |>    left_join(existing, by = c("USUBJID", "VISIT")) |>    filter(!is.na(WBCVAL), !is.na(FRACVAL), is.na(HAS_ABS)) |>    mutate(      PARAMCD = abs_code,      AVAL = WBCVAL * FRACVAL,      PARAM = abs_param,      DTYPE = "CALCULATION"    ) |>    select(USUBJID, PARAMCD, AVAL, PARAM, VISIT, DTYPE)}))adlb <- bind_rows(collected, calc)dir.create("/app/output", showWarnings = FALSE)write_csv(adlb, "/app/output/adlb.csv", na = "")

solutions/adam-adlb-lymphocytes/result.py

Edit
65 lines
# Reference solution for the yamaa benchmark adam-adlb-lymphocytes (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plraw = (    pl.read_csv("/app/input/lb.csv", infer_schema=False)    .with_columns(pl.col("LBSTRESN").cast(pl.Float64, strict=False))    .rename({"LBTESTCD": "PARAMCD", "LBSTRESN": "AVAL", "LBTEST": "PARAM"}))# Keep every collected record unchanged, with DTYPE empty.collected = raw.select("USUBJID", "PARAMCD", "AVAL", "PARAM", "VISIT").with_columns(    DTYPE=pl.lit(None, dtype=pl.String))# Differential fraction code -> (absolute code, absolute parameter name).differentials = {    "LYMLE": ("LYMPH", "Lymphocytes Abs (10^9/L)"),    "NEUTLE": ("NEUT", "Neutrophils Abs (10^9/L)"),    "MONOLE": ("MONO", "Monocytes Abs (10^9/L)"),    "EOSLE": ("EOS", "Eosinophils Abs (10^9/L)"),    "BASOLE": ("BASO", "Basophils Abs (10^9/L)"),}wbc = raw.filter(pl.col("PARAMCD") == "WBC").select("USUBJID", "VISIT", WBCVAL="AVAL")frames = []for frac_code, (abs_code, abs_param) in differentials.items():    frac = (        raw.filter(pl.col("PARAMCD") == frac_code)        .select("USUBJID", "VISIT", FRACVAL="AVAL")    )    existing = (        raw.filter(pl.col("PARAMCD") == abs_code)        .select("USUBJID", "VISIT")        .with_columns(pl.lit(True).alias("HAS_ABS"))    )    calc = (        wbc.join(frac, on=["USUBJID", "VISIT"], how="inner")        .join(existing, on=["USUBJID", "VISIT"], how="left")        .filter(            pl.col("WBCVAL").is_not_null()            & pl.col("FRACVAL").is_not_null()            & pl.col("HAS_ABS").is_null()        )        .with_columns(            PARAMCD=pl.lit(abs_code),            AVAL=pl.col("WBCVAL") * pl.col("FRACVAL"),            PARAM=pl.lit(abs_param),            DTYPE=pl.lit("CALCULATION"),        )        .select("USUBJID", "PARAMCD", "AVAL", "PARAM", "VISIT", "DTYPE")    )    frames.append(calc)adlb = pl.concat([collected, *frames], how="diagonal")Path("/app/output").mkdir(exist_ok=True)adlb.write_csv("/app/output/adlb.csv")

solutions/adam-adlb-lymphocytes/result.sas

Edit
27 lines
/* opensas reference solution for the yamaa benchmark adam-adlb-lymphocytes.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data lb;  length USUBJID LBTESTCD LBTEST VISIT $1024;  infile "/app/input/lb.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input USUBJID : $char1024. LBTESTCD : $char1024. LBSTRESN LBTEST : $char1024. VISIT : $char1024.;run;proc sql; create table derived asselect a.USUBJID,'LYMPH' as PARAMCD,a.LBSTRESN*b.LBSTRESN as AVAL,'Lymphocytes Abs (10^9/L)' as PARAM,a.VISIT,'CALCULATION' as DTYPE from lb as a inner join lb as b on a.USUBJID=b.USUBJID and a.VISIT=b.VISIT left join lb as c on a.USUBJID=c.USUBJID and a.VISIT=c.VISIT and c.LBTESTCD='LYMPH' where a.LBTESTCD='LYMLE' and b.LBTESTCD='WBC' and not missing(a.LBSTRESN) and not missing(b.LBSTRESN) and missing(c.LBTESTCD)union allselect a.USUBJID,'NEUT' as PARAMCD,a.LBSTRESN*b.LBSTRESN as AVAL,'Neutrophils Abs (10^9/L)' as PARAM,a.VISIT,'CALCULATION' as DTYPE from lb as a inner join lb as b on a.USUBJID=b.USUBJID and a.VISIT=b.VISIT left join lb as c on a.USUBJID=c.USUBJID and a.VISIT=c.VISIT and c.LBTESTCD='NEUT' where a.LBTESTCD='NEUTLE' and b.LBTESTCD='WBC' and not missing(a.LBSTRESN) and not missing(b.LBSTRESN) and missing(c.LBTESTCD)union allselect a.USUBJID,'MONO' as PARAMCD,a.LBSTRESN*b.LBSTRESN as AVAL,'Monocytes Abs (10^9/L)' as PARAM,a.VISIT,'CALCULATION' as DTYPE from lb as a inner join lb as b on a.USUBJID=b.USUBJID and a.VISIT=b.VISIT left join lb as c on a.USUBJID=c.USUBJID and a.VISIT=c.VISIT and c.LBTESTCD='MONO' where a.LBTESTCD='MONOLE' and b.LBTESTCD='WBC' and not missing(a.LBSTRESN) and not missing(b.LBSTRESN) and missing(c.LBTESTCD)union allselect a.USUBJID,'EOS' as PARAMCD,a.LBSTRESN*b.LBSTRESN as AVAL,'Eosinophils Abs (10^9/L)' as PARAM,a.VISIT,'CALCULATION' as DTYPE from lb as a inner join lb as b on a.USUBJID=b.USUBJID and a.VISIT=b.VISIT left join lb as c on a.USUBJID=c.USUBJID and a.VISIT=c.VISIT and c.LBTESTCD='EOS' where a.LBTESTCD='EOSLE' and b.LBTESTCD='WBC' and not missing(a.LBSTRESN) and not missing(b.LBSTRESN) and missing(c.LBTESTCD)union allselect a.USUBJID,'BASO' as PARAMCD,a.LBSTRESN*b.LBSTRESN as AVAL,'Basophils Abs (10^9/L)' as PARAM,a.VISIT,'CALCULATION' as DTYPE from lb as a inner join lb as b on a.USUBJID=b.USUBJID and a.VISIT=b.VISIT left join lb as c on a.USUBJID=c.USUBJID and a.VISIT=c.VISIT and c.LBTESTCD='BASO' where a.LBTESTCD='BASOLE' and b.LBTESTCD='WBC' and not missing(a.LBSTRESN) and not missing(b.LBSTRESN) and missing(c.LBTESTCD); quit;data collected; length PARAMCD $16 PARAM $1024 DTYPE $11; set lb; PARAMCD=LBTESTCD; PARAM=LBTEST; AVAL=LBSTRESN; run;data result; set collected derived; run;proc sql;  create table final as select USUBJID, PARAMCD, AVAL, PARAM, VISIT, DTYPE from result;quit;proc export data=final outfile="/app/output/adlb.csv" dbms=csv replace; run;