Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: reviewedEditUnder review
What this benchmark means
README.md

Goal: turn each collected body measurement (BMI or WEIGHT) into one growth-percentile record holding the percentile in AVAL; the collected records are not kept.

Input: collected measurements carrying the measurement code (PARAMCD), the collected result (AVAL, stored as text), sex (SEX), and age in days (AGE); alongside a sex-and-age growth reference carrying the coefficients L, M, and S for each measurement code, sex, and age in days.

Variables:

  • PARAMCD: BMIPCTL for a collected BMI measurement, WGTPCTL for a collected WEIGHT measurement.
  • PARAM: BMI-for-Age Percentile / Weight-for-Age Percentile, matching PARAMCD.
  • AVAL: the growth percentile, 100 times the standard normal cumulative probability of the LMS z-score: the collected result divided by M, raised to the power L, minus 1, all divided by L times S. The L, M, and S are the reference coefficients matched on measurement code, sex, and age in days. A measurement with no matching reference row, or with no collected result, leaves AVAL missing.

Note: a collected result exactly at the reference median (M) has a z-score of 0 and so a percentile of exactly 50.

Inputs

Under review
2 source files

input/advs_raw.csv

Edit
8 rows
input/advs_raw.csv
STUDYIDUSUBJIDAVISITPARAMCDAVALAGESEX
YAMAA-01MONTH 12BMI16.2365F
YAMAA-01MONTH 12WEIGHT9.1365F
YAMAA-01MONTH 12BMI17.5365M
YAMAA-01MONTH 12WEIGHT10.0365M
YAMAA-01MONTH 24BMI17.0730F
YAMAA-01MONTH 24WEIGHT12.4730F
YAMAA-01MONTH 12BMI16.0365F
YAMAA-01MONTH 12BMI—365F

input/lms_ref.csv

Edit
4 rows
input/lms_ref.csv
PARAMCDSEXAGELMS
BMIF365-0.816.00.08
BMIM365-0.716.50.08
WEIGHTF365-0.49.00.10
WEIGHTM365-0.39.60.10

Expected output

Under review
8 expected rows

expected/advs.csv

Edit
8 rows
expected/advs.csv
STUDYIDUSUBJIDAVISITPARAMCDPARAMAVAL
YAMAA-01MONTH 12BMIPCTLBMI-for-Age Percentile56.13971541306226
YAMAA-01MONTH 12WGTPCTLWeight-for-Age Percentile54.389623884577375
YAMAA-01MONTH 12BMIPCTLBMI-for-Age Percentile76.4411385545167
YAMAA-01MONTH 12WGTPCTLWeight-for-Age Percentile65.75296725058702
YAMAA-01MONTH 24BMIPCTLBMI-for-Age Percentile—
YAMAA-01MONTH 24WGTPCTLWeight-for-Age Percentile—
YAMAA-01MONTH 12BMIPCTLBMI-for-Age Percentile50
YAMAA-01MONTH 12BMIPCTLBMI-for-Age Percentile—
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADVSSTUDYIDStudy IdentifiercharacterCopy value from ADVS_RAW.STUDYID.Predecessor
ADVSUSUBJIDUnique Subject IdentifiercharacterCopy value from ADVS_RAW.USUBJID.Predecessor
ADVSAVISITAnalysis VisitcharacterCopy value from ADVS_RAW.AVISIT.Predecessor
ADVSPARAMCDParameter CodecharacterBMIPCTL, WGTPCTLIf ADVS_RAW.PARAMCD = 'BMI' then "BMIPCTL"; If ADVS_RAW.PARAMCD = 'WEIGHT' then "WGTPCTL"; otherwise blank.Derived
ADVSPARAMParametercharacterIf PARAMCD = 'BMIPCTL' then "BMI-for-Age Percentile"; If PARAMCD = 'WGTPCTL' then "Weight-for-Age Percentile"; otherwise blank.Derived
ADVSAVALAnalysis ValuenumericCompute 100 * NORMCDF.Derived
VariablePermitted values
PARAMCDBMIPCTL, WGTPCTL
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-advs-percentiles/full.md

Edit
23 lines

Following CDISC ADaM standards, use the provided ADVS_RAW and LMS datasets to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AVISIT, PARAMCD, PARAM, AVAL

Turn each collected body measurement into one growth-percentile record; the collected records are not kept. PARAMCD is "BMIPCTL" for a collected "BMI" measurement and "WGTPCTL" for a collected "WEIGHT" measurement, with PARAM "BMI-for-Age Percentile" and "Weight-for-Age Percentile" to match.

AVAL is the growth percentile for the collected result against the reference coefficients L, M, and S matched on measurement code, sex, and age in days: 100 times the standard normal cumulative probability of the LMS z-score ((result / M) ^ L - 1) / (L * S), not rounded. A measurement with no matching reference row, or with no collected result, leaves AVAL with no value. A collected result exactly at the reference median gives 50.

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

prompts/adam-advs-percentiles/conventions.md

Edit
15 lines

Following CDISC ADaM standards, use the provided ADVS_RAW and LMS datasets to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AVISIT, PARAMCD, PARAM, AVAL

PARAMCD is "BMIPCTL" for a collected "BMI" measurement and "WGTPCTL" for a collected "WEIGHT" measurement, with PARAM "BMI-for-Age Percentile" and "Weight-for-Age Percentile" to match.

AVAL is the growth percentile, not rounded.

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

prompts/adam-advs-percentiles/brief.md

Edit
9 lines

Following CDISC ADaM standards, use the provided ADVS_RAW and LMS datasets to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AVISIT, PARAMCD, PARAM, AVAL

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)advs <- yamaa_domain("spec.yaml", environment = "r/environment.yaml")$outputadvs

run.py

Edit
4 lines
import yamaaadvs = yamaa.domain("spec.yaml", environment="python/environment.yaml").outputadvs

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-advs-percentiles/result.R

Edit
39 lines
# Reference solution for the yamaa benchmark adam-advs-percentiles (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)raw <- read_csv(  "/app/input/advs_raw.csv",  col_types = cols(AVAL = col_double(), AGE = col_integer(), .default = col_character()))ref <- read_csv(  "/app/input/lms_ref.csv",  col_types = cols(    AGE = col_integer(), L = col_double(), M = col_double(), S = col_double(),    .default = col_character()  ))# The growth percentile against the matched L, M, and S coefficients.advs <- raw |>  left_join(ref, by = c("PARAMCD", "SEX", "AGE")) |>  mutate(    Z = case_when(      is.na(AVAL) | is.na(L) | is.na(M) | is.na(S) | M == 0 | S == 0 ~ NA_real_,      L == 0 ~ log(AVAL / M) / S,      .default = ((AVAL / M) ^ L - 1) / (L * S)    ),    AVAL = 100 * pnorm(Z),    PARAMCD = if_else(PARAMCD == "BMI", "BMIPCTL", "WGTPCTL"),    PARAM = if_else(PARAMCD == "BMIPCTL", "BMI-for-Age Percentile", "Weight-for-Age Percentile")  ) |>  arrange(USUBJID, AVISIT, PARAMCD) |>  select(STUDYID, USUBJID, AVISIT, PARAMCD, PARAM, AVAL)dir.create("/app/output", showWarnings = FALSE)write_csv(advs, "/app/output/advs.csv", na = "")

solutions/adam-advs-percentiles/result.py

Edit
68 lines
# Reference solution for the yamaa benchmark adam-advs-percentiles (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.import mathfrom pathlib import Pathimport polars as plraw = pl.read_csv("/app/input/advs_raw.csv", infer_schema=False).with_columns(    pl.col("AVAL").cast(pl.Float64, strict=False),    pl.col("AGE").cast(pl.Int64, strict=False),)ref = pl.read_csv("/app/input/lms_ref.csv", infer_schema=False).with_columns(    pl.col("AGE").cast(pl.Int64, strict=False),    pl.col("L").cast(pl.Float64, strict=False),    pl.col("M").cast(pl.Float64, strict=False),    pl.col("S").cast(pl.Float64, strict=False),)def percentile(aval: float | None, ell: float | None, emm: float | None, ess: float | None) -> float | None:    if aval is None or ell is None or emm is None or ess is None:        return None    if emm == 0 or ess == 0:        return None    if ell == 0:        z = math.log(aval / emm) / ess    else:        z = ((aval / emm) ** ell - 1) / (ell * ess)    return 100 * 0.5 * (1 + math.erf(z / math.sqrt(2)))joined = raw.join(ref, on=["PARAMCD", "SEX", "AGE"], how="left")rows = []for record in joined.sort(["USUBJID", "AVISIT", "PARAMCD"]).iter_rows(named=True):    if record["PARAMCD"] == "BMI":        out_code, out_label = "BMIPCTL", "BMI-for-Age Percentile"    else:        out_code, out_label = "WGTPCTL", "Weight-for-Age Percentile"    rows.append(        {            "STUDYID": record["STUDYID"],            "USUBJID": record["USUBJID"],            "AVISIT": record["AVISIT"],            "PARAMCD": out_code,            "PARAM": out_label,            "AVAL": percentile(record["AVAL"], record["L"], record["M"], record["S"]),        }    )advs = pl.DataFrame(    rows,    schema={        "STUDYID": pl.String,        "USUBJID": pl.String,        "AVISIT": pl.String,        "PARAMCD": pl.String,        "PARAM": pl.String,        "AVAL": pl.Float64,    },)Path("/app/output").mkdir(parents=True, exist_ok=True)advs.write_csv("/app/output/advs.csv")

solutions/adam-advs-percentiles/result.sas

Edit
35 lines
/* opensas reference solution for the yamaa benchmark adam-advs-percentiles.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data advs_raw;  length STUDYID USUBJID AVISIT PARAMCD SEX $1024;  infile "/app/input/advs_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. AVISIT : $char1024. PARAMCD : $char1024. AVAL AGE SEX : $char1024.;run;data lms_ref;  length PARAMCD SEX $1024;  infile "/app/input/lms_ref.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input PARAMCD : $char1024. SEX : $char1024. AGE L M S;run;proc sql;  create table joined as select a.*,b.L,b.M,b.S from advs_raw as a left join lms_ref as b on a.PARAMCD=b.PARAMCD and a.SEX=b.SEX and a.AGE=b.AGE;quit;data result;  length PARAM $40;  set joined;  value=AVAL; AVAL=.;  if not missing(value) and not missing(M) and not missing(L) and not missing(S) then do;    if L=0 then z=log(value/M)/S; else z=((value/M)**L-1)/(L*S);    AVAL=100*probnorm(z);  end;  if PARAMCD='BMI' then do; PARAMCD='BMIPCTL'; PARAM='BMI-for-Age Percentile'; end;  else if PARAMCD='WEIGHT' then do; PARAMCD='WGTPCTL'; PARAM='Weight-for-Age Percentile'; end;run;proc sql;  create table final as select STUDYID, USUBJID, AVISIT, PARAMCD, PARAM, AVAL from result;quit;proc export data=final outfile="/app/output/advs.csv" dbms=csv replace; run;