Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

  1. Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
  2. Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
  3. Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

Lifecycle: reviewedEditUnder review
What this benchmark means
README.md

Goal: carry each collected vital signs record through and add one body surface area (BSA) record for each subject and visit that has both a height and a weight result.

Input: collected vital signs with height records coded HEIGHT and weight records coded WEIGHT; height is collected in centimeters and weight in kilograms.

Variables:

  • AVAL keeps each collected result. On a body surface area record it is the Mosteller value: the square root of height in centimeters times weight in kilograms, divided by 3600. No record is added when the visit's height or weight is absent or has no result.
  • DTYPE is CALCULATION on a body surface area record and blank on a collected record.

Inputs

Under review
1 source file

input/advs.csv

Edit
13 rows
input/advs.csv
STUDYIDUSUBJIDPARAMCDPARAMAVALVISIT
YAMAA-01HEIGHTHeight (cm)170.5BASELINE
YAMAA-01WEIGHTWeight (kg)75.2BASELINE
YAMAA-01WEIGHTWeight (kg)76.0WEEK 2
YAMAA-01HEIGHTHeight (cm)165.0BASELINE
YAMAA-01WEIGHTWeight (kg)60.5BASELINE
YAMAA-01WEIGHTWeight (kg)80.0BASELINE
YAMAA-01HEIGHTHeight (cm)180.0BASELINE
YAMAA-01WEIGHTWeight (kg)—BASELINE
YAMAA-01HEIGHTHeight (cm)180.0WEEK 4
YAMAA-01WEIGHTWeight (kg)82.0WEEK 4
YAMAA-01HEIGHTHeight (cm)172.0BASELINE
YAMAA-01HEIGHTHeight (cm)175.0BASELINE
YAMAA-01WEIGHTWeight (kg)0.0BASELINE

Expected output

Under review
17 expected rows

expected/advs.csv

Edit
17 rows
expected/advs.csv
STUDYIDUSUBJIDPARAMCDPARAMAVALVISITDTYPE
YAMAA-01HEIGHTHeight (cm)170.5BASELINE—
YAMAA-01WEIGHTWeight (kg)75.2BASELINE—
YAMAA-01WEIGHTWeight (kg)76WEEK 2—
YAMAA-01HEIGHTHeight (cm)165BASELINE—
YAMAA-01WEIGHTWeight (kg)60.5BASELINE—
YAMAA-01WEIGHTWeight (kg)80BASELINE—
YAMAA-01HEIGHTHeight (cm)180BASELINE—
YAMAA-01WEIGHTWeight (kg)—BASELINE—
YAMAA-01HEIGHTHeight (cm)180WEEK 4—
YAMAA-01WEIGHTWeight (kg)82WEEK 4—
YAMAA-01HEIGHTHeight (cm)172BASELINE—
YAMAA-01HEIGHTHeight (cm)175BASELINE—
YAMAA-01WEIGHTWeight (kg)0BASELINE—
YAMAA-01BSABody Surface Area (m^2)1.887208402788509BASELINECALCULATION
YAMAA-01BSABody Surface Area (m^2)1.6652076947536205BASELINECALCULATION
YAMAA-01BSABody Surface Area (m^2)2.0248456731316584WEEK 4CALCULATION
YAMAA-01BSABody Surface Area (m^2)0BASELINECALCULATION
Shaded columns are derived— = missing value

Comments and assessment materials

Comments

Signed in with GitHub; saved in repository Discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADVSSTUDYIDStudy IdentifiercharacterCopy value from ADVS_RAW.STUDYID.Predecessor
ADVSUSUBJIDUnique Subject IdentifiercharacterCopy value from ADVS_RAW.USUBJID.Predecessor
ADVSPARAMCDParameter CodecharacterSet per constructed row; see the Row construction sheet.Derived
ADVSPARAMParametercharacterSet per constructed row; see the Row construction sheet.Derived
ADVSAVALAnalysis ValuenumericSet per constructed row; see the Row construction sheet.Derived
ADVSVISITVisit NamecharacterCopy value from ADVS_RAW.VISIT.Predecessor
ADVSDTYPEDerivation TypecharacterSet per constructed row; see the Row construction sheet.Derived
TemplateInclude whenPARAMCDPARAMHEIGHT_VALWEIGHT_VALAVALDTYPE
collectedCopy ADVS_RAW.PARAMCDCopy ADVS_RAW.PARAMConstant "None"Constant "None"Copy ADVS_RAW.AVALConstant "None"
derived_bsaHEIGHT_VAL IS NOT NULL AND WEIGHT_VAL IS NOT NULLConstant "BSA"Constant "Body Surface Area (m^2)"Aggregate ONLY(ADVS_RAW.AVAL) over records matching on the shared output keys, where ADVS_RAW.PARAMCD = 'HEIGHT'Aggregate ONLY(ADVS_RAW.AVAL) over records matching on the shared output keys, where ADVS_RAW.PARAMCD = 'WEIGHT'Compute SQRT(WEIGHT_VAL * HEIGHT_VAL / 3600)Constant "CALCULATION"
VersionDateAuthorDescriptionReviewerSign-off
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-advs-bsa/full.md

Edit
22 lines

Following CDISC ADaM standards, use the provided ADVS_RAW dataset to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE

Keep every collected record, and add one body surface area record for each subject and visit that has both a height and a weight result. Mark it with PARAMCD "BSA" and PARAM "Body Surface Area (m^2)".

AVAL keeps each collected result. On a body surface area record it is the Mosteller value, the square root of the visit's height in centimeters times its weight in kilograms divided by 3600, not rounded; a zero result still counts as a result. No record is added when the visit's height or weight is absent or has no result.

DTYPE is CALCULATION on a body surface area record and has no value on a collected record.

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

prompts/adam-advs-bsa/conventions.md

Edit
16 lines

Following CDISC ADaM standards, use the provided ADVS_RAW dataset to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE

Mark a body surface area record with PARAMCD "BSA" and PARAM "Body Surface Area (m^2)".

On a body surface area record AVAL is the Mosteller value, not rounded.

DTYPE is CALCULATION or has no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

prompts/adam-advs-bsa/brief.md

Edit
9 lines

Following CDISC ADaM standards, use the provided ADVS_RAW dataset to create an ADVS dataset with one record per subject per visit per parameter.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE

Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.

yamaa code

Runs the yamaa specification; its result must match the expected output

run.R

Edit
4 lines
library(yamaa)advs <- yamaa_domain("spec.yaml")$outputadvs

run.py

Edit
4 lines
import yamaaadvs = yamaa.yamaa_domain("spec.yaml").outputadvs

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-advs-bsa/result.R

Edit
44 lines
# Reference solution for the yamaa benchmark adam-advs-bsa (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)raw <- read_csv(  "/app/input/advs.csv",  col_types = cols(AVAL = col_double(), .default = col_character()))# Every collected record stays in place.collected <- raw |>  transmute(STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE = NA_character_)# The visit's height and weight results; a zero result still counts.heights <- raw |>  filter(PARAMCD %in% "HEIGHT", !is.na(AVAL)) |>  group_by(STUDYID, USUBJID, VISIT) |>  summarise(HEIGHT_VAL = first(AVAL), .groups = "drop")weights <- raw |>  filter(PARAMCD %in% "WEIGHT", !is.na(AVAL)) |>  group_by(STUDYID, USUBJID, VISIT) |>  summarise(WEIGHT_VAL = first(AVAL), .groups = "drop")# One body surface area record for each visit with both results.bsa <- heights |>  inner_join(weights, by = c("STUDYID", "USUBJID", "VISIT")) |>  transmute(    STUDYID, USUBJID,    PARAMCD = "BSA", PARAM = "Body Surface Area (m^2)",    AVAL = sqrt(HEIGHT_VAL * WEIGHT_VAL / 3600),    VISIT, DTYPE = "CALCULATION"  )advs <- bind_rows(collected, bsa) |>  arrange(USUBJID, VISIT, PARAMCD)dir.create("/app/output", showWarnings = FALSE)write_csv(advs, "/app/output/advs.csv", na = "")

solutions/adam-advs-bsa/result.py

Edit
58 lines
# Reference solution for the yamaa benchmark adam-advs-bsa (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plraw = pl.read_csv("/app/input/advs.csv", infer_schema=False).with_columns(    pl.col("AVAL").cast(pl.Float64, strict=False))# Every collected record stays in place.collected = raw.select(    "STUDYID", "USUBJID", "PARAMCD", "PARAM", "AVAL", "VISIT",    DTYPE=pl.lit(None, dtype=pl.String),)# The visit's height and weight results; a zero result still counts.heights = (    raw.filter(        (pl.col("PARAMCD") == "HEIGHT") & pl.col("AVAL").is_not_null()    )    .group_by(["STUDYID", "USUBJID", "VISIT"])    .agg(HEIGHT_VAL=pl.col("AVAL").first()))weights = (    raw.filter(        (pl.col("PARAMCD") == "WEIGHT") & pl.col("AVAL").is_not_null()    )    .group_by(["STUDYID", "USUBJID", "VISIT"])    .agg(WEIGHT_VAL=pl.col("AVAL").first()))# One body surface area record for each visit with both results.bsa = (    heights.join(weights, on=["STUDYID", "USUBJID", "VISIT"], how="inner")    .with_columns(AVAL=(pl.col("HEIGHT_VAL") * pl.col("WEIGHT_VAL") / 3600).sqrt())    .select(        "STUDYID",        "USUBJID",        "AVAL",        "VISIT",        PARAMCD=pl.lit("BSA"),        PARAM=pl.lit("Body Surface Area (m^2)"),        DTYPE=pl.lit("CALCULATION"),    )    .select("STUDYID", "USUBJID", "PARAMCD", "PARAM", "AVAL", "VISIT", "DTYPE"))advs = pl.concat([collected, bsa], how="diagonal").sort(    ["USUBJID", "VISIT", "PARAMCD"])Path("/app/output").mkdir(parents=True, exist_ok=True)advs.write_csv("/app/output/advs.csv")

solutions/adam-advs-bsa/result.sas

Edit
21 lines
/* opensas reference solution for the yamaa benchmark adam-advs-bsa.   Reads the benchmark inputs and derives the requested output.   Execute with the existing opensas runtime: opensas /app/output/result.sas. */data advs;  length STUDYID USUBJID PARAMCD PARAM VISIT $1024;  infile "/app/input/advs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;  input STUDYID : $char1024. USUBJID : $char1024. PARAMCD : $char1024. PARAM : $char1024. AVAL VISIT : $char1024.;run;proc sql;  create table computed as select a.STUDYID,a.USUBJID,'BSA' as PARAMCD,'Body Surface Area (m^2)' as PARAM,sqrt(a.AVAL*b.AVAL/3600) as AVAL,a.VISIT,'CALCULATION' as DTYPE from advs as a inner join advs as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.VISIT=b.VISIT  where a.PARAMCD='HEIGHT' and b.PARAMCD='WEIGHT' and not missing(a.AVAL) and not missing(b.AVAL);quit;data collected; length DTYPE $11; set advs; run;data result; set collected computed; run;proc sql;  create table final as select STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE from result;quit;proc export data=final outfile="/app/output/advs.csv" dbms=csv replace; run;