yamaa As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
EditUnder reviewWhat this benchmark meansREADME.mdGoal: derive PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM,
SRCVAR, and SRCSEQ for each qualifying temperature record in an
ADaM analysis dataset (ADFA).
Input: one row per collected Vital Signs (VS) result, carrying the test code, the category, the numeric result, the unit, and the date and time of the assessment.
Variables:
PARAMCDandPARAMare alwaysFEVERandFever Occurrence.AVALCisYfor a temperature of 38 degrees Celsius or higher andNfor a lower one; blank when the result is missing or its unit is not Celsius (C).AVALis 1 when the occurrence flag isY, 0 when it isN, and blank when the flag is blank.ASEQnumbers the subject's analysis records 1, 2, 3 ... in assessment-date order, the collected sequence breaking ties.ADTis the calendar date of the assessment.SRCDOM,SRCVAR, andSRCSEQpoint back to the source result: alwaysVS,VSSTRESN, and the collected sequence number.
Note: only temperature records in the reactogenicity category qualify; other vital signs records give no row.
Inputs
Under review1 source fileinput/vs.csv
Edit7 rows| STUDYID | USUBJID | VSSEQ | VSTESTCD | VSCAT | VSSTRESN | VSSTRESU | VSDTC |
|---|---|---|---|---|---|---|---|
| PROJ1 | 1 | TEMP | REACTOGENICITY | 38.5 | C | 2024-01-15T08:30 | |
| PROJ1 | 2 | TEMP | REACTOGENICITY | 37.5 | C | 2024-01-16T08:30 | |
| PROJ1 | 3 | TEMP | REACTOGENICITY | — | C | 2024-01-17T08:30 | |
| PROJ1 | 4 | SYSBP | REACTOGENICITY | 120 | mmHg | 2024-01-15T08:30 | |
| PROJ1 | 1 | TEMP | REACTOGENICITY | 101.3 | F | 2024-01-15T09:00 | |
| PROJ1 | 2 | TEMP | OTHER | 38.1 | C | 2024-01-16T09:00 | |
| PROJ1 | 3 | TEMP | REACTOGENICITY | 39.2 | C | 2024-01-17T09:00 |
Expected output
Under review5 expected rowsexpected/adfa.csv
Edit5 rows| DOMAIN | STUDYID | USUBJID | ASEQ | PARAMCD | PARAM | AVAL | AVALC | ADT | SRCDOM | SRCVAR | SRCSEQ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ADFA | PROJ1 | 1 | FEVER | Fever Occurrence | 1 | Y | 2024-01-15 | VS | VSSTRESN | 1 | |
| ADFA | PROJ1 | 2 | FEVER | Fever Occurrence | 0 | N | 2024-01-16 | VS | VSSTRESN | 2 | |
| ADFA | PROJ1 | 3 | FEVER | Fever Occurrence | — | — | 2024-01-17 | VS | VSSTRESN | 3 | |
| ADFA | PROJ1 | 1 | FEVER | Fever Occurrence | — | — | 2024-01-15 | VS | VSSTRESN | 1 | |
| ADFA | PROJ1 | 2 | FEVER | Fever Occurrence | 1 | Y | 2024-01-17 | VS | VSSTRESN | 3 |
Comments and assessment materials
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADFA | DOMAIN | Domain Abbreviation | character | Set constant value "ADFA". | Assigned | ||
| ADFA | STUDYID | Study Identifier | character | Copy value from VS.STUDYID. | Predecessor | ||
| ADFA | USUBJID | Unique Subject Identifier | character | Copy value from VS.USUBJID. | Predecessor | ||
| ADFA | ASEQ | Analysis Sequence Number | numeric | Row number within each group of STUDYID, USUBJID, ordered by VS.VSDTC then VS.VSSEQ. | Derived | ||
| ADFA | ADT | Analysis Date | numeric | ISO 8601 | Take the date part of VS.VSDTC. | Derived | |
| ADFA | PARAMCD | Parameter Code | character | Set constant value "FEVER". | Assigned | ||
| ADFA | PARAM | Parameter | character | Set constant value "Fever Occurrence". | Assigned | ||
| ADFA | AVALC | Character Analysis Value | character | If VS.VSSTRESU = 'C' AND VS.VSSTRESN >= 38 then "Y"; If VS.VSSTRESU = 'C' AND VS.VSSTRESN < 38 then "N"; otherwise blank. | Derived | ||
| ADFA | AVAL | Analysis Value | numeric | If AVALC = 'Y' then "1"; If AVALC = 'N' then "0"; otherwise blank. | Derived | ||
| ADFA | SRCDOM | Source Domain | character | Set constant value "VS". | Assigned | ||
| ADFA | SRCVAR | Source Variable | character | Set constant value "VSSTRESN". | Assigned | ||
| ADFA | SRCSEQ | Source Sequence Number | numeric | Copy value from VS.VSSEQ. | Predecessor |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-adfa-fever/full.md
Edit21 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADFA dataset with one record per collected temperature record in the reactogenicity category.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ASEQ, PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM, SRCVAR, SRCSEQ
PARAMCD is FEVER and PARAM is Fever Occurrence on every record. AVALC is Y when the collected temperature is 38 degrees Celsius or higher and N when it is lower; it has no value when the result was not collected or its unit is not Celsius. AVAL is 1 when AVALC is Y and 0 when AVALC is N, and has no value when AVALC has no value. ASEQ numbers the subject's records 1, 2, 3 and so on in assessment-date order, with the collected sequence number breaking ties on the same date. ADT is the calendar date of the assessment. SRCDOM, SRCVAR and SRCSEQ trace each record to its source: the VS domain, the VSSTRESN variable, and the collected sequence number.
Read the source datasets from /app/input and save the completed dataset as /app/output/adfa.csv.
prompts/adam-adfa-fever/conventions.md
Edit15 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADFA dataset with one record per collected temperature record in the reactogenicity category.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ASEQ, PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM, SRCVAR, SRCSEQ
AVALC is Y at 38 degrees Celsius or higher and N below it, blank when the result is missing or not in Celsius. AVAL is 1 for Y and 0 for N. ASEQ numbers the subject's records from 1 in assessment-date order, ties broken by the collected sequence number.
Read the source datasets from /app/input and save the completed dataset as /app/output/adfa.csv.
prompts/adam-adfa-fever/brief.md
Edit10 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADFA dataset with one record per collected temperature record in the reactogenicity category.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, ASEQ, PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM, SRCVAR, SRCSEQ
Read the source datasets from /app/input and save the completed dataset as /app/output/adfa.csv.
run.R
Edit4 lineslibrary(yamaa)adfa <- yamaa_domain("spec.yaml")$outputadfarun.py
Edit4 linesimport yamaaadfa = yamaa.yamaa_domain("spec.yaml").outputadfasolutions/adam-adfa-fever/result.R
Edit52 lines# Reference solution for the yamaa benchmark adam-adfa-fever (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)vs <- read_csv( "/app/input/vs.csv", col_types = cols( VSSEQ = col_integer(), VSSTRESN = col_double(), .default = col_character() ))# Only temperature records in the reactogenicity category qualify.adfa <- vs |> filter(VSTESTCD == "TEMP", VSCAT == "REACTOGENICITY") |> mutate( DOMAIN = "ADFA", PARAMCD = "FEVER", PARAM = "Fever Occurrence", AVALC = case_when( VSSTRESU == "C" & VSSTRESN >= 38 ~ "Y", VSSTRESU == "C" & VSSTRESN < 38 ~ "N" ), AVAL = case_when( AVALC == "Y" ~ 1, AVALC == "N" ~ 0 ), ADT = as.Date(substr(VSDTC, 1, 10)), SRCDOM = "VS", SRCVAR = "VSSTRESN", SRCSEQ = VSSEQ ) |> # Number each subject's records 1, 2, 3 ... in assessment-date order, # the collected sequence breaking ties on the same date. arrange(STUDYID, USUBJID, ADT, VSSEQ) |> group_by(STUDYID, USUBJID) |> mutate(ASEQ = row_number()) |> ungroup() |> arrange(USUBJID, ASEQ) |> select( DOMAIN, STUDYID, USUBJID, ASEQ, PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM, SRCVAR, SRCSEQ )dir.create("/app/output", showWarnings = FALSE)write_csv(adfa, "/app/output/adfa.csv", na = "")solutions/adam-adfa-fever/result.py
Edit62 lines# Reference solution for the yamaa benchmark adam-adfa-fever (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plvs = pl.read_csv("/app/input/vs.csv", infer_schema=False).with_columns( pl.col("VSSEQ").cast(pl.Int64), pl.col("VSSTRESN").cast(pl.Float64, strict=False),)# Only temperature records in the reactogenicity category qualify.temp = vs.filter( (pl.col("VSTESTCD") == "TEMP") & (pl.col("VSCAT") == "REACTOGENICITY")).with_columns( pl.when((pl.col("VSSTRESU") == "C") & (pl.col("VSSTRESN") >= 38)) .then(pl.lit("Y")) .when((pl.col("VSSTRESU") == "C") & (pl.col("VSSTRESN") < 38)) .then(pl.lit("N")) .alias("AVALC"),)adfa = ( temp.with_columns( pl.lit("ADFA").alias("DOMAIN"), pl.lit("FEVER").alias("PARAMCD"), pl.lit("Fever Occurrence").alias("PARAM"), pl.when(pl.col("AVALC") == "Y") .then(1.0) .when(pl.col("AVALC") == "N") .then(0.0) .alias("AVAL"), pl.col("VSDTC").str.slice(0, 10).str.to_date(strict=False).alias("ADT"), pl.lit("VS").alias("SRCDOM"), pl.lit("VSSTRESN").alias("SRCVAR"), pl.col("VSSEQ").alias("SRCSEQ"), ) .sort(["STUDYID", "USUBJID", "ADT", "VSSEQ"], nulls_last=True) # Number each subject's records 1, 2, 3 ... in assessment-date order. .with_columns(pl.col("USUBJID").cum_count().over(["STUDYID", "USUBJID"]).alias("ASEQ")) .select( "DOMAIN", "STUDYID", "USUBJID", "ASEQ", "PARAMCD", "PARAM", "AVAL", "AVALC", "ADT", "SRCDOM", "SRCVAR", "SRCSEQ", ))Path("/app/output").mkdir(exist_ok=True)adfa.write_csv("/app/output/adfa.csv")solutions/adam-adfa-fever/result.sas
Edit28 lines/* opensas reference solution for the yamaa benchmark adam-adfa-fever. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data vs; length STUDYID USUBJID VSTESTCD VSCAT VSSTRESU VSDTC $1024; infile "/app/input/vs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ VSTESTCD : $char1024. VSCAT : $char1024. VSSTRESN VSSTRESU : $char1024. VSDTC : $char1024.;run;data dated; set vs; length ADT $10; ADT=substr(VSDTC,1,10); NODATE=missing(ADT); run;proc sort data=dated(where=(VSTESTCD='TEMP' and VSCAT='REACTOGENICITY')) out=ordered; by STUDYID USUBJID NODATE ADT VSSEQ; run;data result; length DOMAIN $4 PARAMCD $16 PARAM $40 AVALC $1 ADT $10 SRCDOM $2 SRCVAR $16; set ordered; by STUDYID USUBJID; if first.USUBJID then ASEQ=0; ASEQ+1; DOMAIN='ADFA'; PARAMCD='FEVER'; PARAM='Fever Occurrence'; ADT=substr(VSDTC,1,10); SRCDOM='VS'; SRCVAR='VSSTRESN'; SRCSEQ=VSSEQ; if not missing(VSSTRESN) and VSSTRESU='C' then do; if VSSTRESN>=38 then do; AVAL=1; AVALC='Y'; end; else do; AVAL=0; AVALC='N'; end; end;run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, ASEQ, PARAMCD, PARAM, AVAL, AVALC, ADT, SRCDOM, SRCVAR, SRCSEQ from result;quit;proc export data=final outfile="/app/output/adfa.csv" dbms=csv replace; run;
Comments
Signed in with GitHub; saved in repository Discussions