As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: build one ADVS row per collected character-result row,
carrying SERIES, AVISITN, and AVALC through unchanged and
adding PREVAVALC for the closest earlier non-blank result.
Input: collected character results for three subjects, with series, visit number, and character result.
Variables:
SERIESidentifies the analysis series; rows with no series value share one series.AVISITNis the visit number that orders rows within a series.AVALCis the current character result, kept as collected; blank when no result was collected.PREVAVALCis the closest earlier non-blank result for the same subject and series, never the row's own; blank when no earlier row in the series has a result.
Note: within a series, rows are ordered by visit number with
missing numbers last, and rows sharing a visit number (or both
missing one) keep their collected order. The look-back skips blank
results, so it can cross consecutive blank rows, but it never
crosses into another series. A subject with a single collected row,
and a series whose earlier rows are all blank, both leave PREVAVALC
blank.
Inputs
Under review1 source fileinput/vs.csv
Edit14 rows| STUDYID | USUBJID | VSSEQ | SERIES | AVISITN | AVALC |
|---|---|---|---|---|---|
| YAMAA-01 | 1 | A | — | NULL-ORDER | |
| YAMAA-01 | 2 | A | 1 | FIRST | |
| YAMAA-01 | 3 | A | 1 | — | |
| YAMAA-01 | 4 | A | 2 | — | |
| YAMAA-01 | 5 | B | 1 | B-FIRST | |
| YAMAA-01 | 6 | B | 2 | — | |
| YAMAA-01 | 7 | — | 1 | UNGROUPED | |
| YAMAA-01 | 8 | — | 2 | — | |
| YAMAA-01 | 9 | A | 1 | SOLO | |
| YAMAA-01 | 10 | C | 1 | — | |
| YAMAA-01 | 11 | C | 2 | — | |
| YAMAA-01 | 12 | C | 3 | LATE | |
| YAMAA-01 | 13 | C | — | — | |
| YAMAA-01 | 14 | C | — | — |
Expected output
Under review14 expected rowsexpected/advs.csv
Edit14 rows| STUDYID | USUBJID | VSSEQ | SERIES | AVISITN | AVALC | PREVAVALC |
|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | A | — | NULL-ORDER | FIRST | |
| YAMAA-01 | 2 | A | 1 | FIRST | — | |
| YAMAA-01 | 3 | A | 1 | — | FIRST | |
| YAMAA-01 | 4 | A | 2 | — | FIRST | |
| YAMAA-01 | 5 | B | 1 | B-FIRST | — | |
| YAMAA-01 | 6 | B | 2 | — | B-FIRST | |
| YAMAA-01 | 7 | — | 1 | UNGROUPED | — | |
| YAMAA-01 | 8 | — | 2 | — | UNGROUPED | |
| YAMAA-01 | 9 | A | 1 | SOLO | — | |
| YAMAA-01 | 10 | C | 1 | — | — | |
| YAMAA-01 | 11 | C | 2 | — | — | |
| YAMAA-01 | 12 | C | 3 | LATE | — | |
| YAMAA-01 | 13 | C | — | — | LATE | |
| YAMAA-01 | 14 | C | — | — | LATE |
Comments and assessment materials
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADVS | STUDYID | Study Identifier | character | Copy value from VS.STUDYID. | Predecessor | ||
| ADVS | USUBJID | Unique Subject Identifier | character | Copy value from VS.USUBJID. | Predecessor | ||
| ADVS | VSSEQ | Sequence Number | numeric | Copy value from VS.VSSEQ. | Predecessor | ||
| ADVS | SERIES | Analysis Series | character | Copy value from VS.SERIES. | Predecessor | ||
| ADVS | AVISITN | Analysis Visit Number | numeric | Copy value from VS.AVISITN. | Predecessor | ||
| ADVS | AVALC | Analysis Value (C) | character | Copy value from VS.AVALC. | Predecessor | ||
| ADVS | PREVAVALC | Previous Non-Missing Analysis Value (C) | character | Most recent non-missing AVALC from an earlier row within each group of STUDYID, USUBJID, SERIES, ordered by AVISITN then VSSEQ. | Derived |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-advs-prior-result/full.md
Edit20 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per vital-signs record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
SERIES identifies the analysis series; rows with no series value share one series. AVISITN is the visit number that orders rows within a series. AVALC is the current character result, kept as collected, and has no value when no result was collected.
PREVAVALC is the closest earlier result with a value for the same subject and series, never the row's own; it has no value when no earlier row in the series has a result. Within a series, rows are ordered by visit number with missing numbers last, and rows sharing a visit number, or both missing one, keep their collected order. The look-back skips blank results but never crosses into another series.
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
prompts/adam-advs-prior-result/conventions.md
Edit8 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per vital-signs record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
prompts/adam-advs-prior-result/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per vital-signs record.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
run.R
Edit4 lineslibrary(yamaa)advs <- yamaa_domain("spec.yaml")$outputadvsrun.py
Edit4 linesimport yamaaadvs = yamaa.yamaa_domain("spec.yaml").outputadvssolutions/adam-advs-prior-result/result.R
Edit40 lines# Reference solution for the yamaa benchmark adam-advs-prior-result (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)library(tidyr)vs <- read_csv( "/app/input/vs.csv", col_types = cols( VSSEQ = col_integer(), AVISITN = col_integer(), .default = col_character() ))# Within a series, rows are ordered by visit number with missing numbers# last, keeping the collected order for ties. The look-back skips blank# results but never crosses into another series.ordered <- vs |> arrange(STUDYID, USUBJID, SERIES, is.na(AVISITN), AVISITN)# Forward-fill the ordered results, then lag by one so the row's own# value never counts.prev <- ordered |> group_by(STUDYID, USUBJID, SERIES) |> fill(AVALC, .direction = "down") |> mutate(PREVAVALC = lag(AVALC)) |> ungroup() |> select(STUDYID, USUBJID, VSSEQ, PREVAVALC)advs <- vs |> left_join(prev, by = c("STUDYID", "USUBJID", "VSSEQ")) |> arrange(USUBJID, VSSEQ) |> select(STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC)dir.create("/app/output", showWarnings = FALSE)write_csv(advs, "/app/output/advs.csv", na = "")solutions/adam-advs-prior-result/result.py
Edit46 lines# Reference solution for the yamaa benchmark adam-advs-prior-result (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plvs = pl.read_csv("/app/input/vs.csv", infer_schema=False).with_columns( pl.col("VSSEQ").cast(pl.Int64, strict=False), pl.col("AVISITN").cast(pl.Int64, strict=False),)# Rows with no series value share one series within each subject; order# within a series by visit number with missing numbers last, keeping the# collected order (input row order) for ties.ordered = vs.sort( ["STUDYID", "USUBJID", "SERIES", "AVISITN"], nulls_last=True, maintain_order=True,)# The closest earlier result with a value in the same subject and series.prev_by_key: dict[tuple, str | None] = {}last_seen: dict[tuple, str | None] = {}for record in ordered.iter_rows(named=True): key = (record["STUDYID"], record["USUBJID"], record["SERIES"]) prev_by_key[(record["STUDYID"], record["USUBJID"], record["VSSEQ"])] = last_seen.get(key) if record["AVALC"] is not None: last_seen[key] = record["AVALC"]advs = ( vs.with_columns( PREVAVALC=pl.struct(["STUDYID", "USUBJID", "VSSEQ"]).map_elements( lambda s: prev_by_key.get((s["STUDYID"], s["USUBJID"], s["VSSEQ"])), return_dtype=pl.String, ) ) .sort(["USUBJID", "VSSEQ"]) .select("STUDYID", "USUBJID", "VSSEQ", "SERIES", "AVISITN", "AVALC", "PREVAVALC"))Path("/app/output").mkdir(parents=True, exist_ok=True)advs.write_csv("/app/output/advs.csv")solutions/adam-advs-prior-result/result.sas
Edit25 lines/* opensas reference solution for the yamaa benchmark adam-advs-prior-result. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data vs; length STUDYID USUBJID SERIES AVALC $1024; infile "/app/input/vs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ SERIES : $char1024. AVISITN AVALC : $char1024.;run;data ordered; set vs; ordinal=_n_; novisit=missing(AVISITN); run;proc sort data=ordered; by STUDYID USUBJID SERIES novisit AVISITN ordinal; run;data result; length PREVAVALC carried $1024; retain carried; set ordered; by STUDYID USUBJID SERIES; if first.SERIES then carried=''; PREVAVALC=carried; if not missing(AVALC) then carried=AVALC;run;proc sql; create table final as select STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC from result;quit;proc export data=final outfile="/app/output/advs.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions