As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: build one ADVS row per collected character-result row,
carrying SERIES, AVISITN, and AVALC through unchanged and
adding PREVAVALC for the closest earlier non-blank result.
Input: collected character results for three subjects, with series, visit number, and character result.
Variables:
SERIESidentifies the analysis series; rows with no series value share one series.AVISITNis the visit number that orders rows within a series.AVALCis the current character result, kept as collected; blank when no result was collected.PREVAVALCis the closest earlier non-blank result for the same subject and series, never the row's own; blank when no earlier row in the series has a result.
Note: within a series, rows are ordered by visit number with
missing numbers last, and rows sharing a visit number (or both
missing one) keep their collected order. The look-back skips blank
results, so it can cross consecutive blank rows, but it never
crosses into another series. A subject with a single collected row,
and a series whose earlier rows are all blank, both leave PREVAVALC
blank.
Inputs
Under review1 source fileinput/vs.csv
Edit14 rows| STUDYID | USUBJID | VSSEQ | SERIES | AVISITN | AVALC |
|---|---|---|---|---|---|
| YAMAA-01 | 1 | A | — | NULL-ORDER | |
| YAMAA-01 | 2 | A | 1 | FIRST | |
| YAMAA-01 | 3 | A | 1 | — | |
| YAMAA-01 | 4 | A | 2 | — | |
| YAMAA-01 | 5 | B | 1 | B-FIRST | |
| YAMAA-01 | 6 | B | 2 | — | |
| YAMAA-01 | 7 | — | 1 | UNGROUPED | |
| YAMAA-01 | 8 | — | 2 | — | |
| YAMAA-01 | 9 | A | 1 | SOLO | |
| YAMAA-01 | 10 | C | 1 | — | |
| YAMAA-01 | 11 | C | 2 | — | |
| YAMAA-01 | 12 | C | 3 | LATE | |
| YAMAA-01 | 13 | C | — | — | |
| YAMAA-01 | 14 | C | — | — |
Expected output
Under review14 expected rowsexpected/advs.csv
Edit14 rows| STUDYID | USUBJID | VSSEQ | SERIES | AVISITN | AVALC | PREVAVALC |
|---|---|---|---|---|---|---|
| YAMAA-01 | 1 | A | — | NULL-ORDER | FIRST | |
| YAMAA-01 | 2 | A | 1 | FIRST | — | |
| YAMAA-01 | 3 | A | 1 | — | FIRST | |
| YAMAA-01 | 4 | A | 2 | — | FIRST | |
| YAMAA-01 | 5 | B | 1 | B-FIRST | — | |
| YAMAA-01 | 6 | B | 2 | — | B-FIRST | |
| YAMAA-01 | 7 | — | 1 | UNGROUPED | — | |
| YAMAA-01 | 8 | — | 2 | — | UNGROUPED | |
| YAMAA-01 | 9 | A | 1 | SOLO | — | |
| YAMAA-01 | 10 | C | 1 | — | — | |
| YAMAA-01 | 11 | C | 2 | — | — | |
| YAMAA-01 | 12 | C | 3 | LATE | — | |
| YAMAA-01 | 13 | C | — | — | LATE | |
| YAMAA-01 | 14 | C | — | — | LATE |
Human review and assessment
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADVS | STUDYID | Study Identifier | character | Copy value from VS.STUDYID. | Predecessor | ||
| ADVS | USUBJID | Unique Subject Identifier | character | Copy value from VS.USUBJID. | Predecessor | ||
| ADVS | VSSEQ | Sequence Number | numeric | Copy value from VS.VSSEQ. | Predecessor | ||
| ADVS | SERIES | Analysis Series | character | Copy value from VS.SERIES. | Predecessor | ||
| ADVS | AVISITN | Analysis Visit Number | numeric | Copy value from VS.AVISITN. | Predecessor | ||
| ADVS | AVALC | Analysis Value (C) | character | Copy value from VS.AVALC. | Predecessor | ||
| ADVS | PREVAVALC | Previous Non-Missing Analysis Value (C) | character | Most recent non-missing AVALC from an earlier row within each group of STUDYID, USUBJID, SERIES, ordered by AVISITN, nulls last then VSSEQ. | Derived |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/adam-advs-prior-result/full.md
Edit17 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per collected character-result row.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
PREVAVALC is the closest earlier non-blank AVALC for the same subject and series, never the row's own AVALC. Rows with no SERIES value share one series. Within a series, rows are ordered by AVISITN, with a missing AVISITN last, and rows sharing an AVISITN, or both missing one, keep their collected order. Blank results are skipped, so the closest earlier non-blank AVALC may lie beyond consecutive blank rows, but it never comes from another series. PREVAVALC has no value when no earlier row in the series has a non-blank AVALC.
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
prompts/adam-advs-prior-result/conventions.md
Edit8 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per collected character-result row.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
prompts/adam-advs-prior-result/brief.md
Edit8 linesFollowing CDISC ADaM standards, use the provided VS dataset to create an ADVS dataset with one record per collected character-result row.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC
Read the source datasets from /app/input and save the completed dataset as /app/output/advs.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/adam-advs-prior-result/result.R
Edit40 lines# Reference solution for the yamaa benchmark adam-advs-prior-result (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)library(tidyr)vs <- read_csv( "/app/input/vs.csv", col_types = cols( VSSEQ = col_integer(), AVISITN = col_integer(), .default = col_character() ))# Within a series, rows are ordered by visit number with missing numbers# last, keeping the collected order for ties. The look-back skips blank# results but never crosses into another series.ordered <- vs |> arrange(STUDYID, USUBJID, SERIES, is.na(AVISITN), AVISITN)# Forward-fill the ordered results, then lag by one so the row's own# value never counts.prev <- ordered |> group_by(STUDYID, USUBJID, SERIES) |> fill(AVALC, .direction = "down") |> mutate(PREVAVALC = lag(AVALC)) |> ungroup() |> select(STUDYID, USUBJID, VSSEQ, PREVAVALC)advs <- vs |> left_join(prev, by = c("STUDYID", "USUBJID", "VSSEQ")) |> arrange(USUBJID, VSSEQ) |> select(STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC)dir.create("/app/output", showWarnings = FALSE)write_csv(advs, "/app/output/advs.csv", na = "")solutions/adam-advs-prior-result/result.py
Edit67 lines# Reference solution for the yamaa benchmark adam-advs-prior-result (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build ADVS from the VS dataset: one record per collected row, withPREVAVALC set to the closest earlier non-blank AVALC for the samesubject and series.Rows with no SERIES value share one series. Within a subject andseries, rows are ordered by AVISITN with a missing AVISITN last, androws that tie keep their collected order. Blank results never serve asa previous value, and a previous value never crosses a series or asubject."""import csvINPUT_VS = "/app/input/vs.csv"OUTPUT_ADVS = "/app/output/advs.csv"COLUMNS = ["STUDYID", "USUBJID", "VSSEQ", "SERIES", "AVISITN", "AVALC", "PREVAVALC"]def visit_key(row): """Sort key: AVISITN ascending, a missing AVISITN last. The sort is stable, so rows sharing an AVISITN, or both missing one, keep their collected order. """ text = row["AVISITN"].strip() if text == "": return (1, 0.0) return (0, float(text))def main(): with open(INPUT_VS, newline="") as handle: rows = list(csv.DictReader(handle)) groups = {} for index, row in enumerate(rows): key = (row["USUBJID"], row["SERIES"]) groups.setdefault(key, []).append(index) previous = {} for indexes in groups.values(): last_nonblank = "" for index in sorted(indexes, key=lambda i: visit_key(rows[i])): previous[index] = last_nonblank avalc = rows[index]["AVALC"] if avalc.strip() != "": last_nonblank = avalc with open(OUTPUT_ADVS, "w", newline="") as handle: writer = csv.DictWriter(handle, fieldnames=COLUMNS) writer.writeheader() for index, row in enumerate(rows): out = {name: row[name] for name in COLUMNS if name != "PREVAVALC"} out["PREVAVALC"] = previous[index] writer.writerow(out)if __name__ == "__main__": main()solutions/adam-advs-prior-result/result.sas
Edit25 lines/* opensas reference solution for the yamaa benchmark adam-advs-prior-result. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data vs; length STUDYID USUBJID SERIES AVALC $1024; infile "/app/input/vs.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. VSSEQ SERIES : $char1024. AVISITN AVALC : $char1024.;run;data ordered; set vs; ordinal=_n_; novisit=missing(AVISITN); run;proc sort data=ordered; by STUDYID USUBJID SERIES novisit AVISITN ordinal; run;data result; length PREVAVALC carried $1024; retain carried; set ordered; by STUDYID USUBJID SERIES; if first.SERIES then carried=''; PREVAVALC=carried; if not missing(AVALC) then carried=AVALC;run;proc sql; create table final as select STUDYID, USUBJID, VSSEQ, SERIES, AVISITN, AVALC, PREVAVALC from result;quit;proc export data=final outfile="/app/output/advs.csv" dbms=csv replace; run;
yamaa