As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.
We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.
Summary
README.mdPublic entries: Python and R. Passing coverage requires an installed inventory.
Goal: attach the reference unit and limits to each collected
test result and flag the result against them: LBSTRESU,
LBSTNRLO, LBSTNRHI, and LBNRIND.
Input: collected results, one row per result carrying the test code and numeric result; a demographics input carrying sex by subject; plus a reference dictionary keyed by test code and sex giving the unit and the lower and upper limits.
Variables:
LBTESTCDis the test code as collected; together with sex from DM it selects the reference entry.LBSTRESNis the numeric result in standard units as collected; missing when the result was not collected.LBSTRESUis the reference unit for the test-and-sex combination, in standard units.LBSTNRLOis the lower reference limit in standard units.LBSTNRHIis the upper reference limit in standard units.LBNRINDisLOWwhen the result is below the lower limit,HIGHwhen it is above the upper limit, andNORMALotherwise; blank when the result or either reference limit is missing.
Note: a test-and-sex combination with two reference entries stops the run. One with no entry leaves the unit and both limits blank, and its range indicator stays blank. A record whose result is missing still carries its unit and limits.
Inputs
Under review3 source filesinput/dm.csv
Edit8 rows| STUDYID | USUBJID | SEX |
|---|---|---|
| PILOT7 | M | |
| PILOT7 | F | |
| PILOT7 | M | |
| PILOT7 | F | |
| PILOT7 | M | |
| PILOT7 | F | |
| PILOT7 | M | |
| PILOT7 | M |
input/lb_raw.csv
Edit8 rows| STUDYID | USUBJID | LBSEQ | LBTESTCD | LBSTRESN |
|---|---|---|---|---|
| PILOT7 | 1 | ALT | 15 | |
| PILOT7 | 1 | ALT | 55 | |
| PILOT7 | 1 | HGB | 12.5 | |
| PILOT7 | 1 | HGB | 12.5 | |
| PILOT7 | 1 | CREAT | 0.9 | |
| PILOT7 | 1 | POTASSIUM | 3.2 | |
| PILOT7 | 1 | ALT | — | |
| PILOT7 | 1 | GLUCOSE | 5 |
input/lbrange.csv
Edit6 rows| LBTESTCD | SEX | UNIT | NRLO | NRHI |
|---|---|---|---|---|
| ALT | M | U/L | 0 | 40 |
| ALT | F | U/L | 0 | 40 |
| HGB | M | g/dL | 13.5 | 17.5 |
| HGB | F | g/dL | 12 | 16 |
| CREAT | M | mg/dL | 0.7 | 1.3 |
| POTASSIUM | F | mmol/L | 3.5 | 5.1 |
Expected output
Under review8 expected rowsexpected/lb.csv
Edit8 rows| DOMAIN | STUDYID | USUBJID | LBSEQ | LBTESTCD | LBSTRESN | LBSTRESU | LBSTNRLO | LBSTNRHI | LBNRIND |
|---|---|---|---|---|---|---|---|---|---|
| LB | PILOT7 | 1 | ALT | 15 | U/L | 0 | 40 | NORMAL | |
| LB | PILOT7 | 1 | ALT | 55 | U/L | 0 | 40 | HIGH | |
| LB | PILOT7 | 1 | HGB | 12.5 | g/dL | 13.5 | 17.5 | LOW | |
| LB | PILOT7 | 1 | HGB | 12.5 | g/dL | 12 | 16 | NORMAL | |
| LB | PILOT7 | 1 | CREAT | 0.9 | mg/dL | 0.7 | 1.3 | NORMAL | |
| LB | PILOT7 | 1 | POTASSIUM | 3.2 | mmol/L | 3.5 | 5.1 | LOW | |
| LB | PILOT7 | 1 | ALT | — | U/L | 0 | 40 | — | |
| LB | PILOT7 | 1 | GLUCOSE | 5 | — | — | — | — |
Human review and assessment
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "LB". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from LB_RAW.STUDYID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from LB_RAW.USUBJID. | SDTM | 3 | ||||
| LBSEQ | Sequence Number | Num | Collected | Copy value from LB_RAW.LBSEQ. | SDTM | 4 | ||||
| LBTESTCD | Lab Test or Examination Short Name | Char | Collected | Copy value from LB_RAW.LBTESTCD. | SDTM | 5 | ||||
| LBSTRESN | Numeric Result/Finding in Standard Units | Num | Collected | Copy value from LB_RAW.LBSTRESN. | SDTM | 6 | ||||
| LBSTRESU | Standard Units | Char | Derived | Take UNIT from the LBRANGE record matching on LBTESTCD and SEX, and no match yields blank. | SDTM | 7 | ||||
| LBSTNRLO | Reference Range Lower Limit-Std Units | Num | Derived | Take NRLO from the LBRANGE record matching on LBTESTCD and SEX, and no match yields blank. | SDTM | 8 | ||||
| LBSTNRHI | Reference Range Upper Limit-Std Units | Num | Derived | Take NRHI from the LBRANGE record matching on LBTESTCD and SEX, and no match yields blank. | SDTM | 9 | ||||
| LBNRIND | Reference Range Indicator | Char | Derived | If LBSTRESN IS NULL OR LBSTNRLO IS NULL OR LBSTNRHI IS NULL then "None"; If LBSTRESN < LBSTNRLO then "LOW"; If LBSTRESN > LBSTNRHI then "HIGH"; otherwise "NORMAL". | SDTM | 10 |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.7 | 2026-10-20 | yamaa docs build | Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default | ||
| 1.6 | 2026-10-17 | yamaa docs build | Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight | ||
| 1.5 | 2026-10-17 | yamaa docs build | Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part | ||
| 1.4 | 2026-10-15 | yamaa docs build | State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks | ||
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-lb-ranges/full.md
Edit20 linesFollowing CDISC SDTM standards, use the provided DM, LBRANGE, and LB_RAW datasets to create an LB dataset with one record per collected test result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBSTRESU, LBSTNRLO, LBSTNRHI, LBNRIND
DOMAIN is "LB". The record's test code and the subject's sex in DM together select the reference entry in LBRANGE. LBSTRESU is that entry's unit, LBSTNRLO is its lower limit, and LBSTNRHI is its upper limit. When no entry matches the record's test code and the subject's sex, LBSTRESU, LBSTNRLO, and LBSTNRHI have no value. LBNRIND is "LOW" when the result is below the lower limit, "HIGH" when it is above the upper limit, and "NORMAL" otherwise. LBNRIND has no value when the result or either limit has no value. A record whose result was not collected still carries its unit and its lower and upper limits.
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
prompts/sdtm-lb-ranges/conventions.md
Edit12 linesFollowing CDISC SDTM standards, use the provided DM, LBRANGE, and LB_RAW datasets to create an LB dataset with one record per collected test result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBSTRESU, LBSTNRLO, LBSTNRHI, LBNRIND
DOMAIN is "LB". LBNRIND is "LOW", "HIGH", or "NORMAL", or has no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
prompts/sdtm-lb-ranges/brief.md
Edit10 linesFollowing CDISC SDTM standards, use the provided DM, LBRANGE, and LB_RAW datasets to create an LB dataset with one record per collected test result.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBSTRESU, LBSTNRLO, LBSTNRHI, LBNRIND
Read the source datasets from /app/input and save the completed dataset as /app/output/lb.csv.
R
Edit3 lineslibrary(yamaa)yamaa_spec("spec.yaml")$run()$outputPython
Edit3 linesimport yamaayamaa.spec("spec.yaml").run().outputsolutions/sdtm-lb-ranges/result.R
Edit51 lines# Reference solution for the yamaa benchmark sdtm-lb-ranges (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)lb_raw <- read_csv( "/app/input/lb_raw.csv", col_types = cols(LBSEQ = col_integer(), .default = col_character()))dm <- read_csv( "/app/input/dm.csv", col_types = cols(.default = col_character()))lbrange <- read_csv( "/app/input/lbrange.csv", col_types = cols(.default = col_character()))lb <- lb_raw |> left_join(dm |> select(USUBJID, SEX), by = "USUBJID") |> left_join(lbrange, by = c("LBTESTCD", "SEX")) |> mutate( DOMAIN = "LB", LBSTRESU = UNIT, LBSTNRLO = NRLO, LBSTNRHI = NRHI, LBSTRESN = as.character(LBSTRESN), LBNRIND = case_when( is.na(LBSTRESN) | LBSTRESN == "" ~ NA_character_, is.na(NRLO) | NRLO == "" ~ NA_character_, is.na(NRHI) | NRHI == "" ~ NA_character_, suppressWarnings(as.numeric(LBSTRESN)) < suppressWarnings(as.numeric(NRLO)) ~ "LOW", suppressWarnings(as.numeric(LBSTRESN)) > suppressWarnings(as.numeric(NRHI)) ~ "HIGH", TRUE ~ "NORMAL" ), LBSTRESN = na_if(LBSTRESN, "") ) |> arrange(USUBJID, LBSEQ) |> select( DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBSTRESU, LBSTNRLO, LBSTNRHI, LBNRIND )dir.create("/app/output", showWarnings = FALSE)write_csv(lb, "/app/output/lb.csv", na = "")solutions/sdtm-lb-ranges/result.py
Edit98 lines# Reference solution for the yamaa benchmark sdtm-lb-ranges (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1."""Build the SDTM LB dataset from DM, LBRANGE, and LB_RAW.One record per collected test result in LB_RAW. The reference range(unit and limits) is selected by the record's test code together withthe subject's sex from DM. LBNRIND flags the result against the limits."""import csvimport osINPUT_DIR = "/app/input"OUTPUT_DIR = "/app/output"def read_rows(name): with open(os.path.join(INPUT_DIR, name), newline="") as handle: return list(csv.DictReader(handle))def main(): dm = read_rows("dm.csv") lbrange = read_rows("lbrange.csv") lb_raw = read_rows("lb_raw.csv") sex_by_subject = {row["USUBJID"]: row["SEX"] for row in dm} range_by_key = { (row["LBTESTCD"], row["SEX"]): row for row in lbrange } records = [] for row in lb_raw: sex = sex_by_subject.get(row["USUBJID"], "") entry = range_by_key.get((row["LBTESTCD"], sex)) if entry is None: unit = "" nrlo = "" nrhi = "" else: unit = entry["UNIT"] nrlo = entry["NRLO"] nrhi = entry["NRHI"] result = row["LBSTRESN"] nrind = "" if result != "" and nrlo != "" and nrhi != "": value = float(result) if value < float(nrlo): nrind = "LOW" elif value > float(nrhi): nrind = "HIGH" else: nrind = "NORMAL" records.append( { "DOMAIN": "LB", "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "LBSEQ": row["LBSEQ"], "LBTESTCD": row["LBTESTCD"], "LBSTRESN": result, "LBSTRESU": unit, "LBSTNRLO": nrlo, "LBSTNRHI": nrhi, "LBNRIND": nrind, } ) fieldnames = [ "DOMAIN", "STUDYID", "USUBJID", "LBSEQ", "LBTESTCD", "LBSTRESN", "LBSTRESU", "LBSTNRLO", "LBSTNRHI", "LBNRIND", ] os.makedirs(OUTPUT_DIR, exist_ok=True) with open( os.path.join(OUTPUT_DIR, "lb.csv"), "w", newline="" ) as handle: writer = csv.DictWriter(handle, fieldnames=fieldnames) writer.writeheader() writer.writerows(records)if __name__ == "__main__": main()solutions/sdtm-lb-ranges/result.sas
Edit29 lines/* opensas reference solution for the yamaa benchmark sdtm-lb-ranges. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data dm; length STUDYID USUBJID SEX $1024; infile "/app/input/dm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. SEX : $char1024.;run;data lb_raw; length STUDYID USUBJID LBTESTCD $1024; infile "/app/input/lb_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. LBSEQ LBTESTCD : $char1024. LBSTRESN;run;data lbrange; length LBTESTCD SEX UNIT $1024; infile "/app/input/lbrange.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input LBTESTCD : $char1024. SEX : $char1024. UNIT : $char1024. NRLO NRHI;run;proc sql;create table joined as select a.*,c.UNIT as LBSTRESU,c.NRLO as LBSTNRLO,c.NRHI as LBSTNRHI from lb_raw a left join dm b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID left join lbrange c on a.LBTESTCD=c.LBTESTCD and b.SEX=c.SEX; quit;data result; set joined; length DOMAIN $2 LBNRIND $8; DOMAIN='LB'; LBNRIND=''; if not missing(LBSTRESN) and not missing(LBSTNRLO) and not missing(LBSTNRHI) then do; if LBSTRESN<LBSTNRLO then LBNRIND='LOW'; else if LBSTRESN>LBSTNRHI then LBNRIND='HIGH'; else LBNRIND='NORMAL'; end; run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, LBSEQ, LBTESTCD, LBSTRESN, LBSTRESU, LBSTNRLO, LBSTNRHI, LBNRIND from result;quit;proc export data=final outfile="/app/output/lb.csv" dbms=csv replace; run;
yamaa