Classify a Result, Its Shift from Baseline, and Two Criteria

Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Goal: derive ANRIND, BASE, BNRIND, SHIFT1, R2BASE, CRIT1, CRIT1FL, CRIT2, and CRIT2FL: mark where each analysis value sits against its normal range, how that mark moved since baseline, and whether the record met a high-result criterion or a low-result criterion.

Input: laboratory records carrying the analysis value AVAL, the normal range limits ANRLO and ANRHI, and the baseline flag ABLFL (Y on the record taken as the baseline for the subject and parameter).

Variables:

ANRIND is the record's own mark: LOW below ANRLO, HIGH above ANRHI, and NORMAL between them, limits included, so a result exactly at a limit reads NORMAL; empty when AVAL, ANRLO, or ANRHI is missing, since one missing limit makes the mark unclassifiable.
BASE repeats the baseline record's AVAL on every record of the subject and parameter; empty when no record carries the flag.
BNRIND repeats the baseline record's ANRIND the same way; empty with no flagged baseline, or when the baseline record itself could not be marked.
SHIFT1 joins the baseline mark and the record's own mark, baseline first, so a result that stayed normal reads NORMAL to NORMAL and one that moved out of range reads NORMAL to HIGH; empty when either mark is missing - the record's own or the baseline's.
R2BASE divides AVAL by BASE, so the baseline record itself reads 1 when the ratio can be computed; empty when AVAL is missing, when there is no baseline, and when the baseline is zero since the multiple is not defined there.
CRIT1 states the criterion the record was assessed against, a result greater than three times the upper limit of normal (ULN): Result greater than 3 x ULN; empty when AVAL or ANRHI is missing, since the comparison cannot be made.
CRIT1FL says whether the record met it, Y or N; empty where it could not be assessed, which differs from assessed and not met. A result exactly at three times the limit does not meet it - the comparison is strictly greater than.
CRIT2 states the second criterion the record was assessed against, a result less than the lower limit of normal (LLN): Result less than LLN; empty when AVAL or ANRLO is missing, since the comparison cannot be made. It is assessed independently of the first: a record whose upper limit is missing can still meet the low criterion.
CRIT2FL says whether the record met it, Y or N; empty where it could not be assessed. A result exactly at the lower limit does not meet it - the comparison is strictly less than.

Note: the shift joins the baseline mark with the record's own mark and the ratio rests on the flagged baseline value, while the mark and the criteria rest on the record's own value and limits; each criterion text and its flag always arrive together, as do the mark and the shift. A record that breaks these pairings stops the run, and no output is written.

YAML specification

1 spec file

spec.yaml

Edit217 lines

1schema_version: "1.0"2domain: ADLB3keys: [STUDYID, USUBJID, PARAMCD, ASEQ]4input:5  ADLB_RAW:6    path: input/adlb_raw.csv7    types: {ASEQ: int, AVAL: float, ANRLO: float, ANRHI: float}89output:10  path: adlb.csv11  columns: [STUDYID, USUBJID, PARAMCD, PARAM, ASEQ, AVISIT, AVAL, ANRLO, ANRHI,12            ANRIND, ABLFL, BASE, BNRIND, SHIFT1, R2BASE, CRIT1, CRIT1FL,13            CRIT2, CRIT2FL]1415columns:16  - name: STUDYID17    type: str18    label: Study Identifier19    derivation: ADLB_RAW.STUDYID2021  - name: USUBJID22    type: str23    label: Unique Subject Identifier24    derivation: ADLB_RAW.USUBJID2526  - name: PARAMCD27    type: str28    label: Parameter Code29    derivation: ADLB_RAW.PARAMCD3031  - name: PARAM32    type: str33    label: Parameter34    derivation: ADLB_RAW.PARAM3536  - name: ASEQ37    type: int38    label: Analysis Sequence Number39    derivation: ADLB_RAW.ASEQ4041  - name: AVISIT42    type: str43    label: Analysis Visit44    derivation: ADLB_RAW.AVISIT4546  - name: AVAL47    type: float48    label: Analysis Value49    derivation: ADLB_RAW.AVAL5051  - name: ANRLO52    type: float53    label: Analysis Normal Range Lower Limit54    derivation: ADLB_RAW.ANRLO5556  - name: ANRHI57    type: float58    label: Analysis Normal Range Upper Limit59    derivation: ADLB_RAW.ANRHI6061  # Empty when the value or either limit is missing. The first branch is62  # needed: a comparison that cannot be made falls through, and could end63  # at NORMAL.64  - name: ANRIND65    type: str66    label: Analysis Reference Range Indicator67    derivation:68      case:69        - when: "AVAL IS NULL OR ANRLO IS NULL OR ANRHI IS NULL"70          then: {literal: null}71        - when: "AVAL < ANRLO"72          then: {literal: LOW}73        - when: "AVAL > ANRHI"74          then: {literal: HIGH}75        - otherwise: {literal: NORMAL}76    verifications:77      allowed_values:78        values: [LOW, NORMAL, HIGH]7980  - name: ABLFL81    type: str82    label: Baseline Record Flag83    derivation: ADLB_RAW.ABLFL84    verifications:85      allowed_values:86        values: [Y]8788  # The flagged baseline record's value and mark, repeated on every record of89  # the subject and parameter. With no flagged record both are empty; ONLY()90  # stops the run if more than one record is flagged.91  - name: BASE92    type: float93    label: Baseline Value94    derivation:95      aggregate:96        filter: "ABLFL = 'Y'"97        group_by: [STUDYID, USUBJID, PARAMCD]98        expr: "ONLY(AVAL)"99100  - name: BNRIND101    type: str102    label: Baseline Reference Range Indicator103    derivation:104      aggregate:105        filter: "ABLFL = 'Y'"106        group_by: [STUDYID, USUBJID, PARAMCD]107        expr: "ONLY(ANRIND)"108    verifications:109      allowed_values:110        values: [LOW, NORMAL, HIGH]111112  # Baseline mark first; empty when either mark is missing.113  - name: SHIFT1114    type: str115    label: Shift 1116    derivation:117      str_template:118        template: "{BNRIND} to {ANRIND}"119        missing: null120121  # NULLIF leaves the ratio empty when the baseline value is zero.122  - name: R2BASE123    type: float124    label: Ratio to Baseline125    derivation:126      compute:127        expr: "AVAL / NULLIF(BASE, 0)"128129  # Three times the upper limit of normal (ULN). A condition cannot hold130  # arithmetic, so the limit is its own column; it is not in the output.131  - name: CRITLIM132    type: float133    label: Criterion Limit134    derivation:135      compute:136        expr: "3 * ANRHI"137138  - name: CRIT1139    type: str140    label: Analysis Criterion 1141    derivation:142      case:143        - when: "AVAL IS NOT NULL AND CRITLIM IS NOT NULL"144          then: {literal: Result greater than 3 x ULN}145146  # Y above the limit and N at or below it. The first branch keeps a record147  # with no value or no limit empty rather than N.148  - name: CRIT1FL149    type: str150    label: Criterion 1 Evaluation Result Flag151    derivation:152      case:153        - when: "AVAL IS NULL OR CRITLIM IS NULL"154          then: {literal: null}155        - when: "AVAL > CRITLIM"156          then: {literal: Y}157        - otherwise: {literal: N}158    verifications:159      allowed_values:160        values: [Y, N]161162  # The low tail: a result below the lower limit of normal (LLN).163  - name: CRIT2164    type: str165    label: Analysis Criterion 2166    derivation:167      case:168        - when: "AVAL IS NOT NULL AND ANRLO IS NOT NULL"169          then: {literal: Result less than LLN}170171  # Y below the limit and N at or above it. Independent missingness from172  # CRIT1FL: only the value and the lower limit decide assessability.173  - name: CRIT2FL174    type: str175    label: Criterion 2 Evaluation Result Flag176    derivation:177      case:178        - when: "AVAL IS NULL OR ANRLO IS NULL"179          then: {literal: null}180        - when: "AVAL < ANRLO"181          then: {literal: Y}182        - otherwise: {literal: N}183    verifications:184      allowed_values:185        values: [Y, N]186187verifications:188  - unique:189      columns: [STUDYID, USUBJID, PARAMCD, ASEQ]190  - row_count:191      min: 17192      max: 17193  - all_or_none:194      id: criterion-text-accompanies-its-flag195      columns: [CRIT1, CRIT1FL]196  - all_or_none:197      id: criterion-2-text-accompanies-its-flag198      columns: [CRIT2, CRIT2FL]199  - all_or_none:200      id: shift-needs-both-of-its-classifications201      columns: [ANRIND, SHIFT1]202  - assert:203      id: a-baseline-record-is-its-own-reference204      when: "ABLFL = 'Y' AND R2BASE IS NOT NULL"205      require: "R2BASE = 1"206  - assert:207      id: a-classification-rests-on-both-of-its-limits208      when: "ANRIND IS NOT NULL"209      require: "ANRLO IS NOT NULL AND ANRHI IS NOT NULL"210  - assert:211      id: criterion-is-met-only-above-its-limit212      when: "CRIT1FL = 'Y'"213      require: "AVAL > CRITLIM"214  - assert:215      id: criterion-2-is-met-only-below-its-limit216      when: "CRIT2FL = 'Y'"217      require: "AVAL < ANRLO"

Inputs

Under review1 source file

input/adlb_raw.csv

Edit17 rows

input/adlb_raw.csv
STUDYID	PARAMCD	PARAM	ASEQ	AVISIT	AVAL	ANRLO	ANRHI	ABLFL
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	1	BASELINE	30	0	40	Y
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	2	WEEK 2	60	0	40	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	3	WEEK 4	150	0	40	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	4	WEEK 8	40	0	40	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	1	BASELINE	45	0	40	Y
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	2	WEEK 2	18	0	40	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	3	WEEK 4	—	0	40	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	4	WEEK 8	120	0	40	—
YAMAA-01	EOS	Eosinophils (%)	1	BASELINE	0	1	6	Y
YAMAA-01	EOS	Eosinophils (%)	2	WEEK 2	4	1	6	—
YAMAA-01	EOS	Eosinophils (%)	3	WEEK 4	1	1	6	—
YAMAA-01	LDH	Lactate Dehydrogenase (U/L)	3	BASELINE	200	—	—	Y
YAMAA-01	LDH	Lactate Dehydrogenase (U/L)	4	WEEK 2	260	—	—	—
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	5	BASELINE	55	10	—	Y
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	6	WEEK 2	70	10	—	—
YAMAA-01	EOS	Eosinophils (%)	4	WEEK 8	0.5	1	6	—
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	7	WEEK 4	5	10	—	—

Expected output

Under review17 expected rows

expected/adlb.csv

Edit17 rows

expected/adlb.csv
STUDYID	PARAMCD	PARAM	ASEQ	AVISIT	AVAL	ANRLO	ANRHI	ANRIND	ABLFL	BASE	BNRIND	SHIFT1	R2BASE	CRIT1	CRIT1FL	CRIT2	CRIT2FL
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	1	BASELINE	30	0	40	NORMAL	Y	30	NORMAL	NORMAL to NORMAL	1	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	2	WEEK 2	60	0	40	HIGH	—	30	NORMAL	NORMAL to HIGH	2	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	3	WEEK 4	150	0	40	HIGH	—	30	NORMAL	NORMAL to HIGH	5	Result greater than 3 x ULN	Y	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	4	WEEK 8	40	0	40	NORMAL	—	30	NORMAL	NORMAL to NORMAL	1.3333333333333333	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	1	BASELINE	45	0	40	HIGH	Y	45	HIGH	HIGH to HIGH	1	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	2	WEEK 2	18	0	40	NORMAL	—	45	HIGH	HIGH to NORMAL	0.4	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	3	WEEK 4	—	0	40	—	—	45	HIGH	—	—	—	—	—	—
YAMAA-01	ALT	Alanine Aminotransferase (U/L)	4	WEEK 8	120	0	40	HIGH	—	45	HIGH	HIGH to HIGH	2.6666666666666665	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	EOS	Eosinophils (%)	1	BASELINE	0	1	6	LOW	Y	0	LOW	LOW to LOW	—	Result greater than 3 x ULN	N	Result less than LLN	Y
YAMAA-01	EOS	Eosinophils (%)	2	WEEK 2	4	1	6	NORMAL	—	0	LOW	LOW to NORMAL	—	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	EOS	Eosinophils (%)	3	WEEK 4	1	1	6	NORMAL	—	0	LOW	LOW to NORMAL	—	Result greater than 3 x ULN	N	Result less than LLN	N
YAMAA-01	LDH	Lactate Dehydrogenase (U/L)	3	BASELINE	200	—	—	—	Y	200	—	—	1	—	—	—	—
YAMAA-01	LDH	Lactate Dehydrogenase (U/L)	4	WEEK 2	260	—	—	—	—	200	—	—	1.3	—	—	—	—
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	5	BASELINE	55	10	—	—	Y	55	—	—	1	—	—	Result less than LLN	N
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	6	WEEK 2	70	10	—	—	—	55	—	—	1.2727272727272727	—	—	Result less than LLN	N
YAMAA-01	EOS	Eosinophils (%)	4	WEEK 8	0.5	1	6	LOW	—	0	LOW	LOW to LOW	—	Result greater than 3 x ULN	N	Result less than LLN	Y
YAMAA-01	GGT	Gamma Glutamyl Transferase (U/L)	7	WEEK 4	5	10	—	—	—	55	—	—	0.09090909090909091	—	—	Result less than LLN	Y

Shaded columns are derived— = missing value

Comments and assessment materials

Dataset Name	Variable Name	Variable Label	Type	Codelist/Controlled Terms	Computational Method	Origin
ADLB	STUDYID	Study Identifier	character		Copy value from ADLB_RAW.STUDYID.	Predecessor
ADLB	USUBJID	Unique Subject Identifier	character		Copy value from ADLB_RAW.USUBJID.	Predecessor
ADLB	PARAMCD	Parameter Code	character		Copy value from ADLB_RAW.PARAMCD.	Predecessor
ADLB	PARAM	Parameter	character		Copy value from ADLB_RAW.PARAM.	Predecessor
ADLB	ASEQ	Analysis Sequence Number	numeric		Copy value from ADLB_RAW.ASEQ.	Predecessor
ADLB	AVISIT	Analysis Visit	character		Copy value from ADLB_RAW.AVISIT.	Predecessor
ADLB	AVAL	Analysis Value	numeric		Copy value from ADLB_RAW.AVAL.	Predecessor
ADLB	ANRLO	Analysis Normal Range Lower Limit	numeric		Copy value from ADLB_RAW.ANRLO.	Predecessor
ADLB	ANRHI	Analysis Normal Range Upper Limit	numeric		Copy value from ADLB_RAW.ANRHI.	Predecessor
ADLB	ANRIND	Analysis Reference Range Indicator	character	LOW, NORMAL, HIGH	If AVAL IS NULL OR ANRLO IS NULL OR ANRHI IS NULL then "None"; If AVAL < ANRLO then "LOW"; If AVAL > ANRHI then "HIGH"; otherwise "NORMAL".	Derived
ADLB	ABLFL	Baseline Record Flag	character	Y	Copy value from ADLB_RAW.ABLFL.	Predecessor
ADLB	BASE	Baseline Value	numeric		Aggregate ONLY(AVAL) over records grouped by STUDYID, USUBJID, PARAMCD, matching on the shared output keys, where ABLFL = 'Y'.	Derived
ADLB	BNRIND	Baseline Reference Range Indicator	character	LOW, NORMAL, HIGH	Aggregate ONLY(ANRIND) over records grouped by STUDYID, USUBJID, PARAMCD, matching on the shared output keys, where ABLFL = 'Y'.	Derived
ADLB	SHIFT1	Shift 1	character		Fill the template "{BNRIND} to {ANRIND}"; a missing field → blank.	Derived
ADLB	R2BASE	Ratio to Baseline	numeric		Compute AVAL / NULLIF(BASE, 0).	Derived
ADLB	CRIT1	Analysis Criterion 1	character		If AVAL IS NOT NULL AND CRITLIM IS NOT NULL then "Result greater than 3 x ULN"; otherwise blank.	Derived
ADLB	CRIT1FL	Criterion 1 Evaluation Result Flag	character	Y, N	If AVAL IS NULL OR CRITLIM IS NULL then "None"; If AVAL > CRITLIM then "Y"; otherwise "N".	Derived
ADLB	CRIT2	Analysis Criterion 2	character		If AVAL IS NOT NULL AND ANRLO IS NOT NULL then "Result less than LLN"; otherwise blank.	Derived
ADLB	CRIT2FL	Criterion 2 Evaluation Result Flag	character	Y, N	If AVAL IS NULL OR ANRLO IS NULL then "None"; If AVAL < ANRLO then "Y"; otherwise "N".	Derived

Variable	Permitted values
ANRIND	LOW, NORMAL, HIGH
ABLFL	Y
BNRIND	LOW, NORMAL, HIGH
CRIT1FL	Y, N
CRIT2FL	Y, N

Version	Date	Author	Description
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/adam-adlb-shift-criteria/full.md

Edit44 lines

Following CDISC ADaM standards, use the provided ADLB_RAW dataset to create an ADLB dataset with one record per laboratory record.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, PARAMCD, PARAM, ASEQ, AVISIT, AVAL, ANRLO, ANRHI, ANRIND, ABLFL, BASE, BNRIND, SHIFT1, R2BASE, CRIT1, CRIT1FL, CRIT2, CRIT2FL

ANRIND is the record's own mark: LOW below the lower limit, HIGH above the upper limit, and NORMAL between them, limits included, so a result exactly at a limit reads NORMAL. It has no value when the result or either limit is missing.

BASE repeats the flagged baseline record's value on every record of the subject and parameter; it has no value when no record carries the flag. BNRIND repeats the baseline record's mark the same way; it has no value with no flagged baseline, or when the baseline record itself could not be marked.

SHIFT1 joins the baseline mark and the record's own mark, baseline first, so a result that stayed normal reads "NORMAL to NORMAL". It has no value when either mark is missing.

R2BASE is the record's value as a multiple of the baseline value, not rounded, so the baseline record itself reads 1 when the ratio can be computed. It has no value when the record's value is missing, when there is no baseline, or when the baseline is zero.

CRIT1 states the criterion the record was assessed against, a result greater than three times the upper limit of normal (ULN): "Result greater than 3 x ULN". It has no value when the result or the upper limit is missing. CRIT1FL says whether the record met it, Y or N; it has no value where it could not be assessed. A result exactly at three times the limit does not meet it.

CRIT2 states a second criterion, a result less than the lower limit of normal (LLN): "Result less than LLN". It has no value when the result or the lower limit is missing, and it is assessed independently of the first criterion. CRIT2FL says whether the record met it, Y or N; it has no value where it could not be assessed. A result exactly at the lower limit does not meet it.

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

prompts/adam-adlb-shift-criteria/conventions.md

Edit20 lines

Following CDISC ADaM standards, use the provided ADLB_RAW dataset to create an ADLB dataset with one record per laboratory record.

ANRIND is LOW, HIGH, or NORMAL, or has no value.

SHIFT1 joins the baseline mark and the record's own mark, baseline first.

R2BASE is not rounded.

CRIT1 is "Result greater than 3 x ULN" or has no value. CRIT1FL is Y or N, or has no value. CRIT2 is "Result less than LLN" or has no value. CRIT2FL is Y or N, or has no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

prompts/adam-adlb-shift-criteria/brief.md

Edit10 lines

Following CDISC ADaM standards, use the provided ADLB_RAW dataset to create an ADLB dataset with one record per laboratory record.

Read the source datasets from /app/input and save the completed dataset as /app/output/adlb.csv.

run.R

Edit4 lines

1library(yamaa)23adlb <- yamaa_domain("spec.yaml")$output4adlb

run.py

Edit4 lines

1import yamaa23adlb = yamaa.yamaa_domain("spec.yaml").output4adlb

solutions/adam-adlb-shift-criteria/result.R

Edit88 lines

1# Reference solution for the yamaa benchmark adam-adlb-shift-criteria (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910raw <- read_csv(11  "/app/input/adlb_raw.csv",12  col_types = cols(13    ASEQ = col_integer(),14    AVAL = col_double(),15    ANRLO = col_double(),16    ANRHI = col_double(),17    .default = col_character()18  )19)2021# The record's own mark: LOW below the lower limit, HIGH above the upper22# limit, NORMAL between them limits included. No value when the result or23# either limit is missing.24based <- raw |>25  mutate(26    ANRIND = case_when(27      is.na(AVAL) | is.na(ANRLO) | is.na(ANRHI) ~ NA_character_,28      AVAL < ANRLO ~ "LOW",29      AVAL > ANRHI ~ "HIGH",30      .default = "NORMAL"31    )32  )3334# BASE repeats the flagged baseline record's value; BNRIND repeats its35# mark. Both have no value when no record carries the flag.36baselines <- based |>37  filter(ABLFL %in% "Y") |>38  select(STUDYID, USUBJID, PARAMCD, BASE = AVAL, BNRIND = ANRIND)3940with_base <- based |>41  left_join(baselines, by = c("STUDYID", "USUBJID", "PARAMCD"))4243# SHIFT1 joins the baseline mark and the record's own mark; R2BASE is the44# record as a multiple of the baseline; CRIT1 assesses greater than three45# times the upper limit; CRIT2 assesses less than the lower limit.46adlb <- with_base |>47  mutate(48    SHIFT1 = if_else(49      is.na(BNRIND) | is.na(ANRIND),50      NA_character_,51      paste(BNRIND, "to", ANRIND)52    ),53    R2BASE = if_else(54      is.na(AVAL) | is.na(BASE) | BASE == 0,55      NA_real_,56      AVAL / BASE57    ),58    CRITLIM = 3 * ANRHI,59    CRIT1 = if_else(60      !is.na(AVAL) & !is.na(CRITLIM),61      "Result greater than 3 x ULN",62      NA_character_63    ),64    CRIT1FL = case_when(65      is.na(AVAL) | is.na(CRITLIM) ~ NA_character_,66      AVAL > CRITLIM ~ "Y",67      .default = "N"68    ),69    CRIT2 = if_else(70      !is.na(AVAL) & !is.na(ANRLO),71      "Result less than LLN",72      NA_character_73    ),74    CRIT2FL = case_when(75      is.na(AVAL) | is.na(ANRLO) ~ NA_character_,76      AVAL < ANRLO ~ "Y",77      .default = "N"78    )79  ) |>80  arrange(USUBJID, ASEQ) |>81  select(82    STUDYID, USUBJID, PARAMCD, PARAM, ASEQ, AVISIT, AVAL, ANRLO, ANRHI,83    ANRIND, ABLFL, BASE, BNRIND, SHIFT1, R2BASE, CRIT1, CRIT1FL,84    CRIT2, CRIT2FL85  )8687dir.create("/app/output", showWarnings = FALSE)88write_csv(adlb, "/app/output/adlb.csv", na = "")

solutions/adam-adlb-shift-criteria/result.py

Edit115 lines

1# Reference solution for the yamaa benchmark adam-adlb-shift-criteria (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011raw = pl.read_csv("/app/input/adlb_raw.csv", infer_schema=False).with_columns(12    pl.col("ASEQ").cast(pl.Int64),13    pl.col("AVAL").cast(pl.Float64, strict=False),14    pl.col("ANRLO").cast(pl.Float64, strict=False),15    pl.col("ANRHI").cast(pl.Float64, strict=False),16)1718# The record's own mark: LOW below the lower limit, HIGH above the upper19# limit, NORMAL between them limits included. No value when the result or20# either limit is missing.21based = raw.with_columns(22    ANRIND=pl.when(23        pl.col("AVAL").is_null()24        | pl.col("ANRLO").is_null()25        | pl.col("ANRHI").is_null()26    )27    .then(None)28    .when(pl.col("AVAL") < pl.col("ANRLO"))29    .then(pl.lit("LOW"))30    .when(pl.col("AVAL") > pl.col("ANRHI"))31    .then(pl.lit("HIGH"))32    .otherwise(pl.lit("NORMAL")),33)3435# BASE repeats the flagged baseline record's value; BNRIND repeats its36# mark. Both have no value when no record carries the flag.37baselines = (38    based.filter(pl.col("ABLFL") == "Y")39    .select("STUDYID", "USUBJID", "PARAMCD", BASE="AVAL", BNRIND="ANRIND")40)41with_base = based.join(42    baselines, on=["STUDYID", "USUBJID", "PARAMCD"], how="left"43)4445# SHIFT1 joins the baseline mark and the record's own mark; R2BASE is the46# record as a multiple of the baseline; CRIT1 assesses greater than three47# times the upper limit; CRIT2 assesses less than the lower limit.48adlb = (49    with_base.with_columns(50        SHIFT1=pl.when(51            pl.col("BNRIND").is_null() | pl.col("ANRIND").is_null()52        )53        .then(None)54        .otherwise(pl.col("BNRIND") + pl.lit(" to ") + pl.col("ANRIND")),55        R2BASE=pl.when(56            pl.col("AVAL").is_null()57            | pl.col("BASE").is_null()58            | (pl.col("BASE") == 0)59        )60        .then(None)61        .otherwise(pl.col("AVAL") / pl.col("BASE")),62        CRITLIM=3 * pl.col("ANRHI"),63    )64    .with_columns(65        CRIT1=pl.when(66            pl.col("AVAL").is_not_null() & pl.col("CRITLIM").is_not_null()67        )68        .then(pl.lit("Result greater than 3 x ULN"))69        .otherwise(None),70        CRIT1FL=pl.when(71            pl.col("AVAL").is_null() | pl.col("CRITLIM").is_null()72        )73        .then(None)74        .when(pl.col("AVAL") > pl.col("CRITLIM"))75        .then(pl.lit("Y"))76        .otherwise(pl.lit("N")),77        CRIT2=pl.when(78            pl.col("AVAL").is_not_null() & pl.col("ANRLO").is_not_null()79        )80        .then(pl.lit("Result less than LLN"))81        .otherwise(None),82        CRIT2FL=pl.when(83            pl.col("AVAL").is_null() | pl.col("ANRLO").is_null()84        )85        .then(None)86        .when(pl.col("AVAL") < pl.col("ANRLO"))87        .then(pl.lit("Y"))88        .otherwise(pl.lit("N")),89    )90    .select(91        "STUDYID",92        "USUBJID",93        "PARAMCD",94        "PARAM",95        "ASEQ",96        "AVISIT",97        "AVAL",98        "ANRLO",99        "ANRHI",100        "ANRIND",101        "ABLFL",102        "BASE",103        "BNRIND",104        "SHIFT1",105        "R2BASE",106        "CRIT1",107        "CRIT1FL",108        "CRIT2",109        "CRIT2FL",110    )111    .sort(["USUBJID", "ASEQ"])112)113114Path("/app/output").mkdir(exist_ok=True)115adlb.write_csv("/app/output/adlb.csv")

solutions/adam-adlb-shift-criteria/result.sas

Edit35 lines

1/* opensas reference solution for the yamaa benchmark adam-adlb-shift-criteria.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data adlb_raw;6  length STUDYID USUBJID PARAMCD PARAM AVISIT ABLFL $1024;7  infile "/app/input/adlb_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. PARAMCD : $char1024. PARAM : $char1024. ASEQ AVISIT : $char1024. AVAL ANRLO ANRHI ABLFL : $char1024.;9run;1011data marked;12  length ANRIND $6;13  set adlb_raw;14  if not missing(AVAL) and not missing(ANRLO) and not missing(ANRHI) then do;15    if AVAL<ANRLO then ANRIND='LOW'; else if AVAL>ANRHI then ANRIND='HIGH'; else ANRIND='NORMAL';16  end;17run;18proc sql; create table combined as select a.*,b.AVAL as BASE,b.ANRIND as BNRIND from marked as a left join marked as b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.PARAMCD=b.PARAMCD and b.ABLFL='Y'; quit;19data result;20  length SHIFT1 $20 CRIT1 $30 CRIT2 $30 CRIT1FL CRIT2FL $1;21  set combined;22  if not missing(ANRIND) and not missing(BNRIND) then SHIFT1=catx(' to ',BNRIND,ANRIND);23  if not missing(AVAL) and not missing(BASE) and BASE ne 0 then R2BASE=AVAL/BASE;24  if not missing(AVAL) and not missing(ANRHI) then do;25    CRIT1='Result greater than 3 x ULN'; CRIT1FL='N'; if AVAL>3*ANRHI then CRIT1FL='Y';26  end;27  if not missing(AVAL) and not missing(ANRLO) then do;28    CRIT2='Result less than LLN'; CRIT2FL='N'; if AVAL<ANRLO then CRIT2FL='Y';29  end;30run;3132proc sql;33  create table final as select STUDYID, USUBJID, PARAMCD, PARAM, ASEQ, AVISIT, AVAL, ANRLO, ANRHI, ANRIND, ABLFL, BASE, BNRIND, SHIFT1, R2BASE, CRIT1, CRIT1FL, CRIT2, CRIT2FL from result;34quit;35proc export data=final outfile="/app/output/adlb.csv" dbms=csv replace; run;

input/adlb_raw.csv

expected/adlb.csv

Comments

Mapping spec

Agent prompt

prompts/adam-adlb-shift-criteria/full.md

prompts/adam-adlb-shift-criteria/conventions.md

prompts/adam-adlb-shift-criteria/brief.md

yamaa code

run.R

run.py

Solution without yamaa

solutions/adam-adlb-shift-criteria/result.R

solutions/adam-adlb-shift-criteria/result.py

solutions/adam-adlb-shift-criteria/result.sas