Supplemental Qualifiers

Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Goal: build one supplemental record per collected qualifier, carrying IDVARVAL, QLABEL, QVAL, QORIG, and QEVAL, with each record pointing back at its parent medical history (MH) record.

Input: a medical history extract with one record per condition, carrying the parent record sequence and the two collected qualifiers.

Variables:

IDVARVAL is the parent record sequence, written as text; a parent record with no sequence number stops the run.
QLABEL is the qualifier label: Family History for MHFAMHX, Confirmed by Medical Records for MHCONF.
QVAL is the collected qualifier value, Y or N.
QORIG records the collected origin as Collected (Define-XML 2.1).
QEVAL is blank, since a collected value is not an assessment.

Note: each parent record contributes one supplemental record per qualifier actually collected, so a record with only one of the two collected contributes only one. Records are ordered by study, then subject, then parent record sequence compared as text (so 10 sorts before 2), then qualifier name.

YAML specification

1 spec file

spec.yaml

Edit98 lines

1schema_version: "1.0"2domain: SUPPMH3keys: [STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM]4input:5  MH_RAW: input/mh_raw.csv67output:8  path: suppmh.csv9  columns: [STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL,10            QORIG, QEVAL]11  # IDVARVAL is text, so parent sequences sort as text: 10 before 2.12  order_by: [STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM]1314columns:15  - name: STUDYID16    type: str17    label: Study Identifier18    derivation: MH_RAW.STUDYID1920  - name: RDOMAIN21    type: str22    label: Related Domain Abbreviation23    derivation: {literal: MH}2425  - name: USUBJID26    type: str27    label: Unique Subject Identifier28    derivation: MH_RAW.USUBJID2930  - name: IDVAR31    type: str32    label: Identifying Variable33    derivation: {literal: MHSEQ}3435  - name: IDVARVAL36    type: str37    label: Identifying Variable Value38    derivation: MH_RAW.MHSEQ39    verifications:40      not_missing: {}4142  - name: QNAM43    type: str44    label: Qualifier Variable Name45    verifications:46      allowed_values:47        values: [MHFAMHX, MHCONF]4849  - name: QLABEL50    type: str51    label: Qualifier Label5253  - name: QVAL54    type: str55    label: Data Value56    verifications:57      - not_missing: {}58      - allowed_values:59          values: [Y, N]6061  - name: QORIG62    type: str63    label: Origin64    derivation: {literal: Collected}6566  - name: QEVAL67    type: str68    label: Evaluator69    derivation: {literal: null}7071# One row template per qualifier. Each writes a row only for an MH record72# where that qualifier was collected; a blank answer writes none.73rows:74  - id: family_history75    filter: "MH_RAW.MHFAMHX IS NOT NULL"76    derivations:77      QNAM: {literal: MHFAMHX}78      QLABEL: {literal: Family History}79      QVAL: MH_RAW.MHFAMHX8081  - id: medical_record_confirmation82    filter: "MH_RAW.MHCONF IS NOT NULL"83    derivations:84      QNAM: {literal: MHCONF}85      QLABEL: {literal: Confirmed by Medical Records}86      QVAL: MH_RAW.MHCONF8788verifications:89  - unique:90      columns: [STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM]91  - row_count:92      min: 793      max: 794  - assert:95      id: qualifier-value-has-parent-identifier96      when: "QVAL IS NOT NULL"97      require: "RDOMAIN IS NOT NULL AND IDVAR IS NOT NULL98        AND IDVARVAL IS NOT NULL"

Inputs

Under review1 source file

input/mh_raw.csv

Edit4 rows

input/mh_raw.csv
STUDYID	MHSEQ	MHTERM	MHFAMHX	MHCONF
CATH	1	ASTHMA	Y	Y
CATH	2	ECZEMA	N	—
CATH	1	HYPERTENSION	N	N
CATH	2	MIGRAINE	Y	Y

Expected output

Under review7 expected rows

expected/suppmh.csv

Edit7 rows

expected/suppmh.csv
STUDYID	RDOMAIN	IDVAR	IDVARVAL	QNAM	QLABEL	QVAL	QORIG	QEVAL
CATH	MH	MHSEQ	1	MHCONF	Confirmed by Medical Records	Y	Collected	—
CATH	MH	MHSEQ	1	MHFAMHX	Family History	Y	Collected	—
CATH	MH	MHSEQ	2	MHFAMHX	Family History	N	Collected	—
CATH	MH	MHSEQ	1	MHCONF	Confirmed by Medical Records	N	Collected	—
CATH	MH	MHSEQ	1	MHFAMHX	Family History	N	Collected	—
CATH	MH	MHSEQ	2	MHCONF	Confirmed by Medical Records	Y	Collected	—
CATH	MH	MHSEQ	2	MHFAMHX	Family History	Y	Collected	—

Shaded columns are derived— = missing value

Comments and assessment materials

Variable Name	Variable Label	Type	Controlled Terms or Format	Origin	Conversion Definition	Variable Type	Variable Order
STUDYID	Study Identifier	Char		Collected	Copy value from MH_RAW.STUDYID.	SUPP	1
RDOMAIN	Related Domain Abbreviation	Char		Assigned	Set constant value "MH".	SUPP	2
USUBJID	Unique Subject Identifier	Char		Collected	Copy value from MH_RAW.USUBJID.	SUPP	3
IDVAR	Identifying Variable	Char		Assigned	Set constant value "MHSEQ".	SUPP	4
IDVARVAL	Identifying Variable Value	Char		Collected	Copy value from MH_RAW.MHSEQ.	SUPP	5
QNAM	Qualifier Variable Name	Char	MHFAMHX, MHCONF	Derived	Set per constructed row; see the Row construction sheet.	SUPP	6
QLABEL	Qualifier Label	Char		Derived	Set per constructed row; see the Row construction sheet.	SUPP	7
QVAL	Data Value	Char	Y, N	Derived	Set per constructed row; see the Row construction sheet.	SUPP	8
QORIG	Origin	Char		Assigned	Set constant value "Collected".	SUPP	9
QEVAL	Evaluator	Char		Assigned	Set constant value "None".	SUPP	10

Template	Include when	QNAM	QLABEL	QVAL
family_history	MH_RAW.MHFAMHX IS NOT NULL	Constant "MHFAMHX"	Constant "Family History"	Copy MH_RAW.MHFAMHX
medical_record_confirmation	MH_RAW.MHCONF IS NOT NULL	Constant "MHCONF"	Constant "Confirmed by Medical Records"	Copy MH_RAW.MHCONF

Variable	Permitted values
QNAM	MHFAMHX, MHCONF
QVAL	Y, N

Version	Date	Author	Description
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-suppmh-qualifiers/full.md

Edit20 lines

Following CDISC SDTM standards, use the provided MH_RAW dataset to create a SUPPMH dataset with one supplemental record per collected qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

RDOMAIN is MH and IDVAR is MHSEQ. IDVARVAL is the parent record sequence, written as text. QNAM is the qualifier name, MHFAMHX or MHCONF. QLABEL is Family History for MHFAMHX and Confirmed by Medical Records for MHCONF. QVAL is the collected qualifier value, Y or N. QORIG records the collected origin as Collected. QEVAL has no value, since a collected value is not an assessment.

Each parent record contributes one supplemental record per qualifier actually collected, so a record with only one of the two collected contributes only one.

Read the source datasets from /app/input and save the completed dataset as /app/output/suppmh.csv.

prompts/sdtm-suppmh-qualifiers/conventions.md

Edit14 lines

Following CDISC SDTM standards, use the provided MH_RAW dataset to create a SUPPMH dataset with one supplemental record per collected qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

RDOMAIN is MH and IDVAR is MHSEQ. IDVARVAL is the parent record sequence, written as text. QNAM is the qualifier name, MHFAMHX or MHCONF. QLABEL is Family History for MHFAMHX and Confirmed by Medical Records for MHCONF. QVAL is Y or N. QORIG records the collected origin as Collected. QEVAL has no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/suppmh.csv.

prompts/sdtm-suppmh-qualifiers/brief.md

Edit9 lines

Following CDISC SDTM standards, use the provided MH_RAW dataset to create a SUPPMH dataset with one supplemental record per collected qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

Read the source datasets from /app/input and save the completed dataset as /app/output/suppmh.csv.

run.R

Edit4 lines

1library(yamaa)23suppmh <- yamaa_domain("spec.yaml")$output4suppmh

run.py

Edit4 lines

1import yamaa23suppmh = yamaa.yamaa_domain("spec.yaml").output4suppmh

solutions/sdtm-suppmh-qualifiers/result.R

Edit40 lines

1# Reference solution for the yamaa benchmark sdtm-suppmh-qualifiers (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)9library(tidyr)1011mh <- read_csv(12  "/app/input/mh_raw.csv",13  col_types = cols(.default = col_character())14)1516suppmh <- mh |>17  pivot_longer(18    cols = c(MHFAMHX, MHCONF),19    names_to = "QNAM",20    values_to = "QVAL"21  ) |>22  filter(!is.na(QVAL) & QVAL != "") |>23  mutate(24    RDOMAIN = "MH",25    IDVAR = "MHSEQ",26    IDVARVAL = MHSEQ,27    QLABEL = case_when(28      QNAM == "MHFAMHX" ~ "Family History",29      QNAM == "MHCONF" ~ "Confirmed by Medical Records"30    ),31    QORIG = "Collected",32    QEVAL = NA_character_33  ) |>34  arrange(USUBJID, suppressWarnings(as.integer(MHSEQ)), QNAM) |>35  select(36    STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL37  )3839dir.create("/app/output", showWarnings = FALSE)40write_csv(suppmh, "/app/output/suppmh.csv", na = "")

solutions/sdtm-suppmh-qualifiers/result.py

Edit55 lines

1# Reference solution for the yamaa benchmark sdtm-suppmh-qualifiers (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011mh = pl.read_csv("/app/input/mh_raw.csv", infer_schema=False)1213labels = {"MHFAMHX": "Family History", "MHCONF": "Confirmed by Medical Records"}1415records = []16for qnam in ("MHFAMHX", "MHCONF"):17    collected = mh.filter(18        pl.col(qnam).is_not_null() & (pl.col(qnam) != "")19    ).select(20        "STUDYID",21        "USUBJID",22        "MHSEQ",23        QNAM=pl.lit(qnam),24        QLABEL=pl.lit(labels[qnam]),25        QVAL=pl.col(qnam),26    )27    records.append(collected)2829suppmh = (30    pl.concat(records)31    .with_columns(32        RDOMAIN=pl.lit("MH"),33        IDVAR=pl.lit("MHSEQ"),34        IDVARVAL=pl.col("MHSEQ"),35        QORIG=pl.lit("Collected"),36        QEVAL=pl.lit(None, dtype=pl.String),37    )38    .with_columns(pl.col("MHSEQ").cast(pl.Int64).alias("_seq"))39    .sort(["USUBJID", "_seq", "QNAM"])40    .select(41        "STUDYID",42        "RDOMAIN",43        "USUBJID",44        "IDVAR",45        "IDVARVAL",46        "QNAM",47        "QLABEL",48        "QVAL",49        "QORIG",50        "QEVAL",51    )52)5354Path("/app/output").mkdir(exist_ok=True)55suppmh.write_csv("/app/output/suppmh.csv")

solutions/sdtm-suppmh-qualifiers/result.sas

Edit18 lines

1/* opensas reference solution for the yamaa benchmark sdtm-suppmh-qualifiers.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data mh_raw;6  length STUDYID USUBJID MHTERM MHFAMHX MHCONF $1024;7  infile "/app/input/mh_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. MHSEQ MHTERM : $char1024. MHFAMHX : $char1024. MHCONF : $char1024.;9run;1011data qualified; set mh_raw; run;12data result; set qualified; length RDOMAIN IDVAR IDVARVAL QNAM QLABEL QVAL QORIG QEVAL $1024; RDOMAIN='MH'; IDVAR='MHSEQ'; IDVARVAL=strip(put(MHSEQ,best32.)); QORIG='Collected'; QEVAL='';13if not missing(MHFAMHX) then do; QNAM='MHFAMHX'; QLABEL='Family History'; QVAL=MHFAMHX; output; end;14if not missing(MHCONF) then do; QNAM='MHCONF'; QLABEL='Confirmed by Medical Records'; QVAL=MHCONF; output; end; run;15proc sql;16  create table final as select STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL from result;17quit;18proc export data=final outfile="/app/output/suppmh.csv" dbms=csv replace; run;

input/mh_raw.csv

expected/suppmh.csv

Comments

Mapping spec

Agent prompt

prompts/sdtm-suppmh-qualifiers/full.md

prompts/sdtm-suppmh-qualifiers/conventions.md

prompts/sdtm-suppmh-qualifiers/brief.md

yamaa code

run.R

run.py

Solution without yamaa

solutions/sdtm-suppmh-qualifiers/result.R

solutions/sdtm-suppmh-qualifiers/result.py

solutions/sdtm-suppmh-qualifiers/result.sas