As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: emit one related-records row per relationship
participation, carrying IDVARVAL, RELTYPE, and RELID.
Input: the finished AE and CM datasets: each record already carries an assigned sequence number (AESEQ on AE, CMSEQ on CM) and up to two link identifiers (AELNKID1/AELNKID2 on AE, CMLNKID1/CMLNKID2 on CM). A second subject reuses link number 1, showing that link numbers are per-subject: the USUBJID key keeps the two relationships apart.
Variables:
IDVARVALis the sequence number of the related record as text; always present.RELTYPEis blank throughout, because each row points at one record rather than a whole dataset.RELIDnames the relationship the row takes part in; rows sharing a value are related to one another, and every row carries one.
Note: a record with no link identifier contributes no row, while a record naming two link identifiers contributes one row per identifier - only when the two identifiers differ. A record repeating its link number in both fields produces two identical rows, which fails the run on the duplicate key; carrying a third relationship would need another link field on the collected record.
Negative case: a link number carried by only one record is a
dangling relationship and fails the run. Adding the collected row
CATH,CATH-UCSD-0001,4,COUGH,3, to ae.csv (AELNKID1=3, no other
record carrying link 3) emits one RELREC row, and the
each-relationship-has-two-or-more-rows check then fails: the
group (STUDYID=CATH, USUBJID=CATH-UCSD-0001, RELID=3) holds 1 row,
below the minimum of 2. That is why the passing input above
carries no such row.
Inputs
Under review2 source filesExpected output
Under review8 expected rowsexpected/relrec.csv
Edit8 rows| STUDYID | USUBJID | RDOMAIN | IDVAR | IDVARVAL | RELTYPE | RELID |
|---|---|---|---|---|---|---|
| CATH | AE | AESEQ | 1 | — | 1 | |
| CATH | CM | CMSEQ | 1 | — | 1 | |
| CATH | CM | CMSEQ | 2 | — | 1 | |
| CATH | AE | AESEQ | 1 | — | 2 | |
| CATH | AE | AESEQ | 2 | — | 2 | |
| CATH | CM | CMSEQ | 2 | — | 2 | |
| CATH | AE | AESEQ | 1 | — | 1 | |
| CATH | CM | CMSEQ | 1 | — | 1 |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| STUDYID | Study Identifier | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 1 | ||||
| USUBJID | Unique Subject Identifier | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 2 | ||||
| RDOMAIN | Related Domain Abbreviation | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 3 | ||||
| IDVAR | Identifying Variable | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 4 | ||||
| IDVARVAL | Identifying Variable Value | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 5 | ||||
| RELTYPE | Relationship Type | Char | Assigned | Set constant value "None". | SDTM | 6 | ||||
| RELID | Relationship Identifier | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 7 |
| Template | Include when | STUDYID | USUBJID | RDOMAIN | IDVAR | IDVARVAL | RELID | SEQNUM |
|---|---|---|---|---|---|---|---|---|
| ae_link_1 | AE.AELNKID1 IS NOT NULL | Copy AE.STUDYID | Copy AE.USUBJID | Constant "AE" | Constant "AESEQ" | Copy AE.AESEQ | Copy AE.AELNKID1 | Copy AE.AESEQ |
| ae_link_2 | AE.AELNKID2 IS NOT NULL | Copy AE.STUDYID | Copy AE.USUBJID | Constant "AE" | Constant "AESEQ" | Copy AE.AESEQ | Copy AE.AELNKID2 | Copy AE.AESEQ |
| cm_link_1 | CM.CMLNKID1 IS NOT NULL | Copy CM.STUDYID | Copy CM.USUBJID | Constant "CM" | Constant "CMSEQ" | Copy CM.CMSEQ | Copy CM.CMLNKID1 | Copy CM.CMSEQ |
| cm_link_2 | CM.CMLNKID2 IS NOT NULL | Copy CM.STUDYID | Copy CM.USUBJID | Constant "CM" | Constant "CMSEQ" | Copy CM.CMSEQ | Copy CM.CMLNKID2 | Copy CM.CMSEQ |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-relrec-links/full.md
Edit20 linesFollowing CDISC SDTM standards, use the provided AE and CM datasets to create a RELREC dataset with one row per relationship participation.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID
RDOMAIN names the related domain, and IDVAR its sequence variable (AESEQ on AE rows, CMSEQ on CM rows). IDVARVAL is the sequence number of the related record as text, and is always present. RELTYPE has no value throughout, because each row points at one record rather than a whole dataset. RELID names the relationship the row takes part in: it is the collected link number, written as text without a decimal point, so rows sharing a value are related to one another, and every row carries one.
A record with no link identifier contributes no row, while a record naming two link identifiers contributes one row per identifier. Link numbers are per-subject: USUBJID keeps a reused number in distinct relationships.
Read the source datasets from /app/input and save the completed dataset as /app/output/relrec.csv.
prompts/sdtm-relrec-links/conventions.md
Edit12 linesFollowing CDISC SDTM standards, use the provided AE and CM datasets to create a RELREC dataset with one row per relationship participation.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID
IDVAR is AESEQ on AE rows and CMSEQ on CM rows. IDVARVAL is the sequence number of the related record as text. RELTYPE has no value throughout. RELID is the collected link number, written as text without a decimal point.
Read the source datasets from /app/input and save the completed dataset as /app/output/relrec.csv.
prompts/sdtm-relrec-links/brief.md
Edit8 linesFollowing CDISC SDTM standards, use the provided AE and CM datasets to create a RELREC dataset with one row per relationship participation.
The output dataset should contain the following columns in this order: STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID
Read the source datasets from /app/input and save the completed dataset as /app/output/relrec.csv.
run.R
Edit4 lineslibrary(yamaa)relrec <- yamaa_domain("spec.yaml")$outputrelrecrun.py
Edit4 linesimport yamaarelrec = yamaa.yamaa_domain("spec.yaml").outputrelrecsolutions/sdtm-relrec-links/result.R
Edit60 lines# Reference solution for the yamaa benchmark sdtm-relrec-links (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)library(tidyr)ae <- read_csv( "/app/input/ae.csv", col_types = cols(.default = col_character()))cm <- read_csv( "/app/input/cm.csv", col_types = cols(.default = col_character()))# A record naming two link identifiers contributes one row per identifier.ae_rows <- ae |> pivot_longer( cols = c(AELNKID1, AELNKID2), values_to = "RELID" ) |> filter(!is.na(RELID) & trimws(RELID) != "") |> mutate( RDOMAIN = "AE", IDVAR = "AESEQ", IDVARVAL = trimws(AESEQ), # Link numbers as text without a decimal point. RELID = as.character(suppressWarnings(as.integer(RELID))), RELTYPE = NA_character_ ) |> select(STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID)cm_rows <- cm |> pivot_longer( cols = c(CMLNKID1, CMLNKID2), values_to = "RELID" ) |> filter(!is.na(RELID) & trimws(RELID) != "") |> mutate( RDOMAIN = "CM", IDVAR = "CMSEQ", IDVARVAL = trimws(CMSEQ), RELID = as.character(suppressWarnings(as.integer(RELID))), RELTYPE = NA_character_ ) |> select(STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID)relrec <- bind_rows(ae_rows, cm_rows) |> arrange( USUBJID, suppressWarnings(as.integer(RELID)), RDOMAIN, suppressWarnings(as.integer(IDVARVAL)) ) |> select(STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID)dir.create("/app/output", showWarnings = FALSE)write_csv(relrec, "/app/output/relrec.csv", na = "")solutions/sdtm-relrec-links/result.py
Edit90 lines# Reference solution for the yamaa benchmark sdtm-relrec-links (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pldef _present(value) -> bool: return value is not None and str(value).strip() != ""def _link(value) -> str | None: """A collected link number as text without a decimal point.""" if not _present(value): return None text = str(value).strip() try: number = float(text) if number.is_integer(): return str(int(number)) return str(number) except ValueError: return textae = pl.read_csv("/app/input/ae.csv", infer_schema=False)cm = pl.read_csv("/app/input/cm.csv", infer_schema=False)rows = []for row in ae.to_dicts(): for link_col in ("AELNKID1", "AELNKID2"): relid = _link(row.get(link_col)) # A record with no link identifier contributes no row. if relid is None: continue rows.append( { "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "RDOMAIN": "AE", "IDVAR": "AESEQ", "IDVARVAL": str(row["AESEQ"]).strip(), "RELTYPE": None, "RELID": relid, } )for row in cm.to_dicts(): for link_col in ("CMLNKID1", "CMLNKID2"): relid = _link(row.get(link_col)) if relid is None: continue rows.append( { "STUDYID": row["STUDYID"], "USUBJID": row["USUBJID"], "RDOMAIN": "CM", "IDVAR": "CMSEQ", "IDVARVAL": str(row["CMSEQ"]).strip(), "RELTYPE": None, "RELID": relid, } )relrec = ( pl.DataFrame( rows, schema={ "STUDYID": pl.String, "USUBJID": pl.String, "RDOMAIN": pl.String, "IDVAR": pl.String, "IDVARVAL": pl.String, "RELTYPE": pl.String, "RELID": pl.String, }, ) .with_columns( pl.col("IDVARVAL").cast(pl.Int64).alias("_seq"), pl.col("RELID").cast(pl.Int64).alias("_rel"), ) .sort(["USUBJID", "_rel", "RDOMAIN", "_seq"]) .select("STUDYID", "USUBJID", "RDOMAIN", "IDVAR", "IDVARVAL", "RELTYPE", "RELID"))Path("/app/output").mkdir(exist_ok=True)relrec.write_csv("/app/output/relrec.csv")solutions/sdtm-relrec-links/result.sas
Edit29 lines/* opensas reference solution for the yamaa benchmark sdtm-relrec-links. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ae; length STUDYID USUBJID AETERM AELNKID1 AELNKID2 $1024; infile "/app/input/ae.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. AESEQ AETERM : $char1024. AELNKID1 : $char1024. AELNKID2 : $char1024.;run;data cm; length STUDYID USUBJID CMTRT CMLNKID1 CMLNKID2 $1024; infile "/app/input/cm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. CMSEQ CMTRT : $char1024. CMLNKID1 : $char1024. CMLNKID2 : $char1024.;run;data aelinks; set ae; length RDOMAIN IDVAR IDVARVAL RELTYPE RELID $1024; RDOMAIN='AE'; IDVAR='AESEQ'; IDVARVAL=strip(put(AESEQ,best32.)); RELTYPE='';if not missing(AELNKID1) then do; RELID=AELNKID1; output; end;if not missing(AELNKID2) then do; RELID=AELNKID2; output; end;run;data cmlinks; set cm; length RDOMAIN IDVAR IDVARVAL RELTYPE RELID $1024; RDOMAIN='CM'; IDVAR='CMSEQ'; IDVARVAL=strip(put(CMSEQ,best32.)); RELTYPE='';if not missing(CMLNKID1) then do; RELID=CMLNKID1; output; end;if not missing(CMLNKID2) then do; RELID=CMLNKID2; output; end;run;data result; set aelinks cmlinks; run;proc sql; create table final as select STUDYID, USUBJID, RDOMAIN, IDVAR, IDVARVAL, RELTYPE, RELID from result;quit;proc export data=final outfile="/app/output/relrec.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions