As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC standards: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: code reported medication names with the World Health Organization drug dictionary (WHODrug), carrying the coder's chosen drug record and Anatomical Therapeutic Chemical (ATC) class into CM.
Input: ODM item data for the medication log: each reported
medication is one repeat of the IG.CM item group, the reported name
is item IT.CM.CMTRT, and the repeat number becomes CMSEQ; the
coder's WHODrug coding output (chosen drug record and ATC code per
collected record); and a WHODrug dictionary extract.
Variables:
CMTRTis the medication name as reported, kept exactly as written.CMDECODis the preferred name of the drug record the coder chose for the reported name; blank when the verbatim is not yet coded.CMCLASis the class name of the ATC code the coder assigned for this use; blank when the verbatim is not yet coded.CMCLASCDis the code of that class; blank when the verbatim is not yet coded.
Note: one drug can carry several ATC codes, and the coder assigns
the class for the use at hand: the same drug record can code to a
different CMCLAS on different records.
Standard: SDTM | Domain: CM
Inputs
Under review3 source filesinput/coding_output.csv
Edit3 rows| STUDYID | USUBJID | CMSEQ | DRUG_RECORD_NO | DRUG_CODE | ATC_CODE |
|---|---|---|---|---|---|
| YAMAA-01 | 1 | 1001 | IBU-200 | M01AE01 | |
| YAMAA-01 | 2 | 1002 | ASA-100 | N02BA01 | |
| YAMAA-01 | 1 | 1002 | ASA-100 | B01AC06 |
input/odm.csv
Edit4 rows| StudyOID | MetaDataVersionOID | SubjectKey | StudyEventOID | StudyEventRepeatKey | ItemGroupOID | ItemGroupRepeatKey | ItemOID | Value |
|---|---|---|---|---|---|---|---|---|
| YAMAA-01 | MDV.1 | 001-101 | TREATMENT | 1 | IG.CM | 1 | IT.CM.CMTRT | ADVIL |
| YAMAA-01 | MDV.1 | 001-101 | TREATMENT | 1 | IG.CM | 2 | IT.CM.CMTRT | Asprin |
| YAMAA-01 | MDV.1 | 001-102 | TREATMENT | 1 | IG.CM | 1 | IT.CM.CMTRT | ASPIRIN |
| YAMAA-01 | MDV.1 | 001-102 | TREATMENT | 1 | IG.CM | 2 | IT.CM.CMTRT | TURMERIC |
input/whodrug_extract.csv
Edit3 rows| DRUG_RECORD_NO | PREFERRED_NAME | DRUG_CODE | ATC_CODE | ATC_CLASS_CODE | ATC_CLASS_NAME |
|---|---|---|---|---|---|
| 1001 | IBUPROFEN | IBU-200 | M01AE01 | M01AE | Propionic acid derivatives |
| 1002 | ACETYLSALICYLIC ACID | ASA-100 | N02BA01 | N02BA | Salicylic acid and derivatives |
| 1002 | ACETYLSALICYLIC ACID | ASA-100 | B01AC06 | B01AC | Platelet aggregation inhibitors excl. heparin |
Expected output
Under review4 expected rowsexpected/cm.csv
Edit4 rows| DOMAIN | STUDYID | USUBJID | CMSEQ | CMTRT | CMDECOD | CMCLAS | CMCLASCD |
|---|---|---|---|---|---|---|---|
| CM | YAMAA-01 | 1 | ADVIL | IBUPROFEN | Propionic acid derivatives | M01AE | |
| CM | YAMAA-01 | 2 | Asprin | ACETYLSALICYLIC ACID | Salicylic acid and derivatives | N02BA | |
| CM | YAMAA-01 | 1 | ASPIRIN | ACETYLSALICYLIC ACID | Platelet aggregation inhibitors excl. heparin | B01AC | |
| CM | YAMAA-01 | 2 | TURMERIC | — | — | — |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | Assigned | Set constant value "CM". | SDTM | 1 | ||||
| STUDYID | Study Identifier | Char | Collected | Copy value from ODM.StudyOID. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from ODM.SubjectKey. | SDTM | 3 | ||||
| CMSEQ | Sequence Number | Num | Collected | Copy value from ODM.ItemGroupRepeatKey. | SDTM | 4 | ||||
| CMTRT | Reported Name of the Medication | Char | Derived | Set per constructed row; see the Row construction sheet. | SDTM | 5 | ||||
| CMDECOD | Standardized Medication Name | Char | Derived | Take PREFERRED_NAME from the WHODRUG record whose DRUG_RECORD_NO equals DRUG_RECORD_NO and ATC_CODE equals ATC_CODE, and no match yields blank. | SDTM | 6 | ||||
| CMCLAS | Medication Class | Char | Derived | Take ATC_CLASS_NAME from the WHODRUG record whose DRUG_RECORD_NO equals DRUG_RECORD_NO and ATC_CODE equals ATC_CODE, and no match yields blank. | SDTM | 7 | ||||
| CMCLASCD | Medication Class Code | Char | Derived | Take ATC_CLASS_CODE from the WHODRUG record whose DRUG_RECORD_NO equals DRUG_RECORD_NO and ATC_CODE equals ATC_CODE, and no match yields blank. | SDTM | 8 |
| Template | Include when | CMTRT | DRUG_RECORD_NO | ATC_CODE |
|---|---|---|---|---|
| medication | Aggregate ONLY(ODM.Value) over records matching on the shared output keys, where ODM.ItemOID = 'IT.CM.CMTRT' | Copy DRUG_RECORD_NO from the CODING record whose STUDYID equals ODM.StudyOID, USUBJID equals ODM.SubjectKey and CMSEQ equals ODM.ItemGroupRepeatKey, and no match yields blank | Copy ATC_CODE from the CODING record whose STUDYID equals ODM.StudyOID, USUBJID equals ODM.SubjectKey and CMSEQ equals ODM.ItemGroupRepeatKey, and no match yields blank |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-cm-whodrug-coding/full.md
Edit22 linesFollowing CDISC SDTM standards, use the provided ODM, CODING, and WHODRUG datasets to create a CM dataset with one record per reported medication.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, CMSEQ, CMTRT, CMDECOD, CMCLAS, CMCLASCD
STUDYID is the ODM StudyOID and USUBJID is the SubjectKey, each as given.
CMSEQ is the medication repeat number within the subject. CMTRT keeps the medication name exactly as written. CMDECOD is the preferred name of the drug record the coder chose for the reported name, and has no value when the reported name is not yet coded. CMCLAS is the class name of the ATC code the coder assigned for this use, and CMCLASCD is the code of that class; both have no value when the reported name is not yet coded. One drug can carry several ATC codes, and the coder assigns the class for the use at hand, so the same drug record can give a different class on different records. A record with no coding, or a code not in the extract, leaves the coded variables with no value.
Read the source datasets from /app/input and save the completed dataset as /app/output/cm.csv.
prompts/sdtm-cm-whodrug-coding/conventions.md
Edit9 linesFollowing CDISC SDTM standards, use the provided ODM, CODING, and WHODRUG datasets to create a CM dataset with one record per reported medication.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, CMSEQ, CMTRT, CMDECOD, CMCLAS, CMCLASCD
Read the source datasets from /app/input and save the completed dataset as /app/output/cm.csv.
prompts/sdtm-cm-whodrug-coding/brief.md
Edit9 linesFollowing CDISC SDTM standards, use the provided ODM, CODING, and WHODRUG datasets to create a CM dataset with one record per reported medication.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, CMSEQ, CMTRT, CMDECOD, CMCLAS, CMCLASCD
Read the source datasets from /app/input and save the completed dataset as /app/output/cm.csv.
run.R
Edit4 lineslibrary(yamaa)cm <- yamaa_domain("spec.yaml")$outputcmrun.py
Edit4 linesimport yamaacm = yamaa.yamaa_domain("spec.yaml").outputcmsolutions/sdtm-cm-whodrug-coding/result.R
Edit61 lines# Reference solution for the yamaa benchmark sdtm-cm-whodrug-coding (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)library(tidyr)odm <- read_csv( "/app/input/odm.csv", col_types = cols(.default = col_character()))coding <- read_csv( "/app/input/coding_output.csv", col_types = cols(CMSEQ = col_integer(), .default = col_character()))whodrug <- read_csv( "/app/input/whodrug_extract.csv", col_types = cols(.default = col_character()))# One record per reported medication; the repeat number is the sequence# number. The preferred name comes from the drug record the coder chose,# and the class from the ATC code assigned for this use, so the same drug# record can give a different class on different records. No coding, or a# code not in the extract, leaves the coded variables empty.reported <- odm |> filter(ItemGroupOID == "IG.CM") |> pivot_wider( id_cols = c(StudyOID, SubjectKey, ItemGroupRepeatKey), names_from = ItemOID, values_from = Value, values_fn = first ) |> mutate( STUDYID = StudyOID, USUBJID = SubjectKey, CMSEQ = as.integer(ItemGroupRepeatKey), CMTRT = `IT.CM.CMTRT` ) |> select(STUDYID, USUBJID, CMSEQ, CMTRT)cm <- reported |> left_join(coding, by = c("STUDYID", "USUBJID", "CMSEQ")) |> left_join(whodrug, by = c("DRUG_RECORD_NO", "ATC_CODE")) |> mutate( DOMAIN = "CM", CMDECOD = if_else( is.na(DRUG_RECORD_NO) | DRUG_RECORD_NO == "" | is.na(PREFERRED_NAME), NA_character_, PREFERRED_NAME ), CMCLAS = if_else(is.na(CMDECOD), NA_character_, ATC_CLASS_NAME), CMCLASCD = if_else(is.na(CMDECOD), NA_character_, ATC_CLASS_CODE) ) |> arrange(USUBJID, CMSEQ) |> select(DOMAIN, STUDYID, USUBJID, CMSEQ, CMTRT, CMDECOD, CMCLAS, CMCLASCD)dir.create("/app/output", showWarnings = FALSE)write_csv(cm, "/app/output/cm.csv", na = "")solutions/sdtm-cm-whodrug-coding/result.py
Edit82 lines# Reference solution for the yamaa benchmark sdtm-cm-whodrug-coding (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as plodm = pl.read_csv("/app/input/odm.csv", infer_schema=False)coding = pl.read_csv("/app/input/coding_output.csv", infer_schema=False).with_columns( pl.col("CMSEQ").cast(pl.Int64))whodrug = pl.read_csv("/app/input/whodrug_extract.csv", infer_schema=False)# One record per reported medication; the repeat number is the sequence# number. The preferred name comes from the drug record the coder chose,# and the class from the ATC code assigned for this use, so the same drug# record can give a different class on different records. No coding, or a# code not in the extract, leaves the coded variables empty.reported = ( odm.filter(pl.col("ItemGroupOID") == "IG.CM") .pivot( index=["StudyOID", "SubjectKey", "ItemGroupRepeatKey"], on="ItemOID", values="Value", aggregate_function="first", ) .with_columns( STUDYID=pl.col("StudyOID"), USUBJID=pl.col("SubjectKey"), CMSEQ=pl.col("ItemGroupRepeatKey").cast(pl.Int64), CMTRT=pl.col("IT.CM.CMTRT"), ) .select("STUDYID", "USUBJID", "CMSEQ", "CMTRT"))cm = ( reported.join(coding, on=["STUDYID", "USUBJID", "CMSEQ"], how="left") .join( whodrug, left_on=["DRUG_RECORD_NO", "ATC_CODE"], right_on=["DRUG_RECORD_NO", "ATC_CODE"], how="left", ) .with_columns( DOMAIN=pl.lit("CM"), CMDECOD=pl.when(pl.col("PREFERRED_NAME").is_null()) .then(None) .otherwise(pl.col("PREFERRED_NAME")), CMCLAS=pl.col("ATC_CLASS_NAME"), CMCLASCD=pl.col("ATC_CLASS_CODE"), ) .with_columns( CMDECOD=pl.when( pl.col("DRUG_RECORD_NO").fill_null("") == "" ) .then(None) .otherwise(pl.col("CMDECOD")), CMCLAS=pl.when(pl.col("CMDECOD").is_null()) .then(None) .otherwise(pl.col("CMCLAS")), CMCLASCD=pl.when(pl.col("CMDECOD").is_null()) .then(None) .otherwise(pl.col("CMCLASCD")), ) .sort(["USUBJID", "CMSEQ"]) .select( "DOMAIN", "STUDYID", "USUBJID", "CMSEQ", "CMTRT", "CMDECOD", "CMCLAS", "CMCLASCD", ))Path("/app/output").mkdir(exist_ok=True)cm.write_csv("/app/output/cm.csv")solutions/sdtm-cm-whodrug-coding/result.sas
Edit43 lines/* opensas reference solution for the yamaa benchmark sdtm-cm-whodrug-coding. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data coding_output; length STUDYID USUBJID DRUG_RECORD_NO DRUG_CODE ATC_CODE $1024; infile "/app/input/coding_output.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDYID : $char1024. USUBJID : $char1024. CMSEQ DRUG_RECORD_NO : $char1024. DRUG_CODE : $char1024. ATC_CODE : $char1024.;run;data odm; length StudyOID MetaDataVersionOID SubjectKey StudyEventOID StudyEventRepeatKey ItemGroupOID ItemGroupRepeatKey ItemOID Value $1024; infile "/app/input/odm.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input StudyOID : $char1024. MetaDataVersionOID : $char1024. SubjectKey : $char1024. StudyEventOID : $char1024. StudyEventRepeatKey : $char1024. ItemGroupOID : $char1024. ItemGroupRepeatKey : $char1024. ItemOID : $char1024. Value : $char1024.;run;data whodrug_extract; length DRUG_RECORD_NO PREFERRED_NAME DRUG_CODE ATC_CODE ATC_CLASS_CODE ATC_CLASS_NAME $1024; infile "/app/input/whodrug_extract.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input DRUG_RECORD_NO : $char1024. PREFERRED_NAME : $char1024. DRUG_CODE : $char1024. ATC_CODE : $char1024. ATC_CLASS_CODE : $char1024. ATC_CLASS_NAME : $char1024.;run;proc sort data=odm out=ordered; by StudyOID SubjectKey ItemGroupRepeatKey; run;data wide; length CMTRT $1024; retain CMTRT; set ordered; where ItemGroupOID='IG.CM'; by StudyOID SubjectKey ItemGroupRepeatKey; if first.ItemGroupRepeatKey then do; CMTRT=''; end; if ItemOID='IT.CM.CMTRT' then CMTRT=Value; if last.ItemGroupRepeatKey then output;run;data medications; set wide; length DOMAIN $2 STUDYID USUBJID $1024; DOMAIN='CM'; STUDYID=StudyOID; USUBJID=SubjectKey; CMSEQ=input(ItemGroupRepeatKey,best32.); run;proc sql; create table result as select a.*,c.PREFERRED_NAME as CMDECOD,c.ATC_CLASS_NAME as CMCLAS,c.ATC_CLASS_CODE as CMCLASCD from medications aleft join coding_output b on a.STUDYID=b.STUDYID and a.USUBJID=b.USUBJID and a.CMSEQ=b.CMSEQleft join whodrug_extract c on b.DRUG_RECORD_NO=c.DRUG_RECORD_NO and b.DRUG_CODE=c.DRUG_CODE and b.ATC_CODE=c.ATC_CODE; quit;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, CMSEQ, CMTRT, CMDECOD, CMCLAS, CMCLASCD from result;quit;proc export data=final outfile="/app/output/cm.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions