Supplemental Qualifier Metadata by Qualifier Name

Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: build the supplemental demographics qualifiers QNAM, QLABEL and QVAL for each subject's additional races and other-race text, and describe each qualifier's values separately in the data-definition document.

Input: one collected record per subject with up to two races and the free text a site entered for a race not on the list.

Variables:

RDOMAIN is DM, the domain the qualifiers belong to.
IDVAR and IDVARVAL are missing: a demographics qualifier belongs to the subject rather than to one record.
QNAM is RACE1 or RACE2 for the first and second reported race, and RACEOTH for the other-race text.
QLABEL is the qualifier's label: Race 1, Race 2 or Race, Other.
QVAL is the reported value: a race from the race codelist for RACE1 and RACE2, and the text exactly as entered for RACEOTH.
QORIG is Collected, since every value comes from the form.
QEVAL is missing: no one evaluated the values.

Note: a subject has a qualifier record only for a race or text actually reported. The data-definition document describes QVAL once for the two race qualifiers, drawn from the race codelist and collected on page 2 of the annotated form, and once for the other-race text, collected on page 3. Each description gives the longest value of its own records, so the other-race text is shorter than the variable.

YAML specification

1 spec file

spec.yaml

Edit169 lines

1yamaa: "1.0"2type: spec3model: SDTM4domain: SUPPDM5keys: [STUDYID, USUBJID, QNAM]6input:7  DM_RAW: input/dm_raw.csv89output:10  path: suppdm.csv11  columns: [STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL,12            QORIG, QEVAL]1314columns:15  - name: STUDYID16    type: str17    label: Study Identifier18    derivation: DM_RAW.STUDYID19    submission:20      core: Req21      length: 2022      origins:23        - type: Protocol24          source: Sponsor2526  - name: RDOMAIN27    type: str28    label: Related Domain Abbreviation29    derivation: "'DM'"30    submission:31      core: Req32      length: 233      origins:34        - type: Assigned35          source: Sponsor3637  - name: USUBJID38    type: str39    label: Unique Subject Identifier40    derivation: DM_RAW.USUBJID41    submission:42      core: Req43      length: 3044      origins:45        - type: Assigned46          source: Sponsor4748  - name: IDVAR49    type: str50    label: Identifying Variable51    derivation: null52    submission:53      core: Exp54      length: 855      origins:56        - type: Assigned57          source: Sponsor5859  - name: IDVARVAL60    type: str61    label: Identifying Variable Value62    derivation: null63    submission:64      core: Exp65      length: 4066      origins:67        - type: Assigned68          source: Sponsor6970  - name: QNAM71    type: str72    label: Qualifier Variable Name73    submission:74      core: Req75      length: 876      origins:77        - type: Assigned78          source: Sponsor7980  - name: QLABEL81    type: str82    label: Qualifier Variable Label83    submission:84      core: Req85      length: 4086      origins:87        - type: Assigned88          source: Sponsor8990  # Each qualifier is described by its own definition: the races from the91  # race codelist, the other-race text on its own page of the form.92  - name: QVAL93    type: str94    label: Data Value95    submission:96      core: Req97      length: 20098      origins:99        - type: Collected100          source: Investigator101          documents:102            - document: acrf103              pages: {type: PhysicalRef, refs: "2"}104      value_level:105        - id: RACE106          where: {QNAM: [RACE1, RACE2]}107          label: Race108          codelist: RACE109        - id: RACEOTH110          where: {QNAM: [RACEOTH]}111          label: Race, Other112          origins:113            - type: Collected114              source: Investigator115              documents:116                - document: acrf117                  pages: {type: PhysicalRef, refs: "3"}118119  - name: QORIG120    type: str121    label: Origin122    derivation: "'Collected'"123    submission:124      core: Req125      length: 20126      origins:127        - type: Assigned128          source: Sponsor129130  - name: QEVAL131    type: str132    label: Evaluator133    derivation: null134    submission:135      core: Exp136      length: 40137      origins:138        - type: Assigned139          source: Sponsor140141rows:142  # One qualifier record per subject and reported race or other-race text.143  - id: race1144    filter: "DM_RAW.RACE1 IS NOT NULL"145    derivations:146      QNAM: "'RACE1'"147      QLABEL: "'Race 1'"148      QVAL: DM_RAW.RACE1149150  - id: race2151    filter: "DM_RAW.RACE2 IS NOT NULL"152    derivations:153      QNAM: "'RACE2'"154      QLABEL: "'Race 2'"155      QVAL: DM_RAW.RACE2156157  - id: raceoth158    filter: "DM_RAW.RACEOTH IS NOT NULL"159    derivations:160      QNAM: "'RACEOTH'"161      QLABEL: "'Race, Other'"162      QVAL: DM_RAW.RACEOTH163164submission:165  label: Supplemental Qualifiers for DM166  class: RELATIONSHIP167  structure: One record per supplemental qualifier per subject168  repeating: true169  domain: DM

Inputs

Under review1 source file

input/dm_raw.csv

Edit3 rows

input/dm_raw.csv
STUDYID	RACE1	RACE2	RACEOTH
CATH	WHITE	ASIAN	—
CATH	—	—	Samoan
CATH	BLACK OR AFRICAN AMERICAN	WHITE	—

Expected output

Under review5 expected rows

expected/define.xml

EditDefine-XML

Define-XML 2.1.11: CATH

Example study carrying supplemental demographic qualifiers; CATH-01; SDTMIG 3.4; CDISC/NCI 2023-12-15

Datasets
Dataset	Description	Class	Structure	Purpose	Keys	Documentation	Location
SUPPDM	Supplemental Qualifiers for DM	RELATIONSHIP	One record per supplemental qualifier per subject	Tabulation	STUDYID, USUBJID, QNAM		suppdm.json

SUPPDM

SUPPDM variables
Variable	Label / Description	Type	Length or Display Format	Origin / Source / Method / Comment
STUDYID	Study Identifier	text	4	Protocol; Sponsor
RDOMAIN	Related Domain Abbreviation	text	2	Assigned; Sponsor
USUBJID	Unique Subject Identifier	text	8	Assigned; Sponsor
IDVAR	Identifying Variable	text	1	Assigned; Sponsor
IDVARVAL	Identifying Variable Value	text	1	Assigned; Sponsor
QNAM	Qualifier Variable Name	text	7	Assigned; Sponsor
QLABEL	Qualifier Variable Label	text	11	Assigned; Sponsor
QVAL	Data Value	text	25	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 2
QORIG	Origin	text	9	Assigned; Sponsor
QEVAL	Evaluator	text	1	Assigned; Sponsor

SUPPDM.QVAL value-level metadata
Where Condition	Variable	Label / Description	Type	Length or Display Format	Controlled Terms or ISO Format	Origin / Source / Method / Comment
QNAM in RACE1, RACE2	QVAL	Race	text	25	Race (CL.RACE) AMERICAN INDIAN OR ALASKA NATIVE ASIAN BLACK OR AFRICAN AMERICAN NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER WHITE	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 2
QNAM = RACEOTH	QVAL	Race, Other	text	6		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 3

Original source

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.sdtm.Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study carrying supplemental demographic qualifiers</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.sdtm" Name="CATH sdtm metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.SDTMIG" Name="SDTMIG" Type="IG" Version="3.4" Status="Final"/>
        <def:Standard OID="STD.CT_SDTM" Name="CDISC/NCI" Type="CT" PublishingSet="SDTM" Version="2023-12-15" Status="Final"/>
      </def:Standards>
      <def:AnnotatedCRF>
        <def:DocumentRef leafID="LF.acrf"/>
      </def:AnnotatedCRF>
      <def:ValueListDef OID="VL.SUPPDM.QVAL">
        <ItemRef ItemOID="IT.SUPPDM.QVAL.RACE" OrderNumber="1" Mandatory="Yes">
          <def:WhereClauseRef WhereClauseOID="WC.SUPPDM.QVAL.RACE"/>
        </ItemRef>
        <ItemRef ItemOID="IT.SUPPDM.QVAL.RACEOTH" OrderNumber="2" Mandatory="Yes">
          <def:WhereClauseRef WhereClauseOID="WC.SUPPDM.QVAL.RACEOTH"/>
        </ItemRef>
      </def:ValueListDef>
      <def:WhereClauseDef OID="WC.SUPPDM.QVAL.RACE">
        <RangeCheck Comparator="IN" SoftHard="Soft" def:ItemOID="IT.SUPPDM.QNAM">
          <CheckValue>RACE1</CheckValue>
          <CheckValue>RACE2</CheckValue>
        </RangeCheck>
      </def:WhereClauseDef>
      <def:WhereClauseDef OID="WC.SUPPDM.QVAL.RACEOTH">
        <RangeCheck Comparator="EQ" SoftHard="Soft" def:ItemOID="IT.SUPPDM.QNAM">
          <CheckValue>RACEOTH</CheckValue>
        </RangeCheck>
      </def:WhereClauseDef>
      <ItemGroupDef OID="IG.SUPPDM" Name="SUPPDM" Domain="DM" Purpose="Tabulation" SASDatasetName="SUPPDM" Repeating="Yes" IsReferenceData="No" def:Structure="One record per supplemental qualifier per subject" def:ArchiveLocationID="LF.SUPPDM" def:StandardOID="STD.SDTMIG">
        <Description>
          <TranslatedText xml:lang="en">Supplemental Qualifiers for DM</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.SUPPDM.STUDYID" OrderNumber="1" Mandatory="Yes" KeySequence="1"/>
        <ItemRef ItemOID="IT.SUPPDM.RDOMAIN" OrderNumber="2" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.USUBJID" OrderNumber="3" Mandatory="Yes" KeySequence="2"/>
        <ItemRef ItemOID="IT.SUPPDM.IDVAR" OrderNumber="4" Mandatory="No" def:HasNoData="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.IDVARVAL" OrderNumber="5" Mandatory="No" def:HasNoData="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.QNAM" OrderNumber="6" Mandatory="Yes" KeySequence="3"/>
        <ItemRef ItemOID="IT.SUPPDM.QLABEL" OrderNumber="7" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.QVAL" OrderNumber="8" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.QORIG" OrderNumber="9" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.SUPPDM.QEVAL" OrderNumber="10" Mandatory="No" def:HasNoData="Yes"/>
        <def:Class Name="RELATIONSHIP"/>
        <def:leaf ID="LF.SUPPDM" xlink:href="suppdm.json">
          <def:title>suppdm.json</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.SUPPDM.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Protocol" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.RDOMAIN" Name="RDOMAIN" DataType="text" Length="2" SASFieldName="RDOMAIN">
        <Description>
          <TranslatedText xml:lang="en">Related Domain Abbreviation</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.USUBJID" Name="USUBJID" DataType="text" Length="8" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.IDVAR" Name="IDVAR" DataType="text" Length="1" SASFieldName="IDVAR">
        <Description>
          <TranslatedText xml:lang="en">Identifying Variable</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.IDVARVAL" Name="IDVARVAL" DataType="text" Length="1" SASFieldName="IDVARVAL">
        <Description>
          <TranslatedText xml:lang="en">Identifying Variable Value</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QNAM" Name="QNAM" DataType="text" Length="7" SASFieldName="QNAM">
        <Description>
          <TranslatedText xml:lang="en">Qualifier Variable Name</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QLABEL" Name="QLABEL" DataType="text" Length="11" SASFieldName="QLABEL">
        <Description>
          <TranslatedText xml:lang="en">Qualifier Variable Label</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QVAL" Name="QVAL" DataType="text" Length="25" SASFieldName="QVAL">
        <Description>
          <TranslatedText xml:lang="en">Data Value</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="2" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
        <def:ValueListRef ValueListOID="VL.SUPPDM.QVAL"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QVAL.RACE" Name="QVAL" DataType="text" Length="25" SASFieldName="QVAL">
        <Description>
          <TranslatedText xml:lang="en">Race</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.RACE"/>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="2" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QVAL.RACEOTH" Name="QVAL" DataType="text" Length="6" SASFieldName="QVAL">
        <Description>
          <TranslatedText xml:lang="en">Race, Other</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="3" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QORIG" Name="QORIG" DataType="text" Length="9" SASFieldName="QORIG">
        <Description>
          <TranslatedText xml:lang="en">Origin</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.SUPPDM.QEVAL" Name="QEVAL" DataType="text" Length="1" SASFieldName="QEVAL">
        <Description>
          <TranslatedText xml:lang="en">Evaluator</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <CodeList OID="CL.RACE" Name="Race" DataType="text" def:StandardOID="STD.CT_SDTM">
        <EnumeratedItem CodedValue="AMERICAN INDIAN OR ALASKA NATIVE">
          <Alias Context="nci:ExtCodeID" Name="C41259"/>
        </EnumeratedItem>
        <EnumeratedItem CodedValue="ASIAN">
          <Alias Context="nci:ExtCodeID" Name="C41260"/>
        </EnumeratedItem>
        <EnumeratedItem CodedValue="BLACK OR AFRICAN AMERICAN">
          <Alias Context="nci:ExtCodeID" Name="C16352"/>
        </EnumeratedItem>
        <EnumeratedItem CodedValue="NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER">
          <Alias Context="nci:ExtCodeID" Name="C41219"/>
        </EnumeratedItem>
        <EnumeratedItem CodedValue="WHITE">
          <Alias Context="nci:ExtCodeID" Name="C41261"/>
        </EnumeratedItem>
        <Alias Context="nci:ExtCodeID" Name="C74457"/>
      </CodeList>
      <def:leaf ID="LF.acrf" xlink:href="acrf.pdf">
        <def:title>Annotated Case Report Form</def:title>
      </def:leaf>
    </MetaDataVersion>
  </Study>
</ODM>

expected/suppdm.csv

Edit5 rows

expected/suppdm.csv
STUDYID	RDOMAIN	IDVAR	IDVARVAL	QNAM	QLABEL	QVAL	QORIG	QEVAL
CATH	DM	—	—	RACE1	Race 1	WHITE	Collected	—
CATH	DM	—	—	RACE1	Race 1	BLACK OR AFRICAN AMERICAN	Collected	—
CATH	DM	—	—	RACE2	Race 2	ASIAN	Collected	—
CATH	DM	—	—	RACE2	Race 2	WHITE	Collected	—
CATH	DM	—	—	RACEOTH	Race, Other	Samoan	Collected	—

expected/suppdm.json

Edit

{
  "datasetJSONCreationDateTime": "2026-09-11T00:00:00",
  "datasetJSONVersion": "1.1.0",
  "fileOID": "CATH.sdtm.Define-XML_2.1.SUPPDM",
  "sourceSystem": {"name": "yamaa", "version": "1.0"},
  "studyOID": "STDY.CATH",
  "metaDataVersionOID": "MDV.CATH.sdtm",
  "metaDataRef": "define.xml",
  "itemGroupOID": "IG.SUPPDM",
  "records": 5,
  "name": "SUPPDM",
  "label": "Supplemental Qualifiers for DM",
  "columns": [
    {"itemOID": "IT.SUPPDM.STUDYID", "name": "STUDYID", "label": "Study Identifier", "dataType": "string", "length": 4, "keySequence": 1},
    {"itemOID": "IT.SUPPDM.RDOMAIN", "name": "RDOMAIN", "label": "Related Domain Abbreviation", "dataType": "string", "length": 2},
    {"itemOID": "IT.SUPPDM.USUBJID", "name": "USUBJID", "label": "Unique Subject Identifier", "dataType": "string", "length": 8, "keySequence": 2},
    {"itemOID": "IT.SUPPDM.IDVAR", "name": "IDVAR", "label": "Identifying Variable", "dataType": "string", "length": 1},
    {"itemOID": "IT.SUPPDM.IDVARVAL", "name": "IDVARVAL", "label": "Identifying Variable Value", "dataType": "string", "length": 1},
    {"itemOID": "IT.SUPPDM.QNAM", "name": "QNAM", "label": "Qualifier Variable Name", "dataType": "string", "length": 7, "keySequence": 3},
    {"itemOID": "IT.SUPPDM.QLABEL", "name": "QLABEL", "label": "Qualifier Variable Label", "dataType": "string", "length": 11},
    {"itemOID": "IT.SUPPDM.QVAL", "name": "QVAL", "label": "Data Value", "dataType": "string", "length": 25},
    {"itemOID": "IT.SUPPDM.QORIG", "name": "QORIG", "label": "Origin", "dataType": "string", "length": 9},
    {"itemOID": "IT.SUPPDM.QEVAL", "name": "QEVAL", "label": "Evaluator", "dataType": "string", "length": 1}
  ],
  "rows": [
    ["CATH", "DM", "CATH-001", null, null, "RACE1", "Race 1", "WHITE", "Collected", null],
    ["CATH", "DM", "CATH-003", null, null, "RACE1", "Race 1", "BLACK OR AFRICAN AMERICAN", "Collected", null],
    ["CATH", "DM", "CATH-001", null, null, "RACE2", "Race 2", "ASIAN", "Collected", null],
    ["CATH", "DM", "CATH-003", null, null, "RACE2", "Race 2", "WHITE", "Collected", null],
    ["CATH", "DM", "CATH-002", null, null, "RACEOTH", "Race, Other", "Samoan", "Collected", null]
  ]
}

Shaded columns are derived— = missing value

Human review and assessment

Variable Name	Variable Label	Type	Length	Origin	Core	Conversion Definition	Variable Type	Variable Order
STUDYID	Study Identifier	Char	20	Protocol	Req	Copy value from DM_RAW.STUDYID.	SUPP	1
RDOMAIN	Related Domain Abbreviation	Char	2	Assigned	Req	Set constant value "DM".	SUPP	2
USUBJID	Unique Subject Identifier	Char	30	Assigned	Req	Copy value from DM_RAW.USUBJID.	SUPP	3
IDVAR	Identifying Variable	Char	8	Assigned	Exp	Set constant value "None".	SUPP	4
IDVARVAL	Identifying Variable Value	Char	40	Assigned	Exp	Set constant value "None".	SUPP	5
QNAM	Qualifier Variable Name	Char	8	Assigned	Req	Set per constructed row; see the Row construction sheet.	SUPP	6
QLABEL	Qualifier Variable Label	Char	40	Assigned	Req	Set per constructed row; see the Row construction sheet.	SUPP	7
QVAL	Data Value	Char	200	Collected	Req	Set per constructed row; see the Row construction sheet.	SUPP	8
QORIG	Origin	Char	20	Assigned	Req	Set constant value "Collected".	SUPP	9
QEVAL	Evaluator	Char	40	Assigned	Exp	Set constant value "None".	SUPP	10

Template	Include when	QNAM	QLABEL	QVAL
race1	DM_RAW.RACE1 IS NOT NULL	Constant "RACE1"	Constant "Race 1"	Copy DM_RAW.RACE1
race2	DM_RAW.RACE2 IS NOT NULL	Constant "RACE2"	Constant "Race 2"	Copy DM_RAW.RACE2
raceoth	DM_RAW.RACEOTH IS NOT NULL	Constant "RACEOTH"	Constant "Race, Other"	Copy DM_RAW.RACEOTH

Codelist ID	Name	Value	Codelist NCI alias	Value NCI alias	Extensible
RACE	Race	AMERICAN INDIAN OR ALASKA NATIVE	C74457	C41259	Yes
RACE	Race	ASIAN	C74457	C41260	Yes
RACE	Race	BLACK OR AFRICAN AMERICAN	C74457	C16352	Yes
RACE	Race	NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER	C74457	C41219	Yes
RACE	Race	WHITE	C74457	C41261	Yes

Version	Date	Author	Description
1.7	2026-10-20	yamaa docs build	Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.6	2026-10-17	yamaa docs build	Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.5	2026-10-17	yamaa docs build	Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.4	2026-10-15	yamaa docs build	State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-suppdm-metadata/full.md

Edit14 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a SUPPDM dataset with one record per subject per supplemental qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

A subject has one record for each reported value of RACE1, RACE2 and RACEOTH, and none for a value that was not reported. QNAM is the source column's name, QLABEL is 'Race 1', 'Race 2' or 'Race, Other' respectively, and QVAL is the value exactly as entered. RDOMAIN is 'DM' and QORIG is 'Collected'; IDVAR, IDVARVAL and QEVAL have no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/suppdm.csv.

prompts/sdtm-suppdm-metadata/conventions.md

Edit12 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a SUPPDM dataset with one record per subject per supplemental qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

QNAM is RACE1, RACE2 or RACEOTH, and QLABEL is 'Race 1', 'Race 2' or 'Race, Other' respectively. RDOMAIN is 'DM' and QORIG is 'Collected'; IDVAR, IDVARVAL and QEVAL have no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/suppdm.csv.

prompts/sdtm-suppdm-metadata/brief.md

Edit8 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a SUPPDM dataset with one record per subject per supplemental qualifier.

The output dataset should contain the following columns in this order: STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM, QLABEL, QVAL, QORIG, QEVAL

Read the source datasets from /app/input and save the completed dataset as /app/output/suppdm.csv.

R

Edit3 lines

1library(yamaa)23yamaa_project("project.yaml")$spec("SUPPDM")$run()$output

Python

Edit3 lines

1import yamaa23yamaa.project("project.yaml").spec("SUPPDM").run().output

solutions/sdtm-suppdm-metadata/result.R

Edit34 lines

1# Reference solution for the yamaa benchmark sdtm-suppdm-metadata (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910dm_raw <- read_csv(11  "/app/input/dm_raw.csv",12  col_types = cols(.default = col_character())13)1415labels <- c(RACE1 = "Race 1", RACE2 = "Race 2", RACEOTH = "Race, Other")16suppdm <- bind_rows(lapply(names(labels), function(qnam) {17  dm_raw |>18    filter(!is.na(.data[[qnam]])) |>19    transmute(20      STUDYID,21      RDOMAIN = "DM",22      USUBJID,23      IDVAR = NA_character_,24      IDVARVAL = NA_character_,25      QNAM = qnam,26      QLABEL = labels[[qnam]],27      QVAL = .data[[qnam]],28      QORIG = "Collected",29      QEVAL = NA_character_30    )31}))3233dir.create("/app/output", showWarnings = FALSE)34write_csv(suppdm, "/app/output/suppdm.csv", na = "")

solutions/sdtm-suppdm-metadata/result.py

Edit32 lines

1# Reference solution for the yamaa benchmark sdtm-suppdm-metadata (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011dm_raw = pl.read_csv("/app/input/dm_raw.csv", infer_schema=False)1213labels = {"RACE1": "Race 1", "RACE2": "Race 2", "RACEOTH": "Race, Other"}14missing = pl.lit(None, dtype=pl.String)15suppdm = pl.concat(16    dm_raw.filter(pl.col(qnam).is_not_null()).select(17        pl.col("STUDYID"),18        pl.lit("DM").alias("RDOMAIN"),19        pl.col("USUBJID"),20        missing.alias("IDVAR"),21        missing.alias("IDVARVAL"),22        pl.lit(qnam).alias("QNAM"),23        pl.lit(label).alias("QLABEL"),24        pl.col(qnam).alias("QVAL"),25        pl.lit("Collected").alias("QORIG"),26        missing.alias("QEVAL"),27    )28    for qnam, label in labels.items()29)3031Path("/app/output").mkdir(exist_ok=True)32suppdm.write_csv("/app/output/suppdm.csv")

solutions/sdtm-suppdm-metadata/result.sas

Edit32 lines

1/* opensas reference solution for the yamaa benchmark sdtm-suppdm-metadata.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data dm_raw;6  length STUDYID USUBJID RACE1 RACE2 RACEOTH $1024;7  infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. RACE1 : $char1024.9    RACE2 : $char1024. RACEOTH : $char1024.;10run;1112data result;13  set dm_raw;14  length RDOMAIN $2 IDVAR $8 IDVARVAL $40 QNAM $8 QLABEL $40 QVAL $200 QORIG $2015    QEVAL $40;16  RDOMAIN = 'DM';17  QORIG = 'Collected';18  if RACE1 ne '' then do;19    QNAM = 'RACE1'; QLABEL = 'Race 1'; QVAL = RACE1; output;20  end;21  if RACE2 ne '' then do;22    QNAM = 'RACE2'; QLABEL = 'Race 2'; QVAL = RACE2; output;23  end;24  if RACEOTH ne '' then do;25    QNAM = 'RACEOTH'; QLABEL = 'Race, Other'; QVAL = RACEOTH; output;26  end;27run;28proc sql;29  create table final as select STUDYID, RDOMAIN, USUBJID, IDVAR, IDVARVAL, QNAM,30    QLABEL, QVAL, QORIG, QEVAL from result;31quit;32proc export data=final outfile="/app/output/suppdm.csv" dbms=csv replace; run;

input/dm_raw.csv

expected/define.xml

Define-XML 2.1.11: CATH

SUPPDM

expected/suppdm.csv

expected/suppdm.json

Your review

Mapping spec

Agent prompt

prompts/sdtm-suppdm-metadata/full.md

prompts/sdtm-suppdm-metadata/conventions.md

prompts/sdtm-suppdm-metadata/brief.md

yamaa code

R

Python

Solution without yamaa

solutions/sdtm-suppdm-metadata/result.R

solutions/sdtm-suppdm-metadata/result.py

solutions/sdtm-suppdm-metadata/result.sas