Metadata Contract

Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: build one DM record per subject carrying SITEID, AGE, AGEU, SEX, and COUNTRY.

Input: one row per subject carrying SITEID, AGE, SEX, and COUNTRY.

Variables:

USUBJID joins the study, site, and subject identifiers with hyphens; an identifier longer than 30 characters stops the run rather than being shortened.
SITEID is the collected site identifier.
AGE is the collected age in whole years, between 0 and 120; missing when not collected.
AGEU is fixed to YEARS, since age is collected in whole years.
SEX is the collected sex, one of F, M, or U.
COUNTRY is the collected three-letter country code.

Note: the dataset and its variables carry the description, labels, provenance, lengths, terminology, class, structure, and standard version a review needs. The same declarations produce the data-definition document beside the data and the exchange file dm.json, which carries the rows and the column metadata together and is the file the document points at.

YAML specification

1 spec file

spec.yaml

Edit163 lines

1yamaa: "1.0"2type: spec3model: SDTM4domain: DM5keys: [STUDYID, USUBJID]6input:7  DM_RAW: input/dm_raw.csv89output:10  path: dm.csv11  columns: [DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY]1213columns:14  - name: DOMAIN15    type: str16    label: Domain Abbreviation17    derivation: "'DM'"18    submission:19      core: Req20      role: Identifier21      length: 222      codelist: DOMAIN23      origins:24        - type: Assigned25          source: Sponsor2627  - name: STUDYID28    type: str29    label: Study Identifier30    derivation: DM_RAW.STUDYID31    submission:32      core: Req33      role: Identifier34      length: 2035      origins:36        - type: Protocol37          source: Sponsor3839  - name: USUBJID40    type: str41    label: Unique Subject Identifier42    derivation:43      str_template: "{DM_RAW.STUDYID}-{DM_RAW.SITEID}-{DM_RAW.SUBJID}"44    verifications:45      - max_length:46          max: 3047    submission:48      core: Req49      role: Identifier50      origins:51        - type: Derived52          source: Sponsor53      method: Study identifier, site identifier, and subject identifier joined54        by hyphens.5556  - name: SUBJID57    type: str58    label: Subject Identifier for the Study59    derivation: DM_RAW.SUBJID60    submission:61      core: Req62      role: Topic63      length: 1064      origins:65        - type: Collected66          source: Investigator67          documents:68            - document: acrf69              pages: {type: PhysicalRef, refs: "1"}7071  - name: SITEID72    type: str73    label: Study Site Identifier74    derivation: DM_RAW.SITEID75    submission:76      core: Req77      role: Record Qualifier78      length: 1079      origins:80        - type: Collected81          source: Investigator82          documents:83            - document: acrf84              pages: {type: PhysicalRef, refs: "1"}8586  - name: AGE87    type: int88    label: Age89    derivation: DM_RAW.AGE90    verifications:91      - range:92          min: 093          max: 12094    submission:95      core: Exp96      role: Record Qualifier97      length: 398      origins:99        - type: Collected100          source: Investigator101          documents:102            - document: acrf103              pages: {type: PhysicalRef, refs: "1"}104105  - name: AGEU106    type: str107    label: Age Units108    derivation: "'YEARS'"109    submission:110      core: Exp111      role: Variable Qualifier112      length: 5113      codelist: AGEU114      origins:115        - type: Assigned116          source: Sponsor117      comment: Defaulted to YEARS because age is collected in whole years.118119  - name: SEX120    type: str121    label: Sex122    derivation: DM_RAW.SEX123    submission:124      core: Req125      role: Record Qualifier126      length: 1127      codelist: SEX128      origins:129        - type: Collected130          source: Investigator131          documents:132            - document: acrf133              pages: {type: PhysicalRef, refs: "1"}134135  - name: COUNTRY136    type: str137    label: Country138    derivation: DM_RAW.COUNTRY139    submission:140      core: Req141      role: Record Qualifier142      length: 3143      codelist: COUNTRY144      origins:145        - type: Collected146          source: Investigator147          documents:148            - document: acrf149              pages: {type: PhysicalRef, refs: "1"}150151verifications:152  - row_count:153      min: 3154      max: 3155156submission:157  label: Demographics158  class: SPECIAL PURPOSE159  structure: One record per subject160  repeating: false161  comment:162    text: Demographics are described in section 2.1 of the reviewer's guide.163    documents: [csdrg]

Inputs

Under review1 source file

input/dm_raw.csv

Edit3 rows

input/dm_raw.csv
STUDYID	SUBJID	SITEID	AGE	SEX	COUNTRY
CATH	0001	UCSD	34	F	USA
CATH	0002	UCSD	67	M	USA
CATH	0003	MCGILL	52	F	CAN

Expected output

Under review3 expected rows

expected/define.xml

EditDefine-XML

Define-XML 2.1.11: STUDY01

Example study demonstrating the Define-XML metadata contract; STUDY-01; SDTMIG 3.4; CDISC/NCI 2023-12-15

Datasets
Dataset	Description	Class	Structure	Purpose	Keys	Documentation	Location
DM	Demographics	SPECIAL PURPOSE	One record per subject	Tabulation	STUDYID, USUBJID	Demographics are described in section 2.1 of the reviewer's guide. Study Data Reviewer's Guide (csdrg.pdf)	dm.json

DM

DM variables
Variable	Label / Description	Type	Role	Length or Display Format	Controlled Terms or ISO Format	Origin / Source / Method / Comment
DOMAIN	Domain Abbreviation	text	Identifier	2	Domain Abbreviation (DM) (CL.DOMAIN) DM = Demographics	Assigned; Sponsor
STUDYID	Study Identifier	text	Identifier	4		Protocol; Sponsor
USUBJID	Unique Subject Identifier	text	Identifier	16		Derived; Sponsor Study identifier, site identifier, and subject identifier joined by hyphens.
SUBJID	Subject Identifier for the Study	text	Topic	4		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
SITEID	Study Site Identifier	text	Record Qualifier	6		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
AGE	Age	integer	Record Qualifier	3		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
AGEU	Age Units	text	Variable Qualifier	5	Age Unit (CL.AGEU) YEARS	Assigned; Sponsor Defaulted to YEARS because age is collected in whole years.
SEX	Sex	text	Record Qualifier	1	Sex (CL.SEX) F = Female M = Male U = Unknown	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
COUNTRY	Country	text	Record Qualifier	3	Country Codes (CL.COUNTRY) ISO 3166 2020	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1

Original source

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="www.example.org/STUDY01/1/Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" Originator="Example Sponsor" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.STUDY01">
    <GlobalVariables>
      <StudyName>STUDY01</StudyName>
      <StudyDescription>Example study demonstrating the Define-XML metadata contract</StudyDescription>
      <ProtocolName>STUDY-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.STUDY01.1" Name="Study STUDY01, Data Definitions" Description="Tabulation data definitions for STUDY01" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.SDTMIG" Name="SDTMIG" Type="IG" Version="3.4" Status="Final"/>
        <def:Standard OID="STD.CT_SDTM" Name="CDISC/NCI" Type="CT" PublishingSet="SDTM" Version="2023-12-15" Status="Final"/>
      </def:Standards>
      <def:AnnotatedCRF>
        <def:DocumentRef leafID="LF.acrf"/>
      </def:AnnotatedCRF>
      <def:SupplementalDoc>
        <def:DocumentRef leafID="LF.csdrg"/>
      </def:SupplementalDoc>
      <ItemGroupDef OID="IG.DM" Name="DM" Domain="DM" Purpose="Tabulation" SASDatasetName="DM" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.DM" def:StandardOID="STD.SDTMIG" def:CommentOID="COM.DM">
        <Description>
          <TranslatedText xml:lang="en">Demographics</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.DM.DOMAIN" OrderNumber="1" Mandatory="Yes" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.STUDYID" OrderNumber="2" Mandatory="Yes" KeySequence="1" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.USUBJID" OrderNumber="3" Mandatory="Yes" KeySequence="2" Role="Identifier" MethodOID="MT.DM.USUBJID"/>
        <ItemRef ItemOID="IT.DM.SUBJID" OrderNumber="4" Mandatory="Yes" Role="Topic"/>
        <ItemRef ItemOID="IT.DM.SITEID" OrderNumber="5" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGE" OrderNumber="6" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGEU" OrderNumber="7" Mandatory="No" Role="Variable Qualifier"/>
        <ItemRef ItemOID="IT.DM.SEX" OrderNumber="8" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.COUNTRY" OrderNumber="9" Mandatory="Yes" Role="Record Qualifier"/>
        <def:Class Name="SPECIAL PURPOSE"/>
        <def:leaf ID="LF.DM" xlink:href="dm.json">
          <def:title>dm.json</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.DM.DOMAIN" Name="DOMAIN" DataType="text" Length="2" SASFieldName="DOMAIN">
        <Description>
          <TranslatedText xml:lang="en">Domain Abbreviation</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.DOMAIN"/>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Protocol" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.USUBJID" Name="USUBJID" DataType="text" Length="16" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SUBJID" Name="SUBJID" DataType="text" Length="4" SASFieldName="SUBJID">
        <Description>
          <TranslatedText xml:lang="en">Subject Identifier for the Study</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.SITEID" Name="SITEID" DataType="text" Length="6" SASFieldName="SITEID">
        <Description>
          <TranslatedText xml:lang="en">Study Site Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGE" Name="AGE" DataType="integer" Length="3" SASFieldName="AGE">
        <Description>
          <TranslatedText xml:lang="en">Age</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGEU" Name="AGEU" DataType="text" Length="5" SASFieldName="AGEU" def:CommentOID="COM.DM.AGEU">
        <Description>
          <TranslatedText xml:lang="en">Age Units</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.AGEU"/>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SEX" Name="SEX" DataType="text" Length="1" SASFieldName="SEX">
        <Description>
          <TranslatedText xml:lang="en">Sex</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.SEX"/>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.COUNTRY" Name="COUNTRY" DataType="text" Length="3" SASFieldName="COUNTRY">
        <Description>
          <TranslatedText xml:lang="en">Country</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.COUNTRY"/>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <CodeList OID="CL.DOMAIN" Name="Domain Abbreviation (DM)" DataType="text" def:StandardOID="STD.CT_SDTM" SASFormatName="$DMDOMAI">
        <CodeListItem CodedValue="DM">
          <Decode>
            <TranslatedText xml:lang="en">Demographics</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C49572"/>
        </CodeListItem>
        <Alias Context="nci:ExtCodeID" Name="C66734"/>
      </CodeList>
      <CodeList OID="CL.AGEU" Name="Age Unit" DataType="text" def:StandardOID="STD.CT_SDTM">
        <EnumeratedItem CodedValue="YEARS">
          <Alias Context="nci:ExtCodeID" Name="C29848"/>
        </EnumeratedItem>
        <Alias Context="nci:ExtCodeID" Name="C66781"/>
      </CodeList>
      <CodeList OID="CL.SEX" Name="Sex" DataType="text" def:StandardOID="STD.CT_SDTM" SASFormatName="$SEX">
        <CodeListItem CodedValue="F">
          <Decode>
            <TranslatedText xml:lang="en">Female</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C16576"/>
        </CodeListItem>
        <CodeListItem CodedValue="M">
          <Decode>
            <TranslatedText xml:lang="en">Male</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C20197"/>
        </CodeListItem>
        <CodeListItem CodedValue="U">
          <Decode>
            <TranslatedText xml:lang="en">Unknown</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C17998"/>
        </CodeListItem>
        <Alias Context="nci:ExtCodeID" Name="C66731"/>
      </CodeList>
      <CodeList OID="CL.COUNTRY" Name="Country Codes" DataType="text" def:StandardOID="STD.CT_SDTM">
        <ExternalCodeList Dictionary="ISO 3166" Version="2020" href="https://www.iso.org/iso-3166-country-codes.html"/>
      </CodeList>
      <MethodDef OID="MT.DM.USUBJID" Name="DM.USUBJID" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Study identifier, site identifier, and subject identifier joined by hyphens.</TranslatedText>
        </Description>
      </MethodDef>
      <def:CommentDef OID="COM.DM">
        <Description>
          <TranslatedText xml:lang="en">Demographics are described in section 2.1 of the reviewer's guide.</TranslatedText>
        </Description>
        <def:DocumentRef leafID="LF.csdrg"/>
      </def:CommentDef>
      <def:CommentDef OID="COM.DM.AGEU">
        <Description>
          <TranslatedText xml:lang="en">Defaulted to YEARS because age is collected in whole years.</TranslatedText>
        </Description>
      </def:CommentDef>
      <def:leaf ID="LF.acrf" xlink:href="acrf.pdf">
        <def:title>Annotated Case Report Form</def:title>
      </def:leaf>
      <def:leaf ID="LF.csdrg" xlink:href="csdrg.pdf">
        <def:title>Study Data Reviewer's Guide</def:title>
      </def:leaf>
    </MetaDataVersion>
  </Study>
</ODM>

expected/dm.csv

Edit3 rows

expected/dm.csv
DOMAIN	STUDYID	SUBJID	SITEID	AGE	AGEU	SEX	COUNTRY
DM	CATH	0001	UCSD	34	YEARS	F	USA
DM	CATH	0002	UCSD	67	YEARS	M	USA
DM	CATH	0003	MCGILL	52	YEARS	F	CAN

expected/dm.json

Edit

{
  "datasetJSONCreationDateTime": "2026-09-11T00:00:00",
  "datasetJSONVersion": "1.1.0",
  "fileOID": "www.example.org/STUDY01/1/Define-XML_2.1.DM",
  "originator": "Example Sponsor",
  "sourceSystem": {"name": "yamaa", "version": "1.0"},
  "studyOID": "STDY.STUDY01",
  "metaDataVersionOID": "MDV.STUDY01.1",
  "metaDataRef": "define.xml",
  "itemGroupOID": "IG.DM",
  "records": 3,
  "name": "DM",
  "label": "Demographics",
  "columns": [
    {"itemOID": "IT.DM.DOMAIN", "name": "DOMAIN", "label": "Domain Abbreviation", "dataType": "string", "length": 2},
    {"itemOID": "IT.DM.STUDYID", "name": "STUDYID", "label": "Study Identifier", "dataType": "string", "length": 4, "keySequence": 1},
    {"itemOID": "IT.DM.USUBJID", "name": "USUBJID", "label": "Unique Subject Identifier", "dataType": "string", "length": 16, "keySequence": 2},
    {"itemOID": "IT.DM.SUBJID", "name": "SUBJID", "label": "Subject Identifier for the Study", "dataType": "string", "length": 4},
    {"itemOID": "IT.DM.SITEID", "name": "SITEID", "label": "Study Site Identifier", "dataType": "string", "length": 6},
    {"itemOID": "IT.DM.AGE", "name": "AGE", "label": "Age", "dataType": "integer", "length": 3},
    {"itemOID": "IT.DM.AGEU", "name": "AGEU", "label": "Age Units", "dataType": "string", "length": 5},
    {"itemOID": "IT.DM.SEX", "name": "SEX", "label": "Sex", "dataType": "string", "length": 1},
    {"itemOID": "IT.DM.COUNTRY", "name": "COUNTRY", "label": "Country", "dataType": "string", "length": 3}
  ],
  "rows": [
    ["DM", "CATH", "CATH-UCSD-0001", "0001", "UCSD", 34, "YEARS", "F", "USA"],
    ["DM", "CATH", "CATH-UCSD-0002", "0002", "UCSD", 67, "YEARS", "M", "USA"],
    ["DM", "CATH", "CATH-MCGILL-0003", "0003", "MCGILL", 52, "YEARS", "F", "CAN"]
  ]
}

Shaded columns are derived— = missing value

Human review and assessment

Variable Name	Variable Label	Type	Length	Controlled Terms or Format	Origin	Core	Conversion Definition	Variable Type	Variable Order	Comments for Define
DOMAIN	Domain Abbreviation	Char	2	Domain Abbreviation (DM)	Assigned	Req	Set constant value "DM".	SDTM	1
STUDYID	Study Identifier	Char	20		Protocol	Req	Copy value from DM_RAW.STUDYID.	SDTM	2
USUBJID	Unique Subject Identifier	Char			Derived	Req	Study identifier, site identifier, and subject identifier joined by hyphens.	SDTM	3
SUBJID	Subject Identifier for the Study	Char	10		Collected	Req	Copy value from DM_RAW.SUBJID.	SDTM	4
SITEID	Study Site Identifier	Char	10		Collected	Req	Copy value from DM_RAW.SITEID.	SDTM	5
AGE	Age	Num	3		Collected	Exp	Copy value from DM_RAW.AGE.	SDTM	6
AGEU	Age Units	Char	5	Age Unit	Assigned	Exp	Set constant value "YEARS".	SDTM	7	Defaulted to YEARS because age is collected in whole years.
SEX	Sex	Char	1	Sex	Collected	Req	Copy value from DM_RAW.SEX.	SDTM	8
COUNTRY	Country	Char	3	Country Codes (ISO 3166, version 2020)	Collected	Req	Copy value from DM_RAW.COUNTRY.	SDTM	9

Codelist ID	Name	Value	Decode	Codelist NCI alias	Value NCI alias	Extensible	SAS format	Dictionary	Version
DOMAIN	Domain Abbreviation (DM)	DM	Demographics	C66734	C49572	No	$DMDOMAI
AGEU	Age Unit	YEARS		C66781	C29848	No
SEX	Sex	F	Female	C66731	C16576	No	$SEX
SEX	Sex	M	Male	C66731	C20197	No	$SEX
SEX	Sex	U	Unknown	C66731	C17998	No	$SEX
COUNTRY	Country Codes					No		ISO 3166	2020

Version	Date	Author	Description
1.7	2026-10-20	yamaa docs build	Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.6	2026-10-17	yamaa docs build	Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.5	2026-10-17	yamaa docs build	Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.4	2026-10-15	yamaa docs build	State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-dm-metadata/full.md

Edit16 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

DOMAIN is "DM". USUBJID joins the study identifier, the site identifier, and the subject identifier with hyphens. SITEID is the collected site identifier. AGE is the collected age in whole years, between 0 and 120, and has no value when no age was collected. AGEU is "YEARS", since age is collected in whole years. SEX is the collected sex, one of "F", "M", or "U". COUNTRY is the collected three-letter country code.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-metadata/conventions.md

Edit10 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

DOMAIN is "DM". SEX is one of "F", "M", or "U".

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-metadata/brief.md

Edit8 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

R

Edit3 lines

1library(yamaa)23yamaa_project("project.yaml")$spec("DM")$run()$output

Python

Edit3 lines

1import yamaa23yamaa.project("project.yaml").spec("DM").run().output

solutions/sdtm-dm-metadata/result.R

Edit25 lines

1# Reference solution for the yamaa benchmark sdtm-dm-metadata (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910dm_raw <- read_csv(11  "/app/input/dm_raw.csv",12  col_types = cols(.default = col_character())13)1415dm <- dm_raw |>16  mutate(17    DOMAIN = "DM",18    USUBJID = paste(STUDYID, SITEID, SUBJID, sep = "-"),19    AGE = suppressWarnings(as.integer(AGE)),20    AGEU = "YEARS"21  ) |>22  select(DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY)2324dir.create("/app/output", showWarnings = FALSE)25write_csv(dm, "/app/output/dm.csv", na = "")

solutions/sdtm-dm-metadata/result.py

Edit73 lines

1# Reference solution for the yamaa benchmark sdtm-dm-metadata (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67"""Build the SDTM DM dataset from the collected DM_RAW dataset.89One record per subject. Reads /app/input/dm_raw.csv and writes10/app/output/dm.csv with columns DOMAIN, STUDYID, USUBJID, SUBJID,11SITEID, AGE, AGEU, SEX, COUNTRY.12"""1314import csv15import os1617INPUT_PATH = "/app/input/dm_raw.csv"18OUTPUT_PATH = "/app/output/dm.csv"1920COLUMNS = [21    "DOMAIN",22    "STUDYID",23    "USUBJID",24    "SUBJID",25    "SITEID",26    "AGE",27    "AGEU",28    "SEX",29    "COUNTRY",30]313233def build_record(row):34    studyid = (row.get("STUDYID") or "").strip()35    siteid = (row.get("SITEID") or "").strip()36    subjid = (row.get("SUBJID") or "").strip()37    age = (row.get("AGE") or "").strip()38    sex = (row.get("SEX") or "").strip()39    country = (row.get("COUNTRY") or "").strip()4041    # AGE is the collected age in whole years; blank when none collected.42    age_value = ""43    if age:44        age_value = str(int(float(age)))4546    return {47        "DOMAIN": "DM",48        "STUDYID": studyid,49        "USUBJID": "-".join([studyid, siteid, subjid]),50        "SUBJID": subjid,51        "SITEID": siteid,52        "AGE": age_value,53        "AGEU": "YEARS",54        "SEX": sex,55        "COUNTRY": country,56    }575859def main():60    with open(INPUT_PATH, newline="", encoding="utf-8") as f:61        rows = list(csv.DictReader(f))6263    records = [build_record(row) for row in rows]6465    os.makedirs(os.path.dirname(OUTPUT_PATH), exist_ok=True)66    with open(OUTPUT_PATH, "w", newline="", encoding="utf-8") as f:67        writer = csv.DictWriter(f, fieldnames=COLUMNS)68        writer.writeheader()69        writer.writerows(records)707172if __name__ == "__main__":73    main()

solutions/sdtm-dm-metadata/result.sas

Edit15 lines

1/* opensas reference solution for the yamaa benchmark sdtm-dm-metadata.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data dm_raw;6  length STUDYID SUBJID SITEID SEX COUNTRY $1024;7  infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. SUBJID : $char1024. SITEID : $char1024. AGE SEX : $char1024. COUNTRY : $char1024.;9run;1011data result; set dm_raw; length DOMAIN $2 USUBJID $1024 AGEU $8; DOMAIN='DM'; USUBJID=catx('-',STUDYID,SITEID,SUBJID); AGEU='YEARS'; run;12proc sql;13  create table final as select DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY from result;14quit;15proc export data=final outfile="/app/output/dm.csv" dbms=csv replace; run;

input/dm_raw.csv

expected/define.xml

Define-XML 2.1.11: STUDY01

DM

expected/dm.csv

expected/dm.json

Your review

Mapping spec

Agent prompt

prompts/sdtm-dm-metadata/full.md

prompts/sdtm-dm-metadata/conventions.md

prompts/sdtm-dm-metadata/brief.md

yamaa code

R

Python

Solution without yamaa

solutions/sdtm-dm-metadata/result.R

solutions/sdtm-dm-metadata/result.py

solutions/sdtm-dm-metadata/result.sas