Demographics Submitted in SAS Transport

Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: build one DM record per subject carrying RFSTDTC, SITEID, AGE, AGEU, SEX, RACE and COUNTRY, and submit it as the SAS transport file dm.xpt beside its data-definition document.

Input: one row per subject carrying the site, reference start date, age, sex, race and country.

Variables:

RFSTDTC is the subject's reference start date; empty when the subject has none.
SITEID is the collected site identifier.
AGE is the collected age in whole years.
AGEU is fixed to YEARS.
SEX, RACE and COUNTRY are as collected.

Note: in the transport file each text variable is exactly as wide as its longest value, and the data-definition document reports that width as the variable's length instead of the largest length the study allows. A date travels as YYYY-MM-DD text, and an empty value is blanks.

YAML specification

1 spec file

spec.yaml

Edit171 lines

1yamaa: "1.0"2type: spec3model: SDTM4domain: DM5keys: [STUDYID, USUBJID]6input:7  DM_RAW: input/dm_raw.csv89output:10  path: dm.csv11  columns: [DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU,12    SEX, RACE, COUNTRY]1314columns:15  - name: DOMAIN16    type: str17    label: Domain Abbreviation18    derivation: "'DM'"19    submission:20      core: Req21      role: Identifier22      length: 223      origins:24        - type: Assigned25          source: Sponsor2627  - name: STUDYID28    type: str29    label: Study Identifier30    derivation: DM_RAW.STUDYID31    submission:32      core: Req33      role: Identifier34      length: 2035      origins:36        - type: Protocol37          source: Sponsor3839  - name: USUBJID40    type: str41    label: Unique Subject Identifier42    derivation: DM_RAW.USUBJID43    submission:44      core: Req45      role: Identifier46      length: 3047      origins:48        - type: Assigned49          source: Sponsor5051  - name: SUBJID52    type: str53    label: Subject Identifier for the Study54    derivation: DM_RAW.SUBJID55    submission:56      core: Req57      role: Topic58      length: 1059      origins:60        - type: Collected61          source: Investigator62          documents:63            - document: acrf64              pages: {type: PhysicalRef, refs: "1"}6566  - name: RFSTDTC67    type: date68    label: Subject Reference Start Date/Time69    derivation: DM_RAW.RFSTDTC70    submission:71      core: Exp72      role: Record Qualifier73      origins:74        - type: Collected75          source: Investigator76          documents:77            - document: acrf78              pages: {type: PhysicalRef, refs: "1"}7980  - name: SITEID81    type: str82    label: Study Site Identifier83    derivation: DM_RAW.SITEID84    submission:85      core: Req86      role: Record Qualifier87      length: 1088      origins:89        - type: Collected90          source: Investigator91          documents:92            - document: acrf93              pages: {type: PhysicalRef, refs: "1"}9495  - name: AGE96    type: int97    label: Age98    derivation: DM_RAW.AGE99    submission:100      core: Exp101      role: Record Qualifier102      length: 3103      origins:104        - type: Collected105          source: Investigator106          documents:107            - document: acrf108              pages: {type: PhysicalRef, refs: "1"}109110  - name: AGEU111    type: str112    label: Age Units113    derivation: "'YEARS'"114    submission:115      core: Exp116      role: Variable Qualifier117      length: 6118      origins:119        - type: Assigned120          source: Sponsor121122  - name: SEX123    type: str124    label: Sex125    derivation: DM_RAW.SEX126    submission:127      core: Req128      role: Record Qualifier129      length: 1130      origins:131        - type: Collected132          source: Investigator133          documents:134            - document: acrf135              pages: {type: PhysicalRef, refs: "1"}136137  - name: RACE138    type: str139    label: Race140    derivation: DM_RAW.RACE141    submission:142      core: Exp143      role: Record Qualifier144      length: 40145      origins:146        - type: Collected147          source: Investigator148          documents:149            - document: acrf150              pages: {type: PhysicalRef, refs: "1"}151152  - name: COUNTRY153    type: str154    label: Country155    derivation: DM_RAW.COUNTRY156    submission:157      core: Req158      role: Record Qualifier159      length: 3160      origins:161        - type: Collected162          source: Investigator163          documents:164            - document: acrf165              pages: {type: PhysicalRef, refs: "1"}166167submission:168  label: Demographics169  class: SPECIAL PURPOSE170  structure: One record per subject171  repeating: false

Inputs

Under review1 source file

input/dm_raw.csv

Edit3 rows

input/dm_raw.csv
STUDYID	SUBJID	SITEID	RFSTDTC	AGE	SEX	RACE	COUNTRY
CATH	0001	UCSD	2025-01-08	34	F	WHITE	USA
CATH	0002	UCSD	2025-01-15	67	M	BLACK OR AFRICAN AMERICAN	USA
CATH	0003	MCGILL	—	52	F	ASIAN	CAN

Expected output

Under review3 expected rows

expected/define.xml

EditDefine-XML

Define-XML 2.1.11: CATH

Example study submitting demographics in SAS transport; CATH-01; SDTMIG 3.4

Datasets
Dataset	Description	Class	Structure	Purpose	Keys	Documentation	Location
DM	Demographics	SPECIAL PURPOSE	One record per subject	Tabulation	STUDYID, USUBJID		dm.xpt

DM

DM variables
Variable	Label / Description	Type	Role	Length or Display Format	Controlled Terms or ISO Format	Origin / Source / Method / Comment
DOMAIN	Domain Abbreviation	text	Identifier	2		Assigned; Sponsor
STUDYID	Study Identifier	text	Identifier	4		Protocol; Sponsor
USUBJID	Unique Subject Identifier	text	Identifier	16		Assigned; Sponsor
SUBJID	Subject Identifier for the Study	text	Topic	4		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
RFSTDTC	Subject Reference Start Date/Time	date	Record Qualifier		ISO 8601	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
SITEID	Study Site Identifier	text	Record Qualifier	6		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
AGE	Age	integer	Record Qualifier	3		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
AGEU	Age Units	text	Variable Qualifier	5		Assigned; Sponsor
SEX	Sex	text	Record Qualifier	1		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
RACE	Race	text	Record Qualifier	25		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
COUNTRY	Country	text	Record Qualifier	3		Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1

Original source

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.sdtm.Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study submitting demographics in SAS transport</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.sdtm" Name="CATH sdtm metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.SDTMIG" Name="SDTMIG" Type="IG" Version="3.4" Status="Final"/>
      </def:Standards>
      <def:AnnotatedCRF>
        <def:DocumentRef leafID="LF.acrf"/>
      </def:AnnotatedCRF>
      <ItemGroupDef OID="IG.DM" Name="DM" Domain="DM" Purpose="Tabulation" SASDatasetName="DM" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.DM" def:StandardOID="STD.SDTMIG">
        <Description>
          <TranslatedText xml:lang="en">Demographics</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.DM.DOMAIN" OrderNumber="1" Mandatory="Yes" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.STUDYID" OrderNumber="2" Mandatory="Yes" KeySequence="1" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.USUBJID" OrderNumber="3" Mandatory="Yes" KeySequence="2" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.SUBJID" OrderNumber="4" Mandatory="Yes" Role="Topic"/>
        <ItemRef ItemOID="IT.DM.RFSTDTC" OrderNumber="5" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.SITEID" OrderNumber="6" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGE" OrderNumber="7" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGEU" OrderNumber="8" Mandatory="No" Role="Variable Qualifier"/>
        <ItemRef ItemOID="IT.DM.SEX" OrderNumber="9" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.RACE" OrderNumber="10" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.COUNTRY" OrderNumber="11" Mandatory="Yes" Role="Record Qualifier"/>
        <def:Class Name="SPECIAL PURPOSE"/>
        <def:leaf ID="LF.DM" xlink:href="dm.xpt">
          <def:title>dm.xpt</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.DM.DOMAIN" Name="DOMAIN" DataType="text" Length="2" SASFieldName="DOMAIN">
        <Description>
          <TranslatedText xml:lang="en">Domain Abbreviation</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Protocol" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.USUBJID" Name="USUBJID" DataType="text" Length="16" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SUBJID" Name="SUBJID" DataType="text" Length="4" SASFieldName="SUBJID">
        <Description>
          <TranslatedText xml:lang="en">Subject Identifier for the Study</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.RFSTDTC" Name="RFSTDTC" DataType="date" SASFieldName="RFSTDTC">
        <Description>
          <TranslatedText xml:lang="en">Subject Reference Start Date/Time</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.SITEID" Name="SITEID" DataType="text" Length="6" SASFieldName="SITEID">
        <Description>
          <TranslatedText xml:lang="en">Study Site Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGE" Name="AGE" DataType="integer" Length="3" SASFieldName="AGE">
        <Description>
          <TranslatedText xml:lang="en">Age</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGEU" Name="AGEU" DataType="text" Length="5" SASFieldName="AGEU">
        <Description>
          <TranslatedText xml:lang="en">Age Units</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SEX" Name="SEX" DataType="text" Length="1" SASFieldName="SEX">
        <Description>
          <TranslatedText xml:lang="en">Sex</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.RACE" Name="RACE" DataType="text" Length="25" SASFieldName="RACE">
        <Description>
          <TranslatedText xml:lang="en">Race</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.COUNTRY" Name="COUNTRY" DataType="text" Length="3" SASFieldName="COUNTRY">
        <Description>
          <TranslatedText xml:lang="en">Country</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <def:leaf ID="LF.acrf" xlink:href="acrf.pdf">
        <def:title>Annotated Case Report Form</def:title>
      </def:leaf>
    </MetaDataVersion>
  </Study>
</ODM>

expected/dm.csv

Edit3 rows

expected/dm.csv
DOMAIN	STUDYID	SUBJID	RFSTDTC	SITEID	AGE	AGEU	SEX	RACE	COUNTRY
DM	CATH	0001	2025-01-08	UCSD	34	YEARS	F	WHITE	USA
DM	CATH	0002	2025-01-15	UCSD	67	YEARS	M	BLACK OR AFRICAN AMERICAN	USA
DM	CATH	0003	—	MCGILL	52	YEARS	F	ASIAN	CAN

Shaded columns are derived— = missing value

Human review and assessment

Variable Name	Variable Label	Type	Length	Controlled Terms or Format	Origin	Core	Conversion Definition	Variable Type	Variable Order
DOMAIN	Domain Abbreviation	Char	2		Assigned	Req	Set constant value "DM".	SDTM	1
STUDYID	Study Identifier	Char	20		Protocol	Req	Copy value from DM_RAW.STUDYID.	SDTM	2
USUBJID	Unique Subject Identifier	Char	30		Assigned	Req	Copy value from DM_RAW.USUBJID.	SDTM	3
SUBJID	Subject Identifier for the Study	Char	10		Collected	Req	Copy value from DM_RAW.SUBJID.	SDTM	4
RFSTDTC	Subject Reference Start Date/Time	Char		ISO 8601	Collected	Exp	Copy value from DM_RAW.RFSTDTC.	SDTM	5
SITEID	Study Site Identifier	Char	10		Collected	Req	Copy value from DM_RAW.SITEID.	SDTM	6
AGE	Age	Num	3		Collected	Exp	Copy value from DM_RAW.AGE.	SDTM	7
AGEU	Age Units	Char	6		Assigned	Exp	Set constant value "YEARS".	SDTM	8
SEX	Sex	Char	1		Collected	Req	Copy value from DM_RAW.SEX.	SDTM	9
RACE	Race	Char	40		Collected	Exp	Copy value from DM_RAW.RACE.	SDTM	10
COUNTRY	Country	Char	3		Collected	Req	Copy value from DM_RAW.COUNTRY.	SDTM	11

Version	Date	Author	Description
1.7	2026-10-20	yamaa docs build	Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.6	2026-10-17	yamaa docs build	Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.5	2026-10-17	yamaa docs build	Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.4	2026-10-15	yamaa docs build	State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-dm-xpt/full.md

Edit14 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU, SEX, RACE, COUNTRY

USUBJID, SUBJID, SITEID, SEX, RACE and COUNTRY are as collected, SUBJID keeping its leading zeros. RFSTDTC is the subject's reference start date, with no value when the subject has none. AGE is the collected age in whole years. AGEU is fixed to YEARS.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-xpt/conventions.md

Edit11 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU, SEX, RACE, COUNTRY

AGEU is fixed to YEARS.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-xpt/brief.md

Edit9 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU, SEX, RACE, COUNTRY

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

R

Edit3 lines

1library(yamaa)23yamaa_project("project.yaml")$spec("DM")$run()$output

Python

Edit3 lines

1import yamaa23yamaa.project("project.yaml").spec("DM").run().output

solutions/sdtm-dm-xpt/result.R

Edit25 lines

1# Reference solution for the yamaa benchmark sdtm-dm-xpt (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910dm_raw <- read_csv(11  "/app/input/dm_raw.csv",12  col_types = cols(.default = col_character())13)1415dm <- dm_raw |>16  mutate(17    DOMAIN = "DM",18    AGE = as.integer(AGE),19    AGEU = "YEARS"20  ) |>21  select(DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU, SEX,22    RACE, COUNTRY)2324dir.create("/app/output", showWarnings = FALSE)25write_csv(dm, "/app/output/dm.csv", na = "")

solutions/sdtm-dm-xpt/result.py

Edit32 lines

1# Reference solution for the yamaa benchmark sdtm-dm-xpt (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011dm_raw = pl.read_csv("/app/input/dm_raw.csv", infer_schema=False)1213dm = dm_raw.with_columns(14    DOMAIN=pl.lit("DM"),15    AGE=pl.col("AGE").cast(pl.Int64, strict=False),16    AGEU=pl.lit("YEARS"),17).select(18    "DOMAIN",19    "STUDYID",20    "USUBJID",21    "SUBJID",22    "RFSTDTC",23    "SITEID",24    "AGE",25    "AGEU",26    "SEX",27    "RACE",28    "COUNTRY",29)3031Path("/app/output").mkdir(exist_ok=True)32dm.write_csv("/app/output/dm.csv")

solutions/sdtm-dm-xpt/result.sas

Edit17 lines

1/* opensas reference solution for the yamaa benchmark sdtm-dm-xpt.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data dm_raw;6  length STUDYID USUBJID SUBJID SITEID RFSTDTC SEX RACE COUNTRY $1024;7  infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. SUBJID : $char1024. SITEID : $char1024.9    RFSTDTC : $char1024. AGE SEX : $char1024. RACE : $char1024. COUNTRY : $char1024.;10run;1112data result; set dm_raw; length DOMAIN $2 AGEU $8; DOMAIN='DM'; AGEU='YEARS'; run;13proc sql;14  create table final as select DOMAIN, STUDYID, USUBJID, SUBJID, RFSTDTC, SITEID, AGE, AGEU,15    SEX, RACE, COUNTRY from result;16quit;17proc export data=final outfile="/app/output/dm.csv" dbms=csv replace; run;

input/dm_raw.csv

expected/define.xml

Define-XML 2.1.11: CATH

DM

expected/dm.csv

Your review

Mapping spec

Agent prompt

prompts/sdtm-dm-xpt/full.md

prompts/sdtm-dm-xpt/conventions.md

prompts/sdtm-dm-xpt/brief.md

yamaa code

R

Python

Solution without yamaa

solutions/sdtm-dm-xpt/result.R

solutions/sdtm-dm-xpt/result.py

solutions/sdtm-dm-xpt/result.sas