Metadata Contract

Review ask

As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.

Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
Expected output: Would independent QC programming reproduce every value from these inputs and rules?

The other tabs beside Comments are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Goal: build one DM record per subject carrying SITEID, AGE, AGEU, SEX, and COUNTRY.

Input: one row per subject carrying SITEID, AGE, SEX, and COUNTRY.

Variables:

USUBJID joins the study, site, and subject identifiers with hyphens; an identifier longer than 30 characters stops the run rather than being shortened.
SITEID is the collected site identifier.
AGE is the collected age in whole years, between 0 and 120; missing when not collected.
AGEU is fixed to YEARS, since age is collected in whole years.
SEX is the collected sex, one of F, M, or U.
COUNTRY is the collected three-letter country code.

Note: the dataset and its variables carry the description, labels, provenance, lengths, terminology, class, structure, and standard version a review needs. The same declarations produce the data-definition document beside the data and the exchange file dm.json, which carries the rows and the column metadata together and is the file the document points at.

YAML specification

1 spec file

spec.yaml

Edit181 lines

1schema_version: "1.0"2domain: DM3keys: [STUDYID, USUBJID]4input:5  DM_RAW: input/dm_raw.csv67output:8  path: dm.csv9  columns: [DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY]1011columns:12  - name: DOMAIN13    type: str14    label: Domain Abbreviation15    derivation: {literal: DM}16    verifications:17      not_missing: {}18    submission:19      core: Req20      role: Identifier21      length: 222      codelist: DOMAIN23      origin:24        type: Assigned25        source: Sponsor2627  - name: STUDYID28    type: str29    label: Study Identifier30    derivation: DM_RAW.STUDYID31    verifications:32      not_missing: {}33    submission:34      core: Req35      role: Identifier36      length: 2037      origin:38        type: Protocol39        source: Sponsor4041  - name: USUBJID42    type: str43    label: Unique Subject Identifier44    derivation:45      str_concat:46        sources:47          - source: DM_RAW.STUDYID48          - literal: '-'49          - source: DM_RAW.SITEID50          - literal: '-'51          - source: DM_RAW.SUBJID52    verifications:53      - not_missing: {}54      - max_length:55          max: 3056    submission:57      core: Req58      role: Identifier59      origin:60        type: Derived61        source: Sponsor62      method: Study identifier, site identifier, and subject identifier joined63        by hyphens.6465  - name: SUBJID66    type: str67    label: Subject Identifier for the Study68    derivation: DM_RAW.SUBJID69    verifications:70      not_missing: {}71    submission:72      core: Req73      role: Topic74      length: 1075      origin:76        type: Collected77        source: Investigator78        documents:79          - document: acrf80            pages: {type: PhysicalRef, refs: "1"}8182  - name: SITEID83    type: str84    label: Study Site Identifier85    derivation: DM_RAW.SITEID86    verifications:87      not_missing: {}88    submission:89      core: Req90      role: Record Qualifier91      length: 1092      origin:93        type: Collected94        source: Investigator9596  - name: AGE97    type: int98    label: Age99    derivation: DM_RAW.AGE100    verifications:101      range:102        min: 0103        max: 120104    submission:105      core: Exp106      role: Record Qualifier107      length: 3108      origin:109        type: Collected110        source: Investigator111        documents:112          - document: acrf113            pages: {type: PhysicalRef, refs: "1"}114115  - name: AGEU116    type: str117    label: Age Units118    derivation: {literal: YEARS}119    submission:120      core: Exp121      role: Variable Qualifier122      length: 5123      codelist: AGEU124      origin:125        type: Assigned126        source: Sponsor127      comment: Defaulted to YEARS because age is collected in whole years.128129  - name: SEX130    type: str131    label: Sex132    derivation: DM_RAW.SEX133    verifications:134      - not_missing: {}135      - allowed_values:136          values: [F, M, U]137    submission:138      core: Req139      role: Record Qualifier140      length: 1141      codelist: SEX142      origin:143        type: Collected144        source: Investigator145        documents:146          - document: acrf147            pages: {type: PhysicalRef, refs: "1"}148149  - name: COUNTRY150    type: str151    label: Country152    derivation: DM_RAW.COUNTRY153    verifications:154      not_missing: {}155    submission:156      core: Req157      role: Record Qualifier158      length: 3159      codelist: COUNTRY160      origin:161        type: Collected162        source: Investigator163        documents:164          - document: acrf165            pages: {type: PhysicalRef, refs: "1"}166167verifications:168  - unique:169      columns: [STUDYID, USUBJID]170  - row_count:171      min: 3172      max: 3173174submission:175  label: Demographics176  class: SPECIAL PURPOSE177  structure: One record per subject178  repeating: false179  comment:180    text: Demographics are described in section 2.1 of the reviewer's guide.181    documents: [csdrg]

Inputs

Under review1 source file

input/dm_raw.csv

Edit3 rows

input/dm_raw.csv
STUDYID	SUBJID	SITEID	AGE	SEX	COUNTRY
CATH	0001	UCSD	34	F	USA
CATH	0002	UCSD	67	M	USA
CATH	0003	MCGILL	52	F	CAN

Expected output

Under review3 expected rows

expected/define.xml

Edit

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="www.example.org/STUDY01/1/Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" Originator="Example Sponsor" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.STUDY01">
    <GlobalVariables>
      <StudyName>STUDY01</StudyName>
      <StudyDescription>Example study demonstrating the Define-XML metadata contract</StudyDescription>
      <ProtocolName>STUDY-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.STUDY01.1" Name="Study STUDY01, Data Definitions" Description="Tabulation data definitions for STUDY01" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.SDTMIG" Name="SDTMIG" Type="IG" Version="3.4" Status="Final"/>
        <def:Standard OID="STD.CT_SDTM" Name="CDISC/NCI" Type="CT" PublishingSet="SDTM" Version="2023-12-15" Status="Final"/>
      </def:Standards>
      <def:AnnotatedCRF>
        <def:DocumentRef leafID="LF.acrf"/>
      </def:AnnotatedCRF>
      <def:SupplementalDoc>
        <def:DocumentRef leafID="LF.csdrg"/>
      </def:SupplementalDoc>
      <ItemGroupDef OID="IG.DM" Name="DM" Domain="DM" Purpose="Tabulation" SASDatasetName="DM" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.DM" def:StandardOID="STD.SDTMIG" def:CommentOID="COM.DM">
        <Description>
          <TranslatedText xml:lang="en">Demographics</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.DM.DOMAIN" OrderNumber="1" Mandatory="Yes" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.STUDYID" OrderNumber="2" Mandatory="Yes" KeySequence="1" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.USUBJID" OrderNumber="3" Mandatory="Yes" KeySequence="2" Role="Identifier" MethodOID="MT.DM.USUBJID"/>
        <ItemRef ItemOID="IT.DM.SUBJID" OrderNumber="4" Mandatory="Yes" Role="Topic"/>
        <ItemRef ItemOID="IT.DM.SITEID" OrderNumber="5" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGE" OrderNumber="6" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.AGEU" OrderNumber="7" Mandatory="No" Role="Variable Qualifier"/>
        <ItemRef ItemOID="IT.DM.SEX" OrderNumber="8" Mandatory="Yes" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.COUNTRY" OrderNumber="9" Mandatory="Yes" Role="Record Qualifier"/>
        <def:Class Name="SPECIAL PURPOSE"/>
        <def:leaf ID="LF.DM" xlink:href="dm.json">
          <def:title>dm.json</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.DM.DOMAIN" Name="DOMAIN" DataType="text" Length="2" SASFieldName="DOMAIN">
        <Description>
          <TranslatedText xml:lang="en">Domain Abbreviation</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.DOMAIN"/>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.STUDYID" Name="STUDYID" DataType="text" Length="20" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Protocol" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.USUBJID" Name="USUBJID" DataType="text" Length="30" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SUBJID" Name="SUBJID" DataType="text" Length="10" SASFieldName="SUBJID">
        <Description>
          <TranslatedText xml:lang="en">Subject Identifier for the Study</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.SITEID" Name="SITEID" DataType="text" Length="10" SASFieldName="SITEID">
        <Description>
          <TranslatedText xml:lang="en">Study Site Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf"/>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGE" Name="AGE" DataType="integer" Length="3" SASFieldName="AGE">
        <Description>
          <TranslatedText xml:lang="en">Age</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.AGEU" Name="AGEU" DataType="text" Length="5" SASFieldName="AGEU" def:CommentOID="COM.DM.AGEU">
        <Description>
          <TranslatedText xml:lang="en">Age Units</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.AGEU"/>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.SEX" Name="SEX" DataType="text" Length="1" SASFieldName="SEX">
        <Description>
          <TranslatedText xml:lang="en">Sex</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.SEX"/>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.COUNTRY" Name="COUNTRY" DataType="text" Length="3" SASFieldName="COUNTRY">
        <Description>
          <TranslatedText xml:lang="en">Country</TranslatedText>
        </Description>
        <CodeListRef CodeListOID="CL.COUNTRY"/>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <CodeList OID="CL.DOMAIN" Name="Domain Abbreviation (DM)" DataType="text" def:StandardOID="STD.CT_SDTM" SASFormatName="$DMDOMAI">
        <CodeListItem CodedValue="DM">
          <Decode>
            <TranslatedText xml:lang="en">Demographics</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C49572"/>
        </CodeListItem>
        <Alias Context="nci:ExtCodeID" Name="C66734"/>
      </CodeList>
      <CodeList OID="CL.AGEU" Name="Age Unit" DataType="text" def:StandardOID="STD.CT_SDTM">
        <EnumeratedItem CodedValue="YEARS">
          <Alias Context="nci:ExtCodeID" Name="C29848"/>
        </EnumeratedItem>
        <Alias Context="nci:ExtCodeID" Name="C66781"/>
      </CodeList>
      <CodeList OID="CL.SEX" Name="Sex" DataType="text" def:StandardOID="STD.CT_SDTM" SASFormatName="$SEX">
        <CodeListItem CodedValue="F">
          <Decode>
            <TranslatedText xml:lang="en">Female</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C16576"/>
        </CodeListItem>
        <CodeListItem CodedValue="M">
          <Decode>
            <TranslatedText xml:lang="en">Male</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C20197"/>
        </CodeListItem>
        <CodeListItem CodedValue="U">
          <Decode>
            <TranslatedText xml:lang="en">Unknown</TranslatedText>
          </Decode>
          <Alias Context="nci:ExtCodeID" Name="C17998"/>
        </CodeListItem>
        <Alias Context="nci:ExtCodeID" Name="C66731"/>
      </CodeList>
      <CodeList OID="CL.COUNTRY" Name="Country Codes" DataType="text" def:StandardOID="STD.CT_SDTM">
        <ExternalCodeList Dictionary="ISO 3166" Version="2020" href="https://www.iso.org/iso-3166-country-codes.html"/>
      </CodeList>
      <MethodDef OID="MT.DM.USUBJID" Name="DM.USUBJID" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Study identifier, site identifier, and subject identifier joined by hyphens.</TranslatedText>
        </Description>
      </MethodDef>
      <def:CommentDef OID="COM.DM">
        <Description>
          <TranslatedText xml:lang="en">Demographics are described in section 2.1 of the reviewer's guide.</TranslatedText>
        </Description>
        <def:DocumentRef leafID="LF.csdrg"/>
      </def:CommentDef>
      <def:CommentDef OID="COM.DM.AGEU">
        <Description>
          <TranslatedText xml:lang="en">Defaulted to YEARS because age is collected in whole years.</TranslatedText>
        </Description>
      </def:CommentDef>
      <def:leaf ID="LF.acrf" xlink:href="acrf.pdf">
        <def:title>Annotated Case Report Form</def:title>
      </def:leaf>
      <def:leaf ID="LF.csdrg" xlink:href="csdrg.pdf">
        <def:title>Study Data Reviewer's Guide</def:title>
      </def:leaf>
    </MetaDataVersion>
  </Study>
</ODM>

expected/dm.csv

Edit3 rows

expected/dm.csv
DOMAIN	STUDYID	SUBJID	SITEID	AGE	AGEU	SEX	COUNTRY
DM	CATH	0001	UCSD	34	YEARS	F	USA
DM	CATH	0002	UCSD	67	YEARS	M	USA
DM	CATH	0003	MCGILL	52	YEARS	F	CAN

expected/dm.json

Edit

{
  "datasetJSONCreationDateTime": "2026-09-11T00:00:00",
  "datasetJSONVersion": "1.1.0",
  "fileOID": "www.example.org/STUDY01/1/Define-XML_2.1.DM",
  "originator": "Example Sponsor",
  "sourceSystem": {"name": "yamaa", "version": "1.0"},
  "studyOID": "STDY.STUDY01",
  "metaDataVersionOID": "MDV.STUDY01.1",
  "metaDataRef": "define.xml",
  "itemGroupOID": "IG.DM",
  "records": 3,
  "name": "DM",
  "label": "Demographics",
  "columns": [
    {"itemOID": "IT.DM.DOMAIN", "name": "DOMAIN", "label": "Domain Abbreviation", "dataType": "string", "length": 2},
    {"itemOID": "IT.DM.STUDYID", "name": "STUDYID", "label": "Study Identifier", "dataType": "string", "length": 20, "keySequence": 1},
    {"itemOID": "IT.DM.USUBJID", "name": "USUBJID", "label": "Unique Subject Identifier", "dataType": "string", "length": 30, "keySequence": 2},
    {"itemOID": "IT.DM.SUBJID", "name": "SUBJID", "label": "Subject Identifier for the Study", "dataType": "string", "length": 10},
    {"itemOID": "IT.DM.SITEID", "name": "SITEID", "label": "Study Site Identifier", "dataType": "string", "length": 10},
    {"itemOID": "IT.DM.AGE", "name": "AGE", "label": "Age", "dataType": "integer", "length": 3},
    {"itemOID": "IT.DM.AGEU", "name": "AGEU", "label": "Age Units", "dataType": "string", "length": 5},
    {"itemOID": "IT.DM.SEX", "name": "SEX", "label": "Sex", "dataType": "string", "length": 1},
    {"itemOID": "IT.DM.COUNTRY", "name": "COUNTRY", "label": "Country", "dataType": "string", "length": 3}
  ],
  "rows": [
    ["DM", "CATH", "CATH-UCSD-0001", "0001", "UCSD", 34, "YEARS", "F", "USA"],
    ["DM", "CATH", "CATH-UCSD-0002", "0002", "UCSD", 67, "YEARS", "M", "USA"],
    ["DM", "CATH", "CATH-MCGILL-0003", "0003", "MCGILL", 52, "YEARS", "F", "CAN"]
  ]
}

Shaded columns are derived— = missing value

Comments and assessment materials

Variable Name	Variable Label	Type	Length	Controlled Terms or Format	Origin	Core	Conversion Definition	Variable Type	Variable Order	Comments for Define
DOMAIN	Domain Abbreviation	Char	2	Domain Abbreviation (DM)	Assigned	Req	Set constant value "DM".	SDTM	1
STUDYID	Study Identifier	Char	20		Protocol	Req	Copy value from DM_RAW.STUDYID.	SDTM	2
USUBJID	Unique Subject Identifier	Char			Derived	Req	Study identifier, site identifier, and subject identifier joined by hyphens.	SDTM	3
SUBJID	Subject Identifier for the Study	Char	10		Collected	Req	Copy value from DM_RAW.SUBJID.	SDTM	4
SITEID	Study Site Identifier	Char	10		Collected	Req	Copy value from DM_RAW.SITEID.	SDTM	5
AGE	Age	Num	3		Collected	Exp	Copy value from DM_RAW.AGE.	SDTM	6
AGEU	Age Units	Char	5	Age Unit	Assigned	Exp	Set constant value "YEARS".	SDTM	7	Defaulted to YEARS because age is collected in whole years.
SEX	Sex	Char	1	Sex	Collected	Req	Copy value from DM_RAW.SEX.	SDTM	8
COUNTRY	Country	Char	3	Country Codes (ISO 3166, version 2020)	Collected	Req	Copy value from DM_RAW.COUNTRY.	SDTM	9

Codelist ID	Name	Value	Decode	Codelist NCI alias	Value NCI alias	Extensible	SAS format	Dictionary	Version
DOMAIN	Domain Abbreviation (DM)	DM	Demographics	C66734	C49572	No	$DMDOMAI
AGEU	Age Unit	YEARS		C66781	C29848	No
SEX	Sex	F	Female	C66731	C16576	No	$SEX
SEX	Sex	M	Male	C66731	C20197	No	$SEX
SEX	Sex	U	Unknown	C66731	C17998	No	$SEX
COUNTRY	Country Codes					No		ISO 3166	2020

Version	Date	Author	Description
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-dm-metadata/full.md

Edit14 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

USUBJID is the study, site, and subject identifiers with hyphens between them. SITEID is the collected site identifier, and SUBJID and COUNTRY are as collected, SUBJID keeping its leading zeros. AGE is the collected age in whole years, with no value when not collected. AGEU is fixed to YEARS. SEX is the collected sex, one of F, M, or U.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-metadata/conventions.md

Edit10 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

SEX is one of F, M, or U.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-metadata/brief.md

Edit8 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

run.R

Edit4 lines

1library(yamaa)23dm <- yamaa_domain("spec.yaml")$output4dm

run.py

Edit4 lines

1import yamaa23dm = yamaa.yamaa_domain("spec.yaml").output4dm

solutions/sdtm-dm-metadata/result.R

Edit25 lines

1# Reference solution for the yamaa benchmark sdtm-dm-metadata (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910dm_raw <- read_csv(11  "/app/input/dm_raw.csv",12  col_types = cols(.default = col_character())13)1415dm <- dm_raw |>16  mutate(17    DOMAIN = "DM",18    USUBJID = paste(STUDYID, SITEID, SUBJID, sep = "-"),19    AGE = suppressWarnings(as.integer(AGE)),20    AGEU = "YEARS"21  ) |>22  select(DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY)2324dir.create("/app/output", showWarnings = FALSE)25write_csv(dm, "/app/output/dm.csv", na = "")

solutions/sdtm-dm-metadata/result.py

Edit34 lines

1# Reference solution for the yamaa benchmark sdtm-dm-metadata (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011dm_raw = pl.read_csv("/app/input/dm_raw.csv", infer_schema=False)1213dm = (14    dm_raw.with_columns(15        DOMAIN=pl.lit("DM"),16        USUBJID=pl.col("STUDYID") + pl.lit("-") + pl.col("SITEID") + pl.lit("-") + pl.col("SUBJID"),17        AGE=pl.col("AGE").cast(pl.Int64, strict=False),18        AGEU=pl.lit("YEARS"),19    )20    .select(21        "DOMAIN",22        "STUDYID",23        "USUBJID",24        "SUBJID",25        "SITEID",26        "AGE",27        "AGEU",28        "SEX",29        "COUNTRY",30    )31)3233Path("/app/output").mkdir(exist_ok=True)34dm.write_csv("/app/output/dm.csv")

solutions/sdtm-dm-metadata/result.sas

Edit15 lines

1/* opensas reference solution for the yamaa benchmark sdtm-dm-metadata.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data dm_raw;6  length STUDYID SUBJID SITEID SEX COUNTRY $1024;7  infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. SUBJID : $char1024. SITEID : $char1024. AGE SEX : $char1024. COUNTRY : $char1024.;9run;1011data result; set dm_raw; length DOMAIN $2 USUBJID $1024 AGEU $8; DOMAIN='DM'; USUBJID=catx('-',STUDYID,SITEID,SUBJID); AGEU='YEARS'; run;12proc sql;13  create table final as select DOMAIN, STUDYID, USUBJID, SUBJID, SITEID, AGE, AGEU, SEX, COUNTRY from result;14quit;15proc export data=final outfile="/app/output/dm.csv" dbms=csv replace; run;

input/dm_raw.csv

expected/define.xml

expected/dm.csv

expected/dm.json

Comments

Mapping spec

Agent prompt

prompts/sdtm-dm-metadata/full.md

prompts/sdtm-dm-metadata/conventions.md

prompts/sdtm-dm-metadata/brief.md

yamaa code

run.R

run.py

Solution without yamaa

solutions/sdtm-dm-metadata/result.R

solutions/sdtm-dm-metadata/result.py

solutions/sdtm-dm-metadata/result.sas