Demographics When No Subject Died

Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: derive the date of death (DTHDTC) and the death flag (DTHFL) for each subject, and submit the demographics as the Dataset-JSON file dm.json beside its data-definition document.

Input: one row per subject carrying the reference start date and the date of death collected on the death page.

Variables:

RFSTDTC is the subject's reference start date; empty when the subject has none.
DTHDTC is the collected date of death; empty when the subject did not die.
DTHFL is Y when the subject has a date of death; empty otherwise.

Note: the dataset keeps both death variables even when no subject died, because the standard expects them in every demographics dataset. The data-definition document then marks each variable that no subject has a value for as having no data; a variable empty for only some subjects, such as RFSTDTC, is not marked.

YAML specification

1 spec file

spec.yaml

Edit97 lines

1yamaa: "1.0"2type: spec3model: SDTM4domain: DM5keys: [STUDYID, USUBJID]6input:7  DM_RAW: input/dm_raw.csv89output:10  path: dm.csv11  columns: [DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL]1213columns:14  - name: DOMAIN15    type: str16    label: Domain Abbreviation17    derivation: "'DM'"18    submission:19      core: Req20      role: Identifier21      length: 222      origins:23        - type: Assigned24          source: Sponsor2526  - name: STUDYID27    type: str28    label: Study Identifier29    derivation: DM_RAW.STUDYID30    submission:31      core: Req32      role: Identifier33      length: 2034      origins:35        - type: Protocol36          source: Sponsor3738  - name: USUBJID39    type: str40    label: Unique Subject Identifier41    derivation: DM_RAW.USUBJID42    submission:43      core: Req44      role: Identifier45      length: 3046      origins:47        - type: Assigned48          source: Sponsor4950  - name: RFSTDTC51    type: date52    label: Subject Reference Start Date/Time53    derivation: DM_RAW.RFSTDTC54    submission:55      core: Exp56      role: Record Qualifier57      origins:58        - type: Collected59          source: Investigator60          documents:61            - document: acrf62              pages: {type: PhysicalRef, refs: "1"}6364  # Expected variables: the dataset keeps them although no subject died.65  - name: DTHDTC66    type: date67    label: Date/Time of Death68    derivation: DM_RAW.DTHDAT69    submission:70      core: Exp71      role: Record Qualifier72      origins:73        - type: Collected74          source: Investigator75          documents:76            - document: acrf77              pages: {type: PhysicalRef, refs: "2"}7879  - name: DTHFL80    type: str81    label: Subject Death Flag82    derivation:83      flag: "DTHDTC IS NOT NULL"84    submission:85      core: Exp86      role: Record Qualifier87      length: 188      origins:89        - type: Derived90          source: Sponsor91      method: Y when the subject has a date of death; otherwise empty.9293submission:94  label: Demographics95  class: SPECIAL PURPOSE96  structure: One record per subject97  repeating: false

Inputs

Under review1 source file

input/dm_raw.csv

Edit2 rows

input/dm_raw.csv
STUDYID	USUBJID	RFSTDTC	DTHDAT
CATH		2025-01-08	—
CATH		—	—

Expected output

Under review2 expected rows

expected/define.xml

EditDefine-XML

Define-XML 2.1.11: CATH

Example study submitting demographics when no subject died; CATH-01; SDTMIG 3.4

Datasets
Dataset	Description	Class	Structure	Purpose	Keys	Documentation	Location
DM	Demographics	SPECIAL PURPOSE	One record per subject	Tabulation	STUDYID, USUBJID		dm.json

DM

DM variables
Variable	Label / Description	Type	Role	Length or Display Format	Controlled Terms or ISO Format	Origin / Source / Method / Comment
DOMAIN	Domain Abbreviation	text	Identifier	2		Assigned; Sponsor
STUDYID	Study Identifier	text	Identifier	4		Protocol; Sponsor
USUBJID	Unique Subject Identifier	text	Identifier	8		Assigned; Sponsor
RFSTDTC	Subject Reference Start Date/Time	date	Record Qualifier		ISO 8601	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 1
DTHDTC	Date/Time of Death	date	Record Qualifier		ISO 8601	Collected; Investigator; Annotated Case Report Form (acrf.pdf); pages 2
DTHFL	Subject Death Flag	text	Record Qualifier	1		Derived; Sponsor Y when the subject has a date of death; otherwise empty.

Original source

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.sdtm.Define-XML_2.1" CreationDateTime="2026-10-28T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study submitting demographics when no subject died</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.sdtm" Name="CATH sdtm metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.SDTMIG" Name="SDTMIG" Type="IG" Version="3.4" Status="Final"/>
      </def:Standards>
      <def:AnnotatedCRF>
        <def:DocumentRef leafID="LF.acrf"/>
      </def:AnnotatedCRF>
      <ItemGroupDef OID="IG.DM" Name="DM" Domain="DM" Purpose="Tabulation" SASDatasetName="DM" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.DM" def:StandardOID="STD.SDTMIG">
        <Description>
          <TranslatedText xml:lang="en">Demographics</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.DM.DOMAIN" OrderNumber="1" Mandatory="Yes" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.STUDYID" OrderNumber="2" Mandatory="Yes" KeySequence="1" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.USUBJID" OrderNumber="3" Mandatory="Yes" KeySequence="2" Role="Identifier"/>
        <ItemRef ItemOID="IT.DM.RFSTDTC" OrderNumber="4" Mandatory="No" Role="Record Qualifier"/>
        <ItemRef ItemOID="IT.DM.DTHDTC" OrderNumber="5" Mandatory="No" Role="Record Qualifier" def:HasNoData="Yes"/>
        <ItemRef ItemOID="IT.DM.DTHFL" OrderNumber="6" Mandatory="No" Role="Record Qualifier" MethodOID="MT.DM.DTHFL" def:HasNoData="Yes"/>
        <def:Class Name="SPECIAL PURPOSE"/>
        <def:leaf ID="LF.DM" xlink:href="dm.json">
          <def:title>dm.json</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.DM.DOMAIN" Name="DOMAIN" DataType="text" Length="2" SASFieldName="DOMAIN">
        <Description>
          <TranslatedText xml:lang="en">Domain Abbreviation</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Protocol" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.USUBJID" Name="USUBJID" DataType="text" Length="8" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Assigned" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.DM.RFSTDTC" Name="RFSTDTC" DataType="date" SASFieldName="RFSTDTC">
        <Description>
          <TranslatedText xml:lang="en">Subject Reference Start Date/Time</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="1" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.DTHDTC" Name="DTHDTC" DataType="date" SASFieldName="DTHDTC">
        <Description>
          <TranslatedText xml:lang="en">Date/Time of Death</TranslatedText>
        </Description>
        <def:Origin Type="Collected" Source="Investigator">
          <def:DocumentRef leafID="LF.acrf">
            <def:PDFPageRef PageRefs="2" Type="PhysicalRef"/>
          </def:DocumentRef>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.DM.DTHFL" Name="DTHFL" DataType="text" Length="1" SASFieldName="DTHFL">
        <Description>
          <TranslatedText xml:lang="en">Subject Death Flag</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <MethodDef OID="MT.DM.DTHFL" Name="DM.DTHFL" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Y when the subject has a date of death; otherwise empty.</TranslatedText>
        </Description>
      </MethodDef>
      <def:leaf ID="LF.acrf" xlink:href="acrf.pdf">
        <def:title>Annotated Case Report Form</def:title>
      </def:leaf>
    </MetaDataVersion>
  </Study>
</ODM>

expected/dm.csv

Edit2 rows

expected/dm.csv
DOMAIN	STUDYID	USUBJID	RFSTDTC	DTHDTC	DTHFL
DM	CATH		2025-01-08	—	—
DM	CATH		—	—	—

expected/dm.json

Edit

{
  "datasetJSONCreationDateTime": "2026-10-28T00:00:00",
  "datasetJSONVersion": "1.1.0",
  "fileOID": "CATH.sdtm.Define-XML_2.1.DM",
  "sourceSystem": {"name": "yamaa", "version": "1.0"},
  "studyOID": "STDY.CATH",
  "metaDataVersionOID": "MDV.CATH.sdtm",
  "metaDataRef": "define.xml",
  "itemGroupOID": "IG.DM",
  "records": 2,
  "name": "DM",
  "label": "Demographics",
  "columns": [
    {"itemOID": "IT.DM.DOMAIN", "name": "DOMAIN", "label": "Domain Abbreviation", "dataType": "string", "length": 2},
    {"itemOID": "IT.DM.STUDYID", "name": "STUDYID", "label": "Study Identifier", "dataType": "string", "length": 4, "keySequence": 1},
    {"itemOID": "IT.DM.USUBJID", "name": "USUBJID", "label": "Unique Subject Identifier", "dataType": "string", "length": 8, "keySequence": 2},
    {"itemOID": "IT.DM.RFSTDTC", "name": "RFSTDTC", "label": "Subject Reference Start Date/Time", "dataType": "date"},
    {"itemOID": "IT.DM.DTHDTC", "name": "DTHDTC", "label": "Date/Time of Death", "dataType": "date"},
    {"itemOID": "IT.DM.DTHFL", "name": "DTHFL", "label": "Subject Death Flag", "dataType": "string", "length": 1}
  ],
  "rows": [
    ["DM", "CATH", "CATH-001", "2025-01-08", null, null],
    ["DM", "CATH", "CATH-002", null, null, null]
  ]
}

Shaded columns are derived— = missing value

Human review and assessment

Variable Name	Variable Label	Type	Length	Controlled Terms or Format	Origin	Core	Conversion Definition	Variable Type	Variable Order
DOMAIN	Domain Abbreviation	Char	2		Assigned	Req	Set constant value "DM".	SDTM	1
STUDYID	Study Identifier	Char	20		Protocol	Req	Copy value from DM_RAW.STUDYID.	SDTM	2
USUBJID	Unique Subject Identifier	Char	30		Assigned	Req	Copy value from DM_RAW.USUBJID.	SDTM	3
RFSTDTC	Subject Reference Start Date/Time	Char		ISO 8601	Collected	Exp	Copy value from DM_RAW.RFSTDTC.	SDTM	4
DTHDTC	Date/Time of Death	Char		ISO 8601	Collected	Exp	Copy value from DM_RAW.DTHDAT.	SDTM	5
DTHFL	Subject Death Flag	Char	1		Derived	Exp	Y when the subject has a date of death; otherwise empty.	SDTM	6

Version	Date	Author	Description
1.7	2026-10-20	yamaa docs build	Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.6	2026-10-17	yamaa docs build	Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.5	2026-10-17	yamaa docs build	Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.4	2026-10-15	yamaa docs build	State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/sdtm-dm-no-deaths/full.md

Edit14 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL

DOMAIN is "DM". RFSTDTC is the subject's reference start date, with no value when the subject has none. DTHDTC is the date of death collected for the subject, with no value when the subject did not die. DTHFL is "Y" when the subject has a date of death, and has no value otherwise. The dataset keeps DTHDTC and DTHFL even when no subject has a value for them.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-no-deaths/conventions.md

Edit11 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL

DOMAIN is "DM". DTHFL is "Y" or has no value. The dataset keeps DTHDTC and DTHFL even when no subject has a value for them.

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

prompts/sdtm-dm-no-deaths/brief.md

Edit8 lines

Following CDISC SDTM standards, use the provided DM_RAW dataset to create a DM dataset with one record per subject.

The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL

Read the source datasets from /app/input and save the completed dataset as /app/output/dm.csv.

R

Edit3 lines

1library(yamaa)23yamaa_project("project.yaml")$spec("DM")$run()$output

Python

Edit3 lines

1import yamaa23yamaa.project("project.yaml").spec("DM").run().output

solutions/sdtm-dm-no-deaths/result.R

Edit24 lines

1# Reference solution for the yamaa benchmark sdtm-dm-no-deaths (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910dm_raw <- read_csv(11  "/app/input/dm_raw.csv",12  col_types = cols(.default = col_character())13)1415dm <- dm_raw |>16  mutate(17    DOMAIN = "DM",18    DTHDTC = DTHDAT,19    DTHFL = ifelse(!is.na(DTHDTC), "Y", NA_character_)20  ) |>21  select(DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL)2223dir.create("/app/output", showWarnings = FALSE)24write_csv(dm, "/app/output/dm.csv", na = "")

solutions/sdtm-dm-no-deaths/result.py

Edit50 lines

1# Reference solution for the yamaa benchmark sdtm-dm-no-deaths (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67"""Create the SDTM DM dataset with the death variables kept.89One record per subject (from DM_RAW). RFSTDTC is the reference start date10and DTHDTC the collected date of death, each blank when the subject has11none. DTHFL is "Y" when the subject has a date of death, and blank12otherwise. Both death variables stay in the dataset even when every value13is blank.14"""1516import csv17from pathlib import Path1819INPUT_DIR = Path("/app/input")20OUTPUT_DIR = Path("/app/output")21COLUMNS = ["DOMAIN", "STUDYID", "USUBJID", "RFSTDTC", "DTHDTC", "DTHFL"]222324def main():25    with open(INPUT_DIR / "dm_raw.csv", newline="", encoding="utf-8") as handle:26        subjects = list(csv.DictReader(handle))2728    rows = []29    for subject in subjects:30        dthdtc = subject["DTHDAT"].strip()31        rows.append(32            {33                "DOMAIN": "DM",34                "STUDYID": subject["STUDYID"],35                "USUBJID": subject["USUBJID"],36                "RFSTDTC": subject["RFSTDTC"].strip(),37                "DTHDTC": dthdtc,38                "DTHFL": "Y" if dthdtc else "",39            }40        )4142    OUTPUT_DIR.mkdir(parents=True, exist_ok=True)43    with open(OUTPUT_DIR / "dm.csv", "w", newline="", encoding="utf-8") as handle:44        writer = csv.DictWriter(handle, fieldnames=COLUMNS)45        writer.writeheader()46        writer.writerows(rows)474849if __name__ == "__main__":50    main()

solutions/sdtm-dm-no-deaths/result.sas

Edit22 lines

1/* opensas reference solution for the yamaa benchmark sdtm-dm-no-deaths.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data dm_raw;6  length STUDYID USUBJID RFSTDTC DTHDAT $1024;7  infile "/app/input/dm_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. RFSTDTC : $char1024. DTHDAT : $char1024.;9run;1011data result;12  set dm_raw;13  length DOMAIN $2 DTHDTC $1024 DTHFL $1;14  DOMAIN = 'DM';15  DTHDTC = DTHDAT;16  if not missing(DTHDTC) then DTHFL = 'Y';17run;1819proc sql;20  create table final as select DOMAIN, STUDYID, USUBJID, RFSTDTC, DTHDTC, DTHFL from result;21quit;22proc export data=final outfile="/app/output/dm.csv" dbms=csv replace; run;

input/dm_raw.csv

expected/define.xml

Define-XML 2.1.11: CATH

DM

expected/dm.csv

expected/dm.json

Your review

Mapping spec

Agent prompt

prompts/sdtm-dm-no-deaths/full.md

prompts/sdtm-dm-no-deaths/conventions.md

prompts/sdtm-dm-no-deaths/brief.md

yamaa code

R

Python

Solution without yamaa

solutions/sdtm-dm-no-deaths/result.R

solutions/sdtm-dm-no-deaths/result.py

solutions/sdtm-dm-no-deaths/result.sas