Subject-Level Analysis Data Submitted in SAS Transport

Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

EditUnder reviewWhat this benchmark means

README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: carry TRT01A, AGE, HEIGHTBL, TRTSDT and TRTSDTM into ADSL and submit it as the SAS transport file adsl.xpt, with the dates as SAS numeric dates.

Input: one pre-analysis row per subject carrying the actual treatment, age, baseline height and the collected date and time of first exposure.

Variables:

TRT01A is the actual treatment.
AGE is the age in years, shown in three positions.
HEIGHTBL is the baseline height in centimetres, shown with one decimal; missing when not measured.
TRTSDT is the date part of TRTSDTM; missing when the subject was never treated.
TRTSDTM is the date and time of first exposure; missing when the subject was never treated.

Note: in the transport file TRTSDT counts days and TRTSDTM seconds from 1 January 1960, shown as DATE9. and DATETIME20., and the data-definition document describes both as eight-byte integers with those formats. Each text variable is exactly as wide as its longest value.

YAML specification

1 spec file

spec.yaml

Edit105 lines

1yamaa: "1.0"2type: spec3model: ADaM4domain: ADSL5keys: [STUDYID, USUBJID]6input:7  ADSL_RAW: input/adsl_raw.csv89output:10  path: adsl.csv11  columns: [STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM]1213columns:14  - name: STUDYID15    type: str16    label: Study Identifier17    derivation: ADSL_RAW.STUDYID18    submission:19      core: Req20      mandatory: true21      length: 2022      origins:23        - type: Predecessor24          description: ADSL_RAW.STUDYID2526  - name: USUBJID27    type: str28    label: Unique Subject Identifier29    derivation: ADSL_RAW.USUBJID30    submission:31      core: Req32      mandatory: true33      length: 3034      origins:35        - type: Predecessor36          description: ADSL_RAW.USUBJID3738  - name: TRT01A39    type: str40    label: Actual Treatment for Period 0141    derivation: ADSL_RAW.TRT01A42    submission:43      core: Cond44      mandatory: true45      length: 2046      origins:47        - type: Predecessor48          description: ADSL_RAW.TRT01A4950  - name: AGE51    type: int52    label: Age53    derivation: ADSL_RAW.AGE54    submission:55      core: Req56      mandatory: true57      length: 358      display_format: "3."59      origins:60        - type: Predecessor61          description: ADSL_RAW.AGE6263  - name: HEIGHTBL64    type: float65    label: Baseline Height (cm)66    derivation: ADSL_RAW.HEIGHTBL67    submission:68      core: Perm69      mandatory: false70      length: 571      significant_digits: 172      display_format: "5.1"73      origins:74        - type: Predecessor75          description: ADSL_RAW.HEIGHTBL7677  - name: TRTSDTM78    type: datetime79    label: Datetime of First Exposure to Treatment80    derivation: ADSL_RAW.TRTSDTC81    submission:82      core: Perm83      mandatory: false84      origins:85        - type: Predecessor86          description: ADSL_RAW.TRTSDTC8788  - name: TRTSDT89    type: date90    label: Date of First Exposure to Treatment91    derivation:92      to_date:93        source: TRTSDTM94    submission:95      core: Cond96      mandatory: false97      origins:98        - type: Derived99      method: Date part of TRTSDTM.100101submission:102  label: Subject-Level Analysis Dataset103  class: SUBJECT LEVEL ANALYSIS DATASET104  structure: One record per subject105  repeating: false

Inputs

Under review1 source file

input/adsl_raw.csv

Edit3 rows

input/adsl_raw.csv
STUDYID	TRT01A	AGE	HEIGHTBL	TRTSDTC
CATH	VITAMIN D3	34	172.5	2025-01-08T08:30
CATH	PLACEBO	67	—	2025-01-15T09:05:30
CATH	NOT TREATED	52	165.2	—

Expected output

Under review3 expected rows

expected/adsl.csv

Edit3 rows

expected/adsl.csv
STUDYID	TRT01A	AGE	HEIGHTBL	TRTSDT	TRTSDTM
CATH	VITAMIN D3	34	172.5	2025-01-08	2025-01-08T08:30:00
CATH	PLACEBO	67	—	2025-01-15	2025-01-15T09:05:30
CATH	NOT TREATED	52	165.2	—	—

expected/define.xml

EditDefine-XML

Define-XML 2.1.11: CATH

Example study submitting subject-level analysis data in SAS transport; CATH-01; ADaMIG 1.3

Datasets
Dataset	Description	Class	Structure	Purpose	Keys	Documentation	Location
ADSL	Subject-Level Analysis Dataset	SUBJECT LEVEL ANALYSIS DATASET	One record per subject	Analysis	STUDYID, USUBJID		adsl.xpt

ADSL

ADSL variables
Variable	Label / Description	Type	Length or Display Format	Origin / Source / Method / Comment
STUDYID	Study Identifier	text	4	Predecessor; ADSL_RAW.STUDYID
USUBJID	Unique Subject Identifier	text	16	Predecessor; ADSL_RAW.USUBJID
TRT01A	Actual Treatment for Period 01	text	11	Predecessor; ADSL_RAW.TRT01A
AGE	Age	integer	3.	Predecessor; ADSL_RAW.AGE
HEIGHTBL	Baseline Height (cm)	float	5.1	Predecessor; ADSL_RAW.HEIGHTBL
TRTSDT	Date of First Exposure to Treatment	integer	DATE9.	Derived; Sponsor Date part of TRTSDTM.
TRTSDTM	Datetime of First Exposure to Treatment	integer	DATETIME20.	Predecessor; ADSL_RAW.TRTSDTC

Original source

<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.adam.Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study submitting subject-level analysis data in SAS transport</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.adam" Name="CATH adam metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.ADaMIG" Name="ADaMIG" Type="IG" Version="1.3" Status="Final"/>
      </def:Standards>
      <ItemGroupDef OID="IG.ADSL" Name="ADSL" Purpose="Analysis" SASDatasetName="ADSL" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.ADSL" def:StandardOID="STD.ADaMIG">
        <Description>
          <TranslatedText xml:lang="en">Subject-Level Analysis Dataset</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.ADSL.STUDYID" OrderNumber="1" Mandatory="Yes" KeySequence="1"/>
        <ItemRef ItemOID="IT.ADSL.USUBJID" OrderNumber="2" Mandatory="Yes" KeySequence="2"/>
        <ItemRef ItemOID="IT.ADSL.TRT01A" OrderNumber="3" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.ADSL.AGE" OrderNumber="4" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.ADSL.HEIGHTBL" OrderNumber="5" Mandatory="No"/>
        <ItemRef ItemOID="IT.ADSL.TRTSDT" OrderNumber="6" Mandatory="No" MethodOID="MT.ADSL.TRTSDT"/>
        <ItemRef ItemOID="IT.ADSL.TRTSDTM" OrderNumber="7" Mandatory="No"/>
        <def:Class Name="SUBJECT LEVEL ANALYSIS DATASET"/>
        <def:leaf ID="LF.ADSL" xlink:href="adsl.xpt">
          <def:title>adsl.xpt</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.ADSL.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.STUDYID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.USUBJID" Name="USUBJID" DataType="text" Length="16" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.USUBJID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRT01A" Name="TRT01A" DataType="text" Length="11" SASFieldName="TRT01A">
        <Description>
          <TranslatedText xml:lang="en">Actual Treatment for Period 01</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.TRT01A</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.AGE" Name="AGE" DataType="integer" Length="3" SASFieldName="AGE" def:DisplayFormat="3.">
        <Description>
          <TranslatedText xml:lang="en">Age</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.AGE</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.HEIGHTBL" Name="HEIGHTBL" DataType="float" Length="5" SignificantDigits="1" SASFieldName="HEIGHTBL" def:DisplayFormat="5.1">
        <Description>
          <TranslatedText xml:lang="en">Baseline Height (cm)</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.HEIGHTBL</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRTSDT" Name="TRTSDT" DataType="integer" Length="8" SASFieldName="TRTSDT" def:DisplayFormat="DATE9.">
        <Description>
          <TranslatedText xml:lang="en">Date of First Exposure to Treatment</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRTSDTM" Name="TRTSDTM" DataType="integer" Length="8" SASFieldName="TRTSDTM" def:DisplayFormat="DATETIME20.">
        <Description>
          <TranslatedText xml:lang="en">Datetime of First Exposure to Treatment</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.TRTSDTC</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <MethodDef OID="MT.ADSL.TRTSDT" Name="ADSL.TRTSDT" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Date part of TRTSDTM.</TranslatedText>
        </Description>
      </MethodDef>
    </MetaDataVersion>
  </Study>
</ODM>

Shaded columns are derived— = missing value

Human review and assessment

Dataset Name	Variable Name	Variable Label	Type	Codelist/Controlled Terms	Core	Computational Method	Origin
ADSL	STUDYID	Study Identifier	character		Req	Copy value from ADSL_RAW.STUDYID.	Predecessor
ADSL	USUBJID	Unique Subject Identifier	character		Req	Copy value from ADSL_RAW.USUBJID.	Predecessor
ADSL	TRT01A	Actual Treatment for Period 01	character		Cond	Copy value from ADSL_RAW.TRT01A.	Predecessor
ADSL	AGE	Age	numeric		Req	Copy value from ADSL_RAW.AGE.	Predecessor
ADSL	HEIGHTBL	Baseline Height (cm)	numeric		Perm	Copy value from ADSL_RAW.HEIGHTBL.	Predecessor
ADSL	TRTSDTM	Datetime of First Exposure to Treatment	numeric	ISO 8601	Perm	Copy value from ADSL_RAW.TRTSDTC.	Predecessor
ADSL	TRTSDT	Date of First Exposure to Treatment	numeric	ISO 8601	Cond	Date part of TRTSDTM.	Derived

Version	Date	Author	Description
1.7	2026-10-20	yamaa docs build	Read a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.6	2026-10-17	yamaa docs build	Describe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.5	2026-10-17	yamaa docs build	Name collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.4	2026-10-15	yamaa docs build	State datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.3	2026-09-29	yamaa docs build	Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.2	2026-09-29	yamaa docs build	Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.1	2026-09-29	yamaa docs build	Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0		yamaa docs build	Initial generation from spec.yaml

prompts/adam-adsl-xpt/full.md

Edit13 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

TRT01A and AGE are as collected. HEIGHTBL is the collected baseline height, with no value when not measured. TRTSDTM is the date and time of first exposure collected in TRTSDTC, and TRTSDT is its date part; both have no value when the subject was never treated.

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

prompts/adam-adsl-xpt/conventions.md

Edit10 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

TRTSDTM comes from TRTSDTC.

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

prompts/adam-adsl-xpt/brief.md

Edit8 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

R

Edit3 lines

1library(yamaa)23yamaa_project("project.yaml")$spec("ADSL")$run()$output

Python

Edit3 lines

1import yamaa23yamaa.project("project.yaml").spec("ADSL").run().output

solutions/adam-adsl-xpt/result.R

Edit26 lines

1# Reference solution for the yamaa benchmark adam-adsl-xpt (R track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67library(dplyr, warn.conflicts = FALSE)8library(readr)910adsl_raw <- read_csv(11  "/app/input/adsl_raw.csv",12  col_types = cols(.default = col_character())13)1415# A collected time without seconds names second zero.16adsl <- adsl_raw |>17  mutate(18    AGE = as.integer(AGE),19    HEIGHTBL = as.numeric(HEIGHTBL),20    TRTSDTM = ifelse(nchar(TRTSDTC) == 16, paste0(TRTSDTC, ":00"), TRTSDTC),21    TRTSDT = substr(TRTSDTC, 1, 10)22  ) |>23  select(STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM)2425dir.create("/app/output", showWarnings = FALSE)26write_csv(adsl, "/app/output/adsl.csv", na = "")

solutions/adam-adsl-xpt/result.py

Edit23 lines

1# Reference solution for the yamaa benchmark adam-adsl-xpt (Python track).2#3# Written from the benchmark's full prompt and its inputs alone, as an agent4# would write it. Harbor's oracle agent runs it to check that the prompt can5# be solved and that the grader scores a correct answer 1.67from pathlib import Path89import polars as pl1011adsl_raw = pl.read_csv("/app/input/adsl_raw.csv", infer_schema=False)1213# A collected time without seconds names second zero.14moment = pl.col("TRTSDTC").str.to_datetime(strict=False)15adsl = adsl_raw.with_columns(16    AGE=pl.col("AGE").cast(pl.Int64),17    HEIGHTBL=pl.col("HEIGHTBL").cast(pl.Float64),18    TRTSDTM=moment.dt.strftime("%Y-%m-%dT%H:%M:%S"),19    TRTSDT=moment.dt.strftime("%Y-%m-%d"),20).select("STUDYID", "USUBJID", "TRT01A", "AGE", "HEIGHTBL", "TRTSDT", "TRTSDTM")2122Path("/app/output").mkdir(exist_ok=True)23adsl.write_csv("/app/output/adsl.csv")

solutions/adam-adsl-xpt/result.sas

Edit26 lines

1/* opensas reference solution for the yamaa benchmark adam-adsl-xpt.2   Reads the benchmark inputs and derives the requested output.3   Execute with the existing opensas runtime: opensas /app/output/result.sas. */45data adsl_raw;6  length STUDYID USUBJID TRT01A TRTSDTC $1024;7  infile "/app/input/adsl_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767;8  input STUDYID : $char1024. USUBJID : $char1024. TRT01A : $char1024. AGE HEIGHTBL9    TRTSDTC : $char1024.;10run;1112data result;13  set adsl_raw;14  length TRTSDTM $19 TRTSDT $10;15  /* A collected time without seconds names second zero. */16  if TRTSDTC ne '' then do;17    if length(TRTSDTC) = 16 then TRTSDTM = cats(TRTSDTC, ':00');18    else TRTSDTM = TRTSDTC;19    TRTSDT = substr(TRTSDTC, 1, 10);20  end;21run;22proc sql;23  create table final as select STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM24    from result;25quit;26proc export data=final outfile="/app/output/adsl.csv" dbms=csv replace; run;

input/adsl_raw.csv

expected/adsl.csv

expected/define.xml

Define-XML 2.1.11: CATH

ADSL

Your review

Mapping spec

Agent prompt

prompts/adam-adsl-xpt/full.md

prompts/adam-adsl-xpt/conventions.md

prompts/adam-adsl-xpt/brief.md

yamaa code

R

Python

Solution without yamaa

solutions/adam-adsl-xpt/result.R

solutions/adam-adsl-xpt/result.py

solutions/adam-adsl-xpt/result.sas