Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: carry TRT01A, AGE, HEIGHTBL, TRTSDT and TRTSDTM into ADSL and submit it as the SAS transport file adsl.xpt, with the dates as SAS numeric dates.

Input: one pre-analysis row per subject carrying the actual treatment, age, baseline height and the collected date and time of first exposure.

Variables:

  • TRT01A is the actual treatment.
  • AGE is the age in years, shown in three positions.
  • HEIGHTBL is the baseline height in centimetres, shown with one decimal; missing when not measured.
  • TRTSDT is the date part of TRTSDTM; missing when the subject was never treated.
  • TRTSDTM is the date and time of first exposure; missing when the subject was never treated.

Note: in the transport file TRTSDT counts days and TRTSDTM seconds from 1 January 1960, shown as DATE9. and DATETIME20., and the data-definition document describes both as eight-byte integers with those formats. Each text variable is exactly as wide as its longest value.

The project.yaml guide declaration is ADaMIG 1.2; the data-definition document records it and points to the SAS Transport version 5 dataset.

Inputs

Under review
1 source file

input/adsl_raw.csv

Edit
3 rows
input/adsl_raw.csv
STUDYIDUSUBJIDTRT01AAGEHEIGHTBLTRTSDTC
CATHVITAMIN D334172.52025-01-08T08:30
CATHPLACEBO67—2025-01-15T09:05:30
CATHNOT TREATED52165.2—

Expected output

Under review
3 expected rows

expected/adsl.csv

Edit
3 rows
expected/adsl.csv
STUDYIDUSUBJIDTRT01AAGEHEIGHTBLTRTSDTTRTSDTM
CATHVITAMIN D334172.52025-01-082025-01-08T08:30:00
CATHPLACEBO67—2025-01-152025-01-15T09:05:30
CATHNOT TREATED52165.2——

expected/define.xml

Edit
Define-XML

Define-XML 2.1.11: CATH

Example study submitting subject-level analysis data in SAS transport; CATH-01; ADaMIG 1.2

Datasets
DatasetDescriptionClassStructurePurposeKeysDocumentationLocation
ADSLSubject-Level Analysis DatasetSUBJECT LEVEL ANALYSIS DATASETOne record per subjectAnalysisSTUDYID, USUBJIDadsl.xpt

ADSL

ADSL variables
VariableLabel / DescriptionTypeLength or Display FormatControlled Terms or ISO FormatOrigin / Source / Method / Comment
STUDYIDStudy Identifiertext4Predecessor; ADSL_RAW.STUDYID
USUBJIDUnique Subject Identifiertext16Predecessor; ADSL_RAW.USUBJID
TRT01AActual Treatment for Period 01text11Predecessor; ADSL_RAW.TRT01A
AGEAgeinteger3.Predecessor; ADSL_RAW.AGE
HEIGHTBLBaseline Height (cm)float5.1Predecessor; ADSL_RAW.HEIGHTBL
TRTSDTDate of First Exposure to TreatmentintegerDATE9.Derived; Sponsor Date part of TRTSDTM.
TRTSDTMDatetime of First Exposure to TreatmentintegerDATETIME20.Predecessor; ADSL_RAW.TRTSDTC
Original source
<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.adam.Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study submitting subject-level analysis data in SAS transport</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.adam" Name="CATH adam metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.ADaMIG" Name="ADaMIG" Type="IG" Version="1.2" Status="Final"/>
      </def:Standards>
      <ItemGroupDef OID="IG.ADSL" Name="ADSL" Purpose="Analysis" SASDatasetName="ADSL" Repeating="No" IsReferenceData="No" def:Structure="One record per subject" def:ArchiveLocationID="LF.ADSL" def:StandardOID="STD.ADaMIG">
        <Description>
          <TranslatedText xml:lang="en">Subject-Level Analysis Dataset</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.ADSL.STUDYID" OrderNumber="1" Mandatory="Yes" KeySequence="1"/>
        <ItemRef ItemOID="IT.ADSL.USUBJID" OrderNumber="2" Mandatory="Yes" KeySequence="2"/>
        <ItemRef ItemOID="IT.ADSL.TRT01A" OrderNumber="3" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.ADSL.AGE" OrderNumber="4" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.ADSL.HEIGHTBL" OrderNumber="5" Mandatory="No"/>
        <ItemRef ItemOID="IT.ADSL.TRTSDT" OrderNumber="6" Mandatory="No" MethodOID="MT.ADSL.TRTSDT"/>
        <ItemRef ItemOID="IT.ADSL.TRTSDTM" OrderNumber="7" Mandatory="No"/>
        <def:Class Name="SUBJECT LEVEL ANALYSIS DATASET"/>
        <def:leaf ID="LF.ADSL" xlink:href="adsl.xpt">
          <def:title>adsl.xpt</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.ADSL.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.STUDYID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.USUBJID" Name="USUBJID" DataType="text" Length="16" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.USUBJID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRT01A" Name="TRT01A" DataType="text" Length="11" SASFieldName="TRT01A">
        <Description>
          <TranslatedText xml:lang="en">Actual Treatment for Period 01</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.TRT01A</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.AGE" Name="AGE" DataType="integer" Length="3" SASFieldName="AGE" def:DisplayFormat="3.">
        <Description>
          <TranslatedText xml:lang="en">Age</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.AGE</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.HEIGHTBL" Name="HEIGHTBL" DataType="float" Length="5" SignificantDigits="1" SASFieldName="HEIGHTBL" def:DisplayFormat="5.1">
        <Description>
          <TranslatedText xml:lang="en">Baseline Height (cm)</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.HEIGHTBL</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRTSDT" Name="TRTSDT" DataType="integer" Length="8" SASFieldName="TRTSDT" def:DisplayFormat="DATE9.">
        <Description>
          <TranslatedText xml:lang="en">Date of First Exposure to Treatment</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.ADSL.TRTSDTM" Name="TRTSDTM" DataType="integer" Length="8" SASFieldName="TRTSDTM" def:DisplayFormat="DATETIME20.">
        <Description>
          <TranslatedText xml:lang="en">Datetime of First Exposure to Treatment</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">ADSL_RAW.TRTSDTC</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <MethodDef OID="MT.ADSL.TRTSDT" Name="ADSL.TRTSDT" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Date part of TRTSDTM.</TranslatedText>
        </Description>
      </MethodDef>
    </MetaDataVersion>
  </Study>
</ODM>
Shaded columns are derived— = missing value

Human review and assessment

Your review

3 questions and your comments · View benchmark discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADSLSTUDYIDStudy IdentifiercharacterReqCopy value from ADSL_RAW.STUDYID.Predecessor
ADSLUSUBJIDUnique Subject IdentifiercharacterReqCopy value from ADSL_RAW.USUBJID.Predecessor
ADSLTRT01AActual Treatment for Period 01characterCondCopy value from ADSL_RAW.TRT01A.Predecessor
ADSLAGEAgenumericReqCopy value from ADSL_RAW.AGE.Predecessor
ADSLHEIGHTBLBaseline Height (cm)numericPermCopy value from ADSL_RAW.HEIGHTBL.Predecessor
ADSLTRTSDTMDatetime of First Exposure to TreatmentnumericISO 8601PermCopy value from ADSL_RAW.TRTSDTC.Predecessor
ADSLTRTSDTDate of First Exposure to TreatmentnumericISO 8601CondDate part of TRTSDTM.Derived
VersionDateAuthorDescriptionReviewerSign-off
1.72026-10-20yamaa docs buildRead a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.62026-10-17yamaa docs buildDescribe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.52026-10-17yamaa docs buildName collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.42026-10-15yamaa docs buildState datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-adsl-adamig12/full.md

Edit
13 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

Use ADaMIG 1.2. TRTSDTM is the first-exposure date and time in ADSL_RAW.TRTSDTC, shown as "YYYY-MM-DDThh:mm:ss", with seconds "00" when the source omits seconds. TRTSDT is the calendar date part of TRTSDTM. TRTSDT and TRTSDTM have no value when the subject was never treated.

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

prompts/adam-adsl-adamig12/conventions.md

Edit
11 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

Use ADaMIG 1.2. TRTSDTM is shown as "YYYY-MM-DDThh:mm:ss". TRTSDT and TRTSDTM can have no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

prompts/adam-adsl-adamig12/brief.md

Edit
8 lines

Following CDISC ADaM standards, use the provided ADSL_RAW dataset to create an ADSL dataset with one record per subject.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, TRT01A, AGE, HEIGHTBL, TRTSDT, TRTSDTM

Read the source datasets from /app/input and save the completed dataset as /app/output/adsl.csv.

yamaa code

Public calls generated from benchmark.yaml; results must match the expected output

R

Edit
3 lines
library(yamaa)yamaa_project("project.yaml")$spec("ADSL")$run()$output

Python

Edit
3 lines
import yamaayamaa.project("project.yaml").spec("ADSL").run().output

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-adsl-adamig12/result.R

Edit
38 lines
# Derive ADSL from ADSL_RAW using the requested ADaMIG 1.2 variables.raw <- read.csv("/app/input/adsl_raw.csv", stringsAsFactors = FALSE,                check.names = FALSE, na.strings = "")required <- c("STUDYID", "USUBJID", "TRT01A", "AGE", "HEIGHTBL", "TRTSDTC")stopifnot(all(required %in% names(raw)))stopifnot(!anyNA(raw$USUBJID), !anyDuplicated(raw$USUBJID))source_datetime <- trimws(as.character(raw$TRTSDTC))never_treated <- !is.na(raw$TRT01A) &  toupper(trimws(raw$TRT01A)) == "NOT TREATED"has_exposure <- !never_treated & !is.na(source_datetime) &  nzchar(source_datetime)trtsdtm <- rep(NA_character_, nrow(raw))if (any(has_exposure)) {  values <- source_datetime[has_exposure]  valid_shape <- grepl(    "^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}(:[0-9]{2})?$",    values  )  if (!all(valid_shape)) stop("Exposure datetime has an unsupported format")  values[nchar(values) == 16L] <- paste0(values[nchar(values) == 16L], ":00")  parsed <- as.POSIXct(values, format = "%Y-%m-%dT%H:%M:%S", tz = "UTC")  normalized <- format(parsed, "%Y-%m-%dT%H:%M:%S", tz = "UTC")  if (anyNA(parsed) || !all(normalized == values)) {    stop("Invalid exposure date or time")  }  trtsdtm[has_exposure] <- normalized}adsl <- raw[c("STUDYID", "USUBJID", "TRT01A", "AGE", "HEIGHTBL")]adsl$TRTSDT <- substr(trtsdtm, 1L, 10L)adsl$TRTSDTM <- trtsdtmstopifnot(nrow(adsl) == nrow(raw), !anyDuplicated(adsl$USUBJID))stopifnot(all(is.na(adsl$TRTSDT[never_treated])),          all(is.na(adsl$TRTSDTM[never_treated])))dir.create("/app/output", showWarnings = FALSE)write.csv(adsl, "/app/output/adsl.csv", row.names = FALSE, na = "")

solutions/adam-adsl-adamig12/result.py

Edit
44 lines
"""Create the requested ADaMIG 1.2 ADSL extract from ADSL_RAW."""import csvfrom datetime import datetimefrom pathlib import PathINPUT = Path("/app/input/adsl_raw.csv")OUTPUT = Path("/app/output/adsl.csv")COLUMNS = ["STUDYID", "USUBJID", "TRT01A", "AGE", "HEIGHTBL", "TRTSDT", "TRTSDTM"]def main():    records = []    subjects = set()    with INPUT.open(newline="", encoding="utf-8-sig") as source:        for raw in csv.DictReader(source):            subject = (raw["STUDYID"], raw["USUBJID"])            if subject in subjects:                raise ValueError(                    f"ADSL_RAW contains multiple records for subject {subject!r}"                )            subjects.add(subject)            record = {column: raw[column] for column in COLUMNS[:5]}            treatment_start = raw["TRTSDTC"].strip()            if treatment_start:                timestamp = datetime.fromisoformat(treatment_start)                record["TRTSDTM"] = timestamp.strftime("%Y-%m-%dT%H:%M:%S")                record["TRTSDT"] = timestamp.date().isoformat()            else:                record["TRTSDT"] = ""                record["TRTSDTM"] = ""            records.append(record)    OUTPUT.parent.mkdir(parents=True, exist_ok=True)    with OUTPUT.open("w", newline="", encoding="utf-8") as destination:        writer = csv.DictWriter(destination, fieldnames=COLUMNS, lineterminator="\n")        writer.writeheader()        writer.writerows(records)if __name__ == "__main__":    main()

solutions/adam-adsl-adamig12/result.sas

Edit
41 lines
/* Derive numeric ADaM treatment date and datetime values. */proc import datafile='/app/input/adsl_raw.csv' out=adsl_raw dbms=csv replace;    getnames=yes;run;data adsl;    retain STUDYID USUBJID TRT01A AGE HEIGHTBL TRTSDT TRTSDTM;    set adsl_raw;    length _trtsdtc $19;    format TRTSDT yymmdd10. TRTSDTM e8601dt19.;    if not missing(TRTSDTC) then do;        _trtsdtc=strip(TRTSDTC);        if lengthn(_trtsdtc)=16 then _trtsdtc=cats(_trtsdtc,':00');        TRTSDTM=input(_trtsdtc,e8601dt.);        TRTSDT=datepart(TRTSDTM);    end;    else do;        TRTSDT=.;        TRTSDTM=.;    end;    keep STUDYID USUBJID TRT01A AGE HEIGHTBL TRTSDT TRTSDTM;run;/* CSV export requires explicit formatted character representations. */data adsl_csv;    retain STUDYID USUBJID TRT01A AGE HEIGHTBL TRTSDT TRTSDTM;    length STUDYID $200 USUBJID $200 TRT01A $200 AGE 8 HEIGHTBL 8 TRTSDT $10 TRTSDTM $19;    set adsl(rename=(TRTSDT=_date TRTSDTM=_datetime));    if not missing(_datetime) then do;        TRTSDT=put(_date,yymmdd10.);        TRTSDTM=put(_datetime,e8601dt19.);    end;    else do;        TRTSDT='';        TRTSDTM='';    end;    keep STUDYID USUBJID TRT01A AGE HEIGHTBL TRTSDT TRTSDTM;run;proc export data=adsl_csv outfile='/app/output/adsl.csv' dbms=csv replace;run;