Review ask

As a clinical trial statistician or programmer, answer three short questions in Your review: the task, the inputs and outputs, and the AI-assisted review. Agreeing with all three approves this benchmark; disagreeing with any needs a comment. Propose a fix with the Edit button on any source file, or add it to your comments.

We suggest spending no more than 5 minutes on a review. The other tabs beside Your review are background for assessment, not under review.

Summary

Lifecycle: draftEditUnder review
What this benchmark means
README.md

Public entries: Python and R. Passing coverage requires an installed inventory.

Goal: derive each adverse event's analysis start date and time (ASTDTM), its date (ASTDT), its time of day (ASTTM) and the time imputation flag (ASTTMF), and submit them in a SAS Transport version 5 file with the data-definition document.

Input: SDTM adverse event (AE) records with the collected start (AESTDTC) as ISO 8601 text, collected to the second, the minute, the hour, the day or the month.

Variables:

  • AETERM is the adverse event as reported.
  • ASTDTM is the start date and time: a start collected to the hour or the minute takes zero for the rest of its time, and a start collected as a date alone takes midnight; missing when the start was not collected to the day.
  • ASTDT is the date part of ASTDTM; missing when ASTDTM is.
  • ASTTM is the time of day of ASTDTM counted in seconds since midnight, so 08:30:15 is 30615, and it is shown as hh:mm:ss; missing when ASTDTM is.
  • ASTTMF says which part of the start time was imputed: H when the start was collected without a time, M when it stops at the hour and S when it stops at the minute; blank when the time was collected to the second or there is no start.

Note: the SAS Transport file stores ASTTM as an eight-byte numeric value with the SAS format TIME8., beside ASTDT with DATE9. and ASTDTM with DATETIME20.. The data-definition document records these formats. A time of zero flagged H is an imputed midnight rather than a collected one.

The project.yaml guide declaration is ADaMIG 1.1; the data-definition document records it and points to the SAS Transport version 5 dataset.

Inputs

Under review
1 source file

input/ae.csv

Edit
5 rows
input/ae.csv
STUDYIDUSUBJIDAESEQAETERMAESTDTC
CATH1HEADACHE2025-03-04T08:30:15
CATH2NAUSEA2025-03-06T14:05
CATH1RASH2025-03-10
CATH2DIZZINESS2025-03
CATH1FATIGUE2025-03-12T21

Expected output

Under review
5 expected rows

expected/adae.csv

Edit
5 rows
expected/adae.csv
STUDYIDUSUBJIDAESEQAETERMASTDTMASTDTASTTMASTTMF
CATH1HEADACHE2025-03-04T08:30:152025-03-0430615—
CATH2NAUSEA2025-03-06T14:05:002025-03-0650700S
CATH1RASH2025-03-10T00:00:002025-03-100H
CATH2DIZZINESS————
CATH1FATIGUE2025-03-12T21:00:002025-03-1275600M

expected/define.xml

Edit
Define-XML

Define-XML 2.1.11: CATH

Example study submitting adverse event onset times in SAS transport; CATH-01; ADaMIG 1.1

Datasets
DatasetDescriptionClassStructurePurposeKeysDocumentationLocation
ADAEAdverse Events Analysis DatasetOCCURRENCE DATA STRUCTUREOne record per adverse eventAnalysisSTUDYID, USUBJID, AESEQadae.xpt

ADAE

ADAE variables
VariableLabel / DescriptionTypeLength or Display FormatControlled Terms or ISO FormatOrigin / Source / Method / Comment
STUDYIDStudy Identifiertext4Predecessor; AE.STUDYID
USUBJIDUnique Subject Identifiertext16Predecessor; AE.USUBJID
AESEQSequence Numberinteger8Predecessor; AE.AESEQ
AETERMReported Term for the Adverse Eventtext9Predecessor; AE.AETERM
ASTDTMAnalysis Start Date/TimeintegerDATETIME20.Derived; Sponsor AESTDTC completed with the earliest time its collected parts admit; missing when AESTDTC carries no complete date.
ASTDTAnalysis Start DateintegerDATE9.Derived; Sponsor Date part of ASTDTM.
ASTTMAnalysis Start TimeintegerTIME8.Derived; Sponsor Time part of ASTDTM as seconds since midnight.
ASTTMFAnalysis Start Time Imputation Flagtext1Derived; Sponsor H when the start was collected without a time, M when it stops at the hour and S when it stops at the minute.
Original source
<?xml version="1.0" encoding="UTF-8"?>
<ODM xmlns="http://www.cdisc.org/ns/odm/v1.3" xmlns:def="http://www.cdisc.org/ns/def/v2.1" xmlns:xlink="http://www.w3.org/1999/xlink" ODMVersion="1.3.2" FileType="Snapshot" FileOID="CATH.adam.Define-XML_2.1" CreationDateTime="2026-09-11T00:00:00" SourceSystem="yamaa" SourceSystemVersion="1.0" def:Context="Submission">
  <Study OID="STDY.CATH">
    <GlobalVariables>
      <StudyName>CATH</StudyName>
      <StudyDescription>Example study submitting adverse event onset times in SAS transport</StudyDescription>
      <ProtocolName>CATH-01</ProtocolName>
    </GlobalVariables>
    <MetaDataVersion OID="MDV.CATH.adam" Name="CATH adam metadata" def:DefineVersion="2.1.11">
      <def:Standards>
        <def:Standard OID="STD.ADaMIG" Name="ADaMIG" Type="IG" Version="1.1" Status="Final"/>
      </def:Standards>
      <ItemGroupDef OID="IG.ADAE" Name="ADAE" Purpose="Analysis" SASDatasetName="ADAE" Repeating="Yes" IsReferenceData="No" def:Structure="One record per adverse event" def:ArchiveLocationID="LF.ADAE" def:StandardOID="STD.ADaMIG">
        <Description>
          <TranslatedText xml:lang="en">Adverse Events Analysis Dataset</TranslatedText>
        </Description>
        <ItemRef ItemOID="IT.ADAE.STUDYID" OrderNumber="1" Mandatory="Yes" KeySequence="1"/>
        <ItemRef ItemOID="IT.ADAE.USUBJID" OrderNumber="2" Mandatory="Yes" KeySequence="2"/>
        <ItemRef ItemOID="IT.ADAE.AESEQ" OrderNumber="3" Mandatory="Yes" KeySequence="3"/>
        <ItemRef ItemOID="IT.ADAE.AETERM" OrderNumber="4" Mandatory="Yes"/>
        <ItemRef ItemOID="IT.ADAE.ASTDTM" OrderNumber="5" Mandatory="No" MethodOID="MT.ADAE.ASTDTM"/>
        <ItemRef ItemOID="IT.ADAE.ASTDT" OrderNumber="6" Mandatory="No" MethodOID="MT.ADAE.ASTDT"/>
        <ItemRef ItemOID="IT.ADAE.ASTTM" OrderNumber="7" Mandatory="No" MethodOID="MT.ADAE.ASTTM"/>
        <ItemRef ItemOID="IT.ADAE.ASTTMF" OrderNumber="8" Mandatory="No" MethodOID="MT.ADAE.ASTTMF"/>
        <def:Class Name="OCCURRENCE DATA STRUCTURE"/>
        <def:leaf ID="LF.ADAE" xlink:href="adae.xpt">
          <def:title>adae.xpt</def:title>
        </def:leaf>
      </ItemGroupDef>
      <ItemDef OID="IT.ADAE.STUDYID" Name="STUDYID" DataType="text" Length="4" SASFieldName="STUDYID">
        <Description>
          <TranslatedText xml:lang="en">Study Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">AE.STUDYID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADAE.USUBJID" Name="USUBJID" DataType="text" Length="16" SASFieldName="USUBJID">
        <Description>
          <TranslatedText xml:lang="en">Unique Subject Identifier</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">AE.USUBJID</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADAE.AESEQ" Name="AESEQ" DataType="integer" Length="8" SASFieldName="AESEQ">
        <Description>
          <TranslatedText xml:lang="en">Sequence Number</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">AE.AESEQ</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADAE.AETERM" Name="AETERM" DataType="text" Length="9" SASFieldName="AETERM">
        <Description>
          <TranslatedText xml:lang="en">Reported Term for the Adverse Event</TranslatedText>
        </Description>
        <def:Origin Type="Predecessor">
          <Description>
            <TranslatedText xml:lang="en">AE.AETERM</TranslatedText>
          </Description>
        </def:Origin>
      </ItemDef>
      <ItemDef OID="IT.ADAE.ASTDTM" Name="ASTDTM" DataType="integer" Length="8" SASFieldName="ASTDTM" def:DisplayFormat="DATETIME20.">
        <Description>
          <TranslatedText xml:lang="en">Analysis Start Date/Time</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.ADAE.ASTDT" Name="ASTDT" DataType="integer" Length="8" SASFieldName="ASTDT" def:DisplayFormat="DATE9.">
        <Description>
          <TranslatedText xml:lang="en">Analysis Start Date</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.ADAE.ASTTM" Name="ASTTM" DataType="integer" Length="8" SASFieldName="ASTTM" def:DisplayFormat="TIME8.">
        <Description>
          <TranslatedText xml:lang="en">Analysis Start Time</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <ItemDef OID="IT.ADAE.ASTTMF" Name="ASTTMF" DataType="text" Length="1" SASFieldName="ASTTMF">
        <Description>
          <TranslatedText xml:lang="en">Analysis Start Time Imputation Flag</TranslatedText>
        </Description>
        <def:Origin Type="Derived" Source="Sponsor"/>
      </ItemDef>
      <MethodDef OID="MT.ADAE.ASTDTM" Name="ADAE.ASTDTM" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">AESTDTC completed with the earliest time its collected parts admit; missing when AESTDTC carries no complete date.</TranslatedText>
        </Description>
      </MethodDef>
      <MethodDef OID="MT.ADAE.ASTDT" Name="ADAE.ASTDT" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Date part of ASTDTM.</TranslatedText>
        </Description>
      </MethodDef>
      <MethodDef OID="MT.ADAE.ASTTM" Name="ADAE.ASTTM" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">Time part of ASTDTM as seconds since midnight.</TranslatedText>
        </Description>
      </MethodDef>
      <MethodDef OID="MT.ADAE.ASTTMF" Name="ADAE.ASTTMF" Type="Computation">
        <Description>
          <TranslatedText xml:lang="en">H when the start was collected without a time, M when it stops at the hour and S when it stops at the minute.</TranslatedText>
        </Description>
      </MethodDef>
    </MetaDataVersion>
  </Study>
</ODM>
Shaded columns are derived— = missing value

Human review and assessment

Your review

3 questions and your comments · View benchmark discussions

Mapping spec

Generated from the benchmark YAML; do not edit by hand

Dataset NameVariable NameVariable LabelTypeCodelist/Controlled TermsCoreComputational MethodOrigin
ADAESTUDYIDStudy IdentifiercharacterReqCopy value from AE.STUDYID.Predecessor
ADAEUSUBJIDUnique Subject IdentifiercharacterReqCopy value from AE.USUBJID.Predecessor
ADAEAESEQSequence NumbernumericReqCopy value from AE.AESEQ.Predecessor
ADAEAETERMReported Term for the Adverse EventcharacterReqCopy value from AE.AETERM.Predecessor
ADAEASTDTMAnalysis Start Date/TimenumericISO 8601PermAESTDTC completed with the earliest time its collected parts admit; missing when AESTDTC carries no complete date.Derived
ADAEASTDTAnalysis Start DatenumericISO 8601CondDate part of ASTDTM.Derived
ADAEASTTMAnalysis Start TimenumericPermTime part of ASTDTM as seconds since midnight.Derived
ADAEASTTMFAnalysis Start Time Imputation FlagcharacterH, M, SCondH when the start was collected without a time, M when it stops at the hour and S when it stops at the minute.Derived
VariablePermitted values
ASTTMFH, M, S
VersionDateAuthorDescriptionReviewerSign-off
1.72026-10-20yamaa docs buildRead a specification in the 1.0 surface as its 0.5 counterpart; state that missing values stay missing where a mapping declares no missing handler, and leave out a declared missing answer that restates a handler's default
1.62026-10-17yamaa docs buildDescribe collected date parts written as ISO 8601 date text and times of day counted in seconds since midnight
1.52026-10-17yamaa docs buildName collected precision in words instead of letter codes; state imputation fallbacks, minimum source precision, bounds ignored outside the collected period, and invalid text with no handler as an error; describe date and datetime shifts, imputation flag codes, and partial text giving no date part
1.42026-10-15yamaa docs buildState datetime precision codes through the hour and minute, and complete only the time parts a collected datetime lacks
1.32026-09-29yamaa docs buildShow study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata
1.22026-09-29yamaa docs buildClassify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived
1.12026-09-29yamaa docs buildResolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML
1.0yamaa docs buildInitial generation from spec.yaml

Agent prompt

The request an agent receives with the input files; each tier states no more than the one above

prompts/adam-adae-adamig11/full.md

Edit
22 lines

Following CDISC ADaM standards, use the provided AE dataset to create an ADAE dataset with one record per adverse event.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, ASTDTM, ASTDT, ASTTM, ASTTMF

ASTDTM is the start date and time from AESTDTC: a start collected to the second has its collected value, a start collected to the minute has seconds 00, a start collected to the hour has minutes and seconds 00, and a complete date without a time has time 00:00:00. ASTDTM has no value when AESTDTC lacks a complete calendar date.

ASTDT is the calendar date of ASTDTM and ASTTM is its time of day as whole seconds since midnight; both have no value when ASTDTM has no value.

ASTTMF is "H" for a start collected as a complete date without a time, "M" for a start collected to the hour, and "S" for a start collected to the minute. ASTTMF has no value when the start was collected to the second or ASTDTM has no value. An ASTTM of 0 with ASTTMF "H" is imputed midnight.

Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.

prompts/adam-adae-adamig11/conventions.md

Edit
10 lines

Following CDISC ADaM standards, use the provided AE dataset to create an ADAE dataset with one record per adverse event.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, ASTDTM, ASTDT, ASTTM, ASTTMF

ASTTMF is "H", "M", or "S", or has no value.

Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.

prompts/adam-adae-adamig11/brief.md

Edit
8 lines

Following CDISC ADaM standards, use the provided AE dataset to create an ADAE dataset with one record per adverse event.

The output dataset should contain the following columns in this order: STUDYID, USUBJID, AESEQ, AETERM, ASTDTM, ASTDT, ASTTM, ASTTMF

Read the source datasets from /app/input and save the completed dataset as /app/output/adae.csv.

yamaa code

Public calls generated from benchmark.yaml; results must match the expected output

R

Edit
3 lines
library(yamaa)yamaa_project("project.yaml")$spec("ADAE")$run()$output

Python

Edit
3 lines
import yamaayamaa.project("project.yaml").spec("ADAE").run().output

Solution without yamaa

Written from the full prompt and the inputs alone; CI checks it reproduces the expected output

solutions/adam-adae-adamig11/result.R

Edit
38 lines
# Derive ADaM adverse-event start date/time and time-imputation flags.ae <- read.csv("/app/input/ae.csv", stringsAsFactors = FALSE, check.names = FALSE,               na.strings = c("", "NA"))required <- c("STUDYID", "USUBJID", "AESEQ", "AETERM", "AESTDTC")stopifnot(all(required %in% names(ae)))n <- nrow(ae)adae <- ae[c("STUDYID", "USUBJID", "AESEQ", "AETERM")]adae$ASTDTM <- rep(NA_character_, n)adae$ASTDT <- rep(NA_character_, n)adae$ASTTM <- rep(NA_integer_, n)adae$ASTTMF <- rep(NA_character_, n)for (i in seq_len(n)) {  dtc <- ae$AESTDTC[i]  if (is.na(dtc)) next  # Each accepted form must contain a complete calendar date.  if (!grepl("^[0-9]{4}-[0-9]{2}-[0-9]{2}(T[0-9]{2}(:[0-9]{2}(:[0-9]{2})?)?)?$", dtc)) next  date_text <- substr(dtc, 1L, 10L)  parsed_date <- as.Date(date_text, format = "%Y-%m-%d")  if (is.na(parsed_date) || format(parsed_date, "%Y-%m-%d") != date_text) next  precision <- nchar(dtc)  hour <- if (precision >= 13L) as.integer(substr(dtc, 12L, 13L)) else 0L  minute <- if (precision >= 16L) as.integer(substr(dtc, 15L, 16L)) else 0L  second <- if (precision >= 19L) as.integer(substr(dtc, 18L, 19L)) else 0L  if (hour > 23L || minute > 59L || second > 59L) next  adae$ASTDTM[i] <- sprintf("%sT%02d:%02d:%02d", date_text, hour, minute, second)  adae$ASTDT[i] <- date_text  adae$ASTTM[i] <- hour * 3600L + minute * 60L + second  adae$ASTTMF[i] <- switch(as.character(precision), `10` = "H", `13` = "M",                         `16` = "S", NA_character_)}stopifnot(nrow(adae) == nrow(ae),          identical(names(adae), c("STUDYID", "USUBJID", "AESEQ", "AETERM",                                   "ASTDTM", "ASTDT", "ASTTM", "ASTTMF")))dir.create("/app/output", showWarnings = FALSE)write.csv(adae, "/app/output/adae.csv", row.names = FALSE, na = "")

solutions/adam-adae-adamig11/result.py

Edit
74 lines
"""Create the event-level ADAE start datetime analysis variables."""import csvimport refrom datetime import datetimefrom pathlib import PathCOLUMNS = [    "STUDYID",    "USUBJID",    "AESEQ",    "AETERM",    "ASTDTM",    "ASTDT",    "ASTTM",    "ASTTMF",]START_PATTERN = re.compile(    r"(\d{4})-(\d{2})-(\d{2})(?:T(\d{2})(?::(\d{2})(?::(\d{2}))?)?)?")def derive_start(value):    """Impute only uncollected time components of a complete calendar date."""    match = START_PATTERN.fullmatch((value or "").strip())    if match is None:        return {column: "" for column in COLUMNS[4:]}    year, month, day, hour, minute, second = match.groups()    try:        start = datetime(            int(year),            int(month),            int(day),            int(hour or 0),            int(minute or 0),            int(second or 0),        )    except ValueError:        return {column: "" for column in COLUMNS[4:]}    flag = (        "H"        if hour is None        else "M"        if minute is None        else "S"        if second is None        else ""    )    return {        "ASTDTM": start.isoformat(timespec="seconds"),        "ASTDT": start.date().isoformat(),        "ASTTM": start.hour * 3600 + start.minute * 60 + start.second,        "ASTTMF": flag,    }def main():    output_path = Path("/app/output/adae.csv")    output_path.parent.mkdir(parents=True, exist_ok=True)    with Path("/app/input/ae.csv").open(newline="", encoding="utf-8-sig") as source:        with output_path.open("w", newline="", encoding="utf-8") as target:            writer = csv.DictWriter(target, fieldnames=COLUMNS)            writer.writeheader()            for event in csv.DictReader(source):                row = {column: event.get(column, "") for column in COLUMNS[:4]}                row.update(derive_start(event.get("AESTDTC")))                writer.writerow(row)if __name__ == "__main__":    main()

solutions/adam-adae-adamig11/result.sas

Edit
40 lines
proc import datafile="/app/input/ae.csv" out=ae dbms=csv replace;  guessingrows=max;run;data adae;  length STUDYID $40 USUBJID $80 AESEQ 8 AETERM $200 ASTDTM $19 ASTDT $10 ASTTM 8 ASTTMF $1;  set ae;  ASTDTM = '';  ASTDT = '';  ASTTM = .;  ASTTMF = '';  _length = length(strip(AESTDTC));  if _length >= 10 then do;    _date = input(substr(AESTDTC,1,10),yymmdd10.);    if not missing(_date) then do;      _hour = 0;      _minute = 0;      _second = 0;      if _length = 10 then ASTTMF = 'H';      else if _length >= 13 then do;        _hour = input(substr(AESTDTC,12,2),2.);        if _length = 13 then ASTTMF = 'M';        else if _length >= 16 then do;          _minute = input(substr(AESTDTC,15,2),2.);          if _length = 16 then ASTTMF = 'S';          else if _length >= 19 then            _second = input(substr(AESTDTC,18,2),2.);        end;      end;      _datetime = dhms(_date,_hour,_minute,_second);      ASTDTM = put(_datetime,e8601dt19.);      ASTDT = put(datepart(_datetime),yymmdd10.);      ASTTM = timepart(_datetime);    end;  end;  keep STUDYID USUBJID AESEQ AETERM ASTDTM ASTDT ASTTM ASTTMF;run;proc export data=adae outfile="/app/output/adae.csv" dbms=csv replace;run;