As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC SDTM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
README.mdGoal: build one output record per collected disposition
record, carrying DSDECOD, DSCAT and DSDTC, with DSSEQ
numbering each subject's records from the earliest collection
date.
Input: one row per collected form, with the subject in the subject field, the recorded outcome in the outcome field, and the collection date text in the date field. The form label does not feed any output column.
Variables:
DSSEQnumbers the subject's records from the earliest completed collection date. A partial entry such as2024-01takes its position at the imputed 15th, after an earlier full date in that month. A record with no date is numbered after dated records.DSDECODcopies the recorded outcome:COMPLETED,RANDOMIZED,ADVERSE EVENTorSCREEN FAILURE.DSCATisPROTOCOL MILESTONEwhen the outcome isRANDOMIZED, andDISPOSITION EVENTfor any other outcome.DSDTCis the collection date; a partial entry such as2024-01is completed to the 15th of that month, and a year alone is completed to June 15th. It is empty when the date is missing or cannot be read.
Inputs
Under review1 source fileinput/ds_raw.csv
Edit7 rows| STUDY | PATNUM | INSTANCE | DSDECOD | DSDTCOL |
|---|---|---|---|---|
| KN189 | KN189-101 | COMPLETION | COMPLETED | 2024-01-10 |
| KN189 | KN189-101 | PROTOCOL MILESTONE | RANDOMIZED | 2023-12-20 |
| KN189 | KN189-102 | COMPLETION | ADVERSE EVENT | 2024-02-15 |
| KN189 | KN189-103 | COMPLETION | COMPLETED | 2024-03-01 |
| KN189 | KN189-104 | PROTOCOL MILESTONE | RANDOMIZED | 2024-01-05 |
| KN189 | KN189-105 | SCREENING | SCREEN FAILURE | 2024-01 |
| KN189 | KN189-105 | PROTOCOL MILESTONE | RANDOMIZED | 2024-01-05 |
Expected output
Under review7 expected rowsexpected/ds.csv
Edit7 rows| DOMAIN | STUDYID | USUBJID | DSSEQ | DSDECOD | DSCAT | DSDTC |
|---|---|---|---|---|---|---|
| DS | KN189 | 2 | COMPLETED | DISPOSITION EVENT | 2024-01-10 | |
| DS | KN189 | 1 | RANDOMIZED | PROTOCOL MILESTONE | 2023-12-20 | |
| DS | KN189 | 1 | ADVERSE EVENT | DISPOSITION EVENT | 2024-02-15 | |
| DS | KN189 | 1 | COMPLETED | DISPOSITION EVENT | 2024-03-01 | |
| DS | KN189 | 1 | RANDOMIZED | PROTOCOL MILESTONE | 2024-01-05 | |
| DS | KN189 | 2 | SCREEN FAILURE | DISPOSITION EVENT | 2024-01-15 | |
| DS | KN189 | 1 | RANDOMIZED | PROTOCOL MILESTONE | 2024-01-05 |
Comments and assessment materials
| Variable Name | Variable Label | Type | Length | Controlled Terms or Format | Origin | Core | Conversion Definition | Variable Type | Variable Order | Comments for Define |
|---|---|---|---|---|---|---|---|---|---|---|
| DOMAIN | Domain Abbreviation | Char | DS | Assigned | Set constant value "DS". | SDTM | 1 | |||
| STUDYID | Study Identifier | Char | Collected | Copy value from DS_RAW.STUDY. | SDTM | 2 | ||||
| USUBJID | Unique Subject Identifier | Char | Collected | Copy value from DS_RAW.PATNUM. | SDTM | 3 | ||||
| DSDTC | Date/Time of Collection | Char | ISO 8601 | Derived | Impute incomplete date from DS_RAW.DSDTCOL, defaulting month=6, day=15; a missing source → blank; invalid text → blank. | SDTM | 4 | |||
| DSSEQ | Sequence Number | Num | Derived | Row number within each group of STUDYID, USUBJID, ordered by DSDTC. | SDTM | 5 | ||||
| DSDECOD | Standardized Disposition Term | Char | COMPLETED, RANDOMIZED, ADVERSE EVENT, SCREEN FAILURE | Collected | Copy value from DS_RAW.DSDECOD. | SDTM | 6 | |||
| DSCAT | Category for Disposition Event | Char | PROTOCOL MILESTONE, DISPOSITION EVENT | Derived | If DSDECOD = 'RANDOMIZED' then "PROTOCOL MILESTONE"; otherwise "DISPOSITION EVENT". | SDTM | 7 |
| Template | Include when |
|---|---|
| disposition |
| Variable | Permitted values |
|---|---|
| DOMAIN | DS |
| DSDECOD | COMPLETED, RANDOMIZED, ADVERSE EVENT, SCREEN FAILURE |
| DSCAT | PROTOCOL MILESTONE, DISPOSITION EVENT |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
prompts/sdtm-ds-sequence/full.md
Edit19 linesFollowing CDISC SDTM standards, use the provided DS_RAW dataset to create a DS dataset with one record per collected disposition record.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DSSEQ, DSDECOD, DSCAT, DSDTC
USUBJID is the collected subject number. DSDECOD is the recorded outcome: COMPLETED, RANDOMIZED, ADVERSE EVENT, or SCREEN FAILURE. DSCAT is PROTOCOL MILESTONE when the outcome is RANDOMIZED, and DISPOSITION EVENT for any other outcome. DSDTC is the collection date; a partial entry such as a year and month is completed to the 15th of that month, and a year alone is completed to June 15th. It has no value when the date is missing or cannot be read. DSSEQ numbers the subject's records from the earliest completed collection date, so a partial entry takes its position at its completed date, and a record with no date is numbered after dated records. The form label feeds no output column.
Read the source datasets from /app/input and save the completed dataset as /app/output/ds.csv.
prompts/sdtm-ds-sequence/conventions.md
Edit12 linesFollowing CDISC SDTM standards, use the provided DS_RAW dataset to create a DS dataset with one record per collected disposition record.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DSSEQ, DSDECOD, DSCAT, DSDTC
DSDECOD is COMPLETED, RANDOMIZED, ADVERSE EVENT, or SCREEN FAILURE. DSCAT is PROTOCOL MILESTONE when the outcome is RANDOMIZED, and DISPOSITION EVENT for any other outcome.
Read the source datasets from /app/input and save the completed dataset as /app/output/ds.csv.
prompts/sdtm-ds-sequence/brief.md
Edit8 linesFollowing CDISC SDTM standards, use the provided DS_RAW dataset to create a DS dataset with one record per collected disposition record.
The output dataset should contain the following columns in this order: DOMAIN, STUDYID, USUBJID, DSSEQ, DSDECOD, DSCAT, DSDTC
Read the source datasets from /app/input and save the completed dataset as /app/output/ds.csv.
run.R
Edit4 lineslibrary(yamaa)ds <- yamaa_domain("spec.yaml")$outputdsrun.py
Edit4 linesimport yamaads = yamaa.yamaa_domain("spec.yaml").outputdssolutions/sdtm-ds-sequence/result.R
Edit61 lines# Reference solution for the yamaa benchmark sdtm-ds-sequence (R track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.library(dplyr, warn.conflicts = FALSE)library(readr)library(stringr)ds_raw <- read_csv( "/app/input/ds_raw.csv", col_types = cols(.default = col_character()))complete_date <- function(x) { x <- str_trim(x) if (is.na(x) || x == "") { return(as.Date(NA)) } parts <- str_split(x, "-", simplify = TRUE) n <- ncol(parts) if (n == 3 && nchar(parts[1]) == 4) { return(suppressWarnings(as.Date(x))) } if (n == 2 && nchar(parts[1]) == 4) { y <- suppressWarnings(as.integer(parts[1])) m <- suppressWarnings(as.integer(parts[2])) if (is.na(y) || is.na(m) || m < 1 || m > 12) { return(as.Date(NA)) } return(suppressWarnings(as.Date(sprintf("%04d-%02d-15", y, m)))) } if (n == 1 && nchar(x) == 4) { y <- suppressWarnings(as.integer(x)) if (is.na(y)) { return(as.Date(NA)) } return(suppressWarnings(as.Date(sprintf("%04d-06-15", y)))) } as.Date(NA)}ds <- ds_raw |> rowwise() |> mutate( DOMAIN = "DS", STUDYID = STUDY, USUBJID = PATNUM, DSCAT = ifelse(DSDECOD == "RANDOMIZED", "PROTOCOL MILESTONE", "DISPOSITION EVENT"), DSDTC = complete_date(DSDTCOL) ) |> ungroup() |> arrange(USUBJID, DSDTC, DSDECOD) |> group_by(USUBJID) |> mutate(DSSEQ = row_number()) |> ungroup() |> select(DOMAIN, STUDYID, USUBJID, DSSEQ, DSDECOD, DSCAT, DSDTC)dir.create("/app/output", showWarnings = FALSE)write_csv(ds, "/app/output/ds.csv", na = "")solutions/sdtm-ds-sequence/result.py
Edit81 lines# Reference solution for the yamaa benchmark sdtm-ds-sequence (Python track).## Written from the benchmark's full prompt and its inputs alone, as an agent# would write it. Harbor's oracle agent runs it to check that the prompt can# be solved and that the grader scores a correct answer 1.from pathlib import Pathimport polars as pldef complete_date(raw: str | None) -> str | None: if raw is None: return None s = raw.strip() if s == "": return None parts = s.split("-") try: if len(parts) == 3: y, m, d = int(parts[0]), int(parts[1]), int(parts[2]) return f"{y:04d}-{m:02d}-{d:02d}" if len(parts[0]) == 4 else None if len(parts) == 2 and len(parts[0]) == 4: y, m = int(parts[0]), int(parts[1]) if 1 <= m <= 12: return f"{y:04d}-{m:02d}-15" return None if len(parts) == 1 and len(s) == 4: y = int(s) return f"{y:04d}-06-15" return None except ValueError: return Noneds_raw = pl.read_csv("/app/input/ds_raw.csv", infer_schema=False)records: list[dict] = []for row in ds_raw.to_dicts(): study = row["STUDY"] pat = row["PATNUM"] decod = row["DSDECOD"] raw = row["DSDTCOL"] completed = complete_date(raw) # Validate the completed date parses. check = pl.DataFrame({"d": [completed]}).with_columns( pl.col("d").str.to_date(strict=False).alias("dt") )["dt"][0] if check is None: completed = None dscat = "PROTOCOL MILESTONE" if decod == "RANDOMIZED" else "DISPOSITION EVENT" records.append( { "STUDYID": study, "USUBJID": pat, "DSDECOD": decod, "DSCAT": dscat, "DSDTC": completed, } )ds = pl.DataFrame( records, schema={ "STUDYID": pl.String, "USUBJID": pl.String, "DSDECOD": pl.String, "DSCAT": pl.String, "DSDTC": pl.String, },).with_columns(DOMAIN=pl.lit("DS"), DSDTC_DT=pl.col("DSDTC").str.to_date(strict=False))ds = ( ds.sort(["USUBJID", "DSDTC_DT", "DSDECOD"], nulls_last=True) .with_columns(DSSEQ=pl.col("USUBJID").cum_count().over("USUBJID")) .with_columns(DSSEQ=pl.col("DSSEQ").cast(pl.Int64)) .select("DOMAIN", "STUDYID", "USUBJID", "DSSEQ", "DSDECOD", "DSCAT", "DSDTC"))Path("/app/output").mkdir(exist_ok=True)ds.write_csv("/app/output/ds.csv")solutions/sdtm-ds-sequence/result.sas
Edit24 lines/* opensas reference solution for the yamaa benchmark sdtm-ds-sequence. Reads the benchmark inputs and derives the requested output. Execute with the existing opensas runtime: opensas /app/output/result.sas. */data ds_raw; length STUDY PATNUM INSTANCE DSDECOD DSDTCOL $1024; infile "/app/input/ds_raw.csv" dsd dlm="," firstobs=2 truncover lrecl=32767; input STUDY : $char1024. PATNUM : $char1024. INSTANCE : $char1024. DSDECOD : $char1024. DSDTCOL : $char1024.;run;data prepared; set ds_raw; length DOMAIN $2 STUDYID USUBJID DSCAT DSDTC $1024; DOMAIN='DS'; D=.; DSDTC=''; STUDYID=STUDY; USUBJID=PATNUM;if DSDECOD='RANDOMIZED' then DSCAT='PROTOCOL MILESTONE'; else DSCAT='DISPOSITION EVENT';if lengthn(DSDTCOL)=10 then D=input(DSDTCOL,?? yymmdd10.); else if lengthn(DSDTCOL)=7 then D=input(cats(DSDTCOL,'-15'),?? yymmdd10.); else if lengthn(DSDTCOL)=4 then D=input(cats(DSDTCOL,'-06-15'),?? yymmdd10.);if not missing(D) then DSDTC=put(D,yymmdd10.); NODATE=missing(D); run;proc sort data=prepared; by STUDYID USUBJID NODATE D; run;data result; set prepared; by STUDYID USUBJID; retain DSSEQ; if first.USUBJID then DSSEQ=0; DSSEQ+1;run;proc sql; create table final as select DOMAIN, STUDYID, USUBJID, DSSEQ, DSDECOD, DSCAT, DSDTC from result;quit;proc export data=final outfile="/app/output/ds.csv" dbms=csv replace; run;
yamaa
Comments
Signed in with GitHub; saved in repository Discussions