yamaa As a clinical trial statistician or programmer, challenge anything in this benchmark: describe a problem in Comments, or propose a fix with the Edit button on any source file.
- Task: Would a real study need this derivation, specified this way? Does it follow CDISC ADaM: variable names, controlled terms, record structure? Flag any rule that is ambiguous or that a sponsor would handle differently.
- Inputs: Do they look like data collected in a real trial, and do they include the situations you would expect to meet in one?
- Expected output: Would independent QC programming reproduce every value from these inputs and rules?
The other tabs beside Comments are background for assessment, not under review.
Summary
EditUnder reviewWhat this benchmark meansREADME.mdGoal: bound the rows a filter matches, by count and by share: upper and lower count bounds, upper and lower share bounds, per-group share bounds with an inclusive edge, and a deliberately violated warning.
Input: eight subjects in two arms of four. Each has an ARM, a
FLAG marking it for review, and a collected VAL. Three are flagged
(one in arm A, two in arm B); seven carry a value and one does not.
Checks:
- Flagged subjects stay at or under one half: 3 of 8 do, so the check holds.
- Subjects with a collected value stay above three quarters: 7 of 8 do, so the check holds.
- Per arm, flagged subjects stay at or under one half: arm A has 1 of 4, arm B has 2 of 4, and the exact half counts as held.
- Flagged subjects number at most three: exactly 3 do, so the check holds and pins the inclusive count boundary.
- Subjects with a collected value number at least seven: exactly 7 do, so the check holds and pins the inclusive count boundary.
- Unflagged subjects stay at or under one half: 5 of 8 do not, so the check is violated. It is a warning, so the run still succeeds and the warning log carries the broken check.
The check log records REPORT_VERSION, ARTIFACT, SPEC_PATH,
VERIFICATION_ID, CHECK, TARGET, REQUIREMENT, SEVERITY,
OUTCOME, CONDITION, EVALUATED_COUNT, FAILURE_COUNT, and
DETAILS. The group check evaluates two groups; the others evaluate the
whole dataset as one unit. A held check leaves CONDITION empty and
reports zero failures; the violated warning names its condition and
reports how many rows matched out of how many.
The warning log records LOG_VERSION, ARTIFACT, SEVERITY,
CONDITION, REQUIREMENT, SPEC_PATH, VERIFICATION_ID,
FAILURE_COUNT, OFFENDING_KEYS, and DETAILS. A dataset-level check
has no offending rows, so it carries an empty key set.
Inputs
Under review1 source fileinput/dm.csv
Edit8 rows| USUBJID | ARM | FLAG | VAL |
|---|---|---|---|
| A | Y | 12 | |
| A | N | 10 | |
| A | N | — | |
| A | N | 9 | |
| B | Y | 11 | |
| B | Y | 8 | |
| B | N | 7 | |
| B | N | 6 |
Expected output
Under review15 expected rowsexpected/adsl-checks.csv
Edit6 rows| REPORT_VERSION | ARTIFACT | SPEC_PATH | VERIFICATION_ID | CHECK | TARGET | REQUIREMENT | SEVERITY | OUTCOME | CONDITION | EVALUATED_COUNT | FAILURE_COUNT | DETAILS |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1.0 | adsl.csv | verifications[0].row_count | flagged-share-within-limit | row_count | — | REQ-0385 | error | held | — | 1 | 0 | {} |
| 1.0 | adsl.csv | verifications[1].row_count | values-mostly-present | row_count | — | REQ-0385 | error | held | — | 1 | 0 | {} |
| 1.0 | adsl.csv | verifications[2].row_count | flagged-share-by-arm | row_count | — | REQ-0385 | error | held | — | 2 | 0 | {} |
| 1.0 | adsl.csv | verifications[3].row_count | flagged-count-within-limit | row_count | — | REQ-0385 | error | held | — | 1 | 0 | {} |
| 1.0 | adsl.csv | verifications[4].row_count | values-count-floor | row_count | — | REQ-0385 | error | held | — | 1 | 0 | {} |
| 1.0 | adsl.csv | verifications[5].row_count | unflagged-share-warning | row_count | — | REQ-0385 | warning | violated | row_count_failed | 1 | 1 | {"count":5,"denominator":8} |
expected/adsl-warnings.csv
Edit1 row| LOG_VERSION | ARTIFACT | SEVERITY | CONDITION | REQUIREMENT | SPEC_PATH | VERIFICATION_ID | FAILURE_COUNT | OFFENDING_KEYS | DETAILS |
|---|---|---|---|---|---|---|---|---|---|
| 1.0 | adsl.csv | warning | row_count_failed | REQ-0385 | verifications[5].row_count | unflagged-share-warning | 1 | [{}] | {"count":5,"denominator":8} |
expected/adsl.csv
Edit8 rows| USUBJID | ARM | FLAG | VAL |
|---|---|---|---|
| A | Y | 12 | |
| A | N | 10 | |
| A | N | — | |
| A | N | 9 | |
| B | Y | 11 | |
| B | Y | 8 | |
| B | N | 7 | |
| B | N | 6 |
Comments and assessment materials
| Dataset Name | Variable Name | Variable Label | Type | Codelist/Controlled Terms | Core | Computational Method | Origin |
|---|---|---|---|---|---|---|---|
| ADSL | USUBJID | Unique Subject Identifier | character | Copy value from DM.USUBJID. | Predecessor | ||
| ADSL | ARM | Planned Arm | character | Copy value from DM.ARM. | Predecessor | ||
| ADSL | FLAG | Flagged for Review | character | Copy value from DM.FLAG. | Predecessor | ||
| ADSL | VAL | Collected Value | numeric | Copy value from DM.VAL. | Predecessor |
| Version | Date | Author | Description | Reviewer | Sign-off |
|---|---|---|---|---|---|
| 1.3 | 2026-09-29 | yamaa docs build | Show study codelist names, external dictionaries, and ISO 8601 formats in variable rows; list the document's codelist values and metadata | ||
| 1.2 | 2026-09-29 | yamaa docs build | Classify ADaM copies from a declared input as Predecessor and recodes as Derived, and SDTM recodes of locally derived columns as Derived | ||
| 1.1 | 2026-09-29 | yamaa docs build | Resolve lookups into the donor record they select, and give every derivation shape a plain-language rule instead of its raw YAML | ||
| 1.0 | yamaa docs build | Initial generation from spec.yaml |
Comments
Signed in with GitHub; saved in repository Discussions