Benchmark review¶
If you already started, this button takes you to your next benchmark.
Purpose¶
Benchmarks are the worked examples and conformance tests of yamaa. This review
asks clinical trial statisticians and programmers to confirm that each sdtm
and adam benchmark represents real work before it is finalized. It covers
200 benchmarks. Comments from before this review do not count.
People¶
| Role | Who |
|---|---|
| Owner | elong0527 |
| Coordinator | pzhang724 |
| Reviewers | Cassie-Shen, LarsaLi, RC0128, hellomingchun, huchentw, pzhang724, sycamoreNN, wangben718 |
Anyone can review. Your GitHub account appears here after your first review, and the coordinator confirms the reviewers before a benchmark is finalized.
How to review¶
We suggest spending no more than 5 minutes on a review. It gives a preliminary approval, and the coordinator confirms the reviewers before a benchmark is finalized.
- Sign in to GitHub in your browser.
- Select Start reviewing.
- Read the Summary, Inputs and Expected output.
- Answer each question with Agree or Disagree: Task, Inputs, and, where shown, the AI-assisted review of three potential concerns. If you disagree, say why in the comments.
- Select Submit and next. In the GitHub tab, tick the search box if GitHub shows it, select Start discussion, then close the tab.
- Select Skip for any benchmark you cannot judge. Your next review lists the benchmarks you skipped, so the status below can count them. At the end of the queue you can publish any skips that are left.
Reviews are public, so use public or invented data only.
A benchmark passes when two reviewers approve it and nobody disagrees. It then stays approved and does not come back.
A review that does not approve opens a rework issue for the benchmark, with the review's comments, so nobody needs to file one. The pull request that reworks the benchmark closes the issue, and GitHub notifies the reviewer, who reviews the benchmark again. The new review replaces their earlier one: agreeing can approve the benchmark, and disagreeing opens another rework issue. When the coordinator opens a new round, a benchmark that still needs rework starts it with no reviews.
Status¶
Updated 2026-11-13 16:33 UTC.
| Round | Benchmarks | Approved | Needs rework | In review | Not reviewed |
|---|---|---|---|---|---|
| Round 1 | 200 | 13 | 5 | 32 | 150 |
| Group | Benchmarks | Approved | Needs rework | In review | Not reviewed |
|---|---|---|---|---|---|
sdtm |
70 | 9 | 4 | 13 | 44 |
adam |
130 | 4 | 1 | 19 | 106 |
| All | 200 | 13 | 5 | 32 | 150 |
Filter the table to find the benchmarks to align on, such as those a reviewer did not approve, and select a decision to read its review. The page address keeps the filters, so you can share the list.