ALL PASS
Strategy 1 (External Data): 45 scenarios • Strategy 2 (MCPower DGP): 45 scenarios • Strategy 3 (Parallel Power): 45 scenarios • Strategy 4 (Statistical Z-test): 95 scenarios
Thresholds: agreement ≥ 95%, β corr ≥ 0.98, SE corr ≥ 0.95, power diff ≤ 5%
All scenarios at a glance — pass/fail across all four strategies.
| Scenario | Formula | Effects | N | Cl. | S1 | S2 | S3 | S4 |
|---|---|---|---|---|---|---|---|---|
| Random Intercept | ||||||||
| intercept_icc0.1_cl10_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl10_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl10_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl10_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl20_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl20_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl20_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl20_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl50_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl50_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl50_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.1_cl50_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl10_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl10_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl10_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl10_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl20_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl20_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl20_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl20_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl50_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl50_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl50_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.2_cl50_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl10_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl10_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl10_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl10_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 10 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl20_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl20_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl20_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl20_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl50_n500_small | y ~ x1 + (1|g) | x1=0.3 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl50_n500_medium | y ~ x1 + (1|g) | x1=0.5 | 500 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl50_n1000_small | y ~ x1 + (1|g) | x1=0.3 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_icc0.5_cl50_n1000_medium | y ~ x1 + (1|g) | x1=0.5 | 1000 | 50 | PASS | PASS | PASS | PASS |
| intercept_2pred_icc0.2_cl20_n1000_varied | y ~ x1 + x2 + (1|g) | x1=0.5, x2=0.3 | 1000 | 20 | PASS | PASS | PASS | PASS |
| intercept_2pred_icc0.2_cl20_n1000_small_both | y ~ x1 + x2 + (1|g) | x1=0.2, x2=0.2 | 1000 | 20 | PASS | PASS | PASS | PASS |
| sens_n50_d05 | y ~ x1 + (1|g) | x1=0.05 | 50 | 10 | — | — | — | PASS |
| sens_n60_d08_hicc | y ~ x1 + (1|g) | x1=0.08 | 60 | 10 | — | — | — | PASS |
| sens_n80_d08 | y ~ x1 + (1|g) | x1=0.08 | 80 | 10 | — | — | — | PASS |
| sens_n100_d10 | y ~ x1 + (1|g) | x1=0.1 | 100 | 10 | — | — | — | PASS |
| sens_n100_d12_hicc | y ~ x1 + (1|g) | x1=0.12 | 100 | 5 | — | — | — | PASS |
| sens_n120_d10_hicc | y ~ x1 + (1|g) | x1=0.1 | 120 | 10 | — | — | — | PASS |
| sens_n150_d10 | y ~ x1 + (1|g) | x1=0.1 | 150 | 10 | — | — | — | PASS |
| sens_n120_d15_hicc | y ~ x1 + (1|g) | x1=0.15 | 120 | 10 | — | — | — | PASS |
| sens_n100_d15_hicc | y ~ x1 + (1|g) | x1=0.15 | 100 | 10 | — | — | — | PASS |
| sens_n180_d15_hicc | y ~ x1 + (1|g) | x1=0.15 | 180 | 10 | — | — | — | PASS |
| sens_n200_d12 | y ~ x1 + (1|g) | x1=0.12 | 200 | 20 | — | — | — | PASS |
| sens_n200_d15_hicc | y ~ x1 + (1|g) | x1=0.15 | 200 | 10 | — | — | — | PASS |
| sens_n150_d20 | y ~ x1 + (1|g) | x1=0.2 | 150 | 10 | — | — | — | PASS |
| sens_n250_d18 | y ~ x1 + (1|g) | x1=0.18 | 250 | 10 | — | — | — | PASS |
| sens_n300_d15 | y ~ x1 + (1|g) | x1=0.15 | 300 | 20 | — | — | — | PASS |
| sens_n260_d15 | y ~ x1 + (1|g) | x1=0.15 | 260 | 20 | — | — | — | PASS |
| sens_n300_d20 | y ~ x1 + (1|g) | x1=0.2 | 300 | 15 | — | — | — | PASS |
| sens_n300_d20_hicc | y ~ x1 + (1|g) | x1=0.2 | 300 | 10 | — | — | — | PASS |
| sens_n400_d20 | y ~ x1 + (1|g) | x1=0.2 | 400 | 20 | — | — | — | PASS |
| sens_n350_d25_hicc | y ~ x1 + (1|g) | x1=0.25 | 350 | 10 | — | — | — | PASS |
| sens_n200_d18 | y ~ x1 + (1|g) | x1=0.18 | 200 | 10 | — | — | — | PASS |
| sens_n250_d20_hicc | y ~ x1 + (1|g) | x1=0.2 | 250 | 10 | — | — | — | PASS |
| sens_n350_d18 | y ~ x1 + (1|g) | x1=0.18 | 350 | 10 | — | — | — | PASS |
| sens_n500_d30 | y ~ x1 + (1|g) | x1=0.3 | 500 | 20 | — | — | — | PASS |
| sens_2pred_n100 | y ~ x1 + x2 + (1|g) | x1=0.1, x2=0.1 | 100 | 10 | — | — | — | PASS |
| sens_2pred_n150 | y ~ x1 + x2 + (1|g) | x1=0.12, x2=0.12 | 150 | 10 | — | — | — | PASS |
| sens_2pred_low | y ~ x1 + x2 + (1|g) | x1=0.15, x2=0.15 | 200 | 10 | — | — | — | PASS |
| sens_2pred_n300_hicc | y ~ x1 + x2 + (1|g) | x1=0.15, x2=0.15 | 300 | 10 | — | — | — | PASS |
| sens_2pred_mid | y ~ x1 + x2 + (1|g) | x1=0.2, x2=0.15 | 300 | 15 | — | — | — | PASS |
| sens_2pred_high | y ~ x1 + x2 + (1|g) | x1=0.2, x2=0.15 | 400 | 10 | — | — | — | PASS |
| sens_2pred_n250 | y ~ x1 + x2 + (1|g) | x1=0.15, x2=0.12 | 250 | 10 | — | — | — | PASS |
| sens_2pred_n350 | y ~ x1 + x2 + (1|g) | x1=0.18, x2=0.15 | 350 | 10 | — | — | — | PASS |
| Random Slopes | ||||||||
| slopes_cl20_n1000 | y ~ x1 + (1 + x1|g) | x1=0.5 | 1000 | 20 | PASS | PASS | PASS | PASS |
| slopes_cl30_n1500 | y ~ x1 + (1 + x1|g) | x1=0.5 | 1500 | 30 | PASS | PASS | PASS | PASS |
| slopes_cl20_n2000 | y ~ x1 + (1 + x1|g) | x1=0.5 | 2000 | 20 | PASS | PASS | PASS | PASS |
| slopes_cl50_n2000 | y ~ x1 + (1 + x1|g) | x1=0.5 | 2000 | 50 | PASS | PASS | PASS | PASS |
| sens_slopes_n150 | y ~ x1 + (1 + x1|g) | x1=0.1 | 150 | 10 | — | — | — | PASS |
| sens_slopes_vlow | y ~ x1 + (1 + x1|g) | x1=0.12 | 200 | 10 | — | — | — | PASS |
| sens_slopes_n250 | y ~ x1 + (1 + x1|g) | x1=0.15 | 250 | 10 | — | — | — | PASS |
| sens_slopes_low | y ~ x1 + (1 + x1|g) | x1=0.15 | 300 | 15 | — | — | — | PASS |
| sens_slopes_n300_k20 | y ~ x1 + (1 + x1|g) | x1=0.2 | 300 | 20 | — | — | — | PASS |
| sens_slopes_n350 | y ~ x1 + (1 + x1|g) | x1=0.2 | 350 | 10 | — | — | — | PASS |
| sens_slopes_mid | y ~ x1 + (1 + x1|g) | x1=0.2 | 400 | 20 | — | — | — | PASS |
| sens_slopes_n450 | y ~ x1 + (1 + x1|g) | x1=0.25 | 450 | 15 | — | — | — | PASS |
| sens_slopes_high | y ~ x1 + (1 + x1|g) | x1=0.25 | 500 | 20 | — | — | — | PASS |
| sens_slopes_n400_k10 | y ~ x1 + (1 + x1|g) | x1=0.18 | 400 | 10 | — | — | — | PASS |
| Nested Random Effects | ||||||||
| nested_cl10_npp3_n1500 | y ~ x1 + (1|school/classroom) | x1=0.5 | 1500 | 10 | PASS | PASS | PASS | PASS |
| nested_cl15_npp4_n2400 | y ~ x1 + (1|school/classroom) | x1=0.5 | 2400 | 15 | PASS | PASS | PASS | PASS |
| nested_cl20_npp3_n1800 | y ~ x1 + (1|school/classroom) | x1=0.5 | 1800 | 20 | PASS | PASS | PASS | PASS |
| sens_nested_n100 | y ~ x1 + (1|school/classroom) | x1=0.1 | 100 | 5 | — | — | — | PASS |
| sens_nested_n150 | y ~ x1 + (1|school/classroom) | x1=0.15 | 150 | 5 | — | — | — | PASS |
| sens_nested_n240 | y ~ x1 + (1|school/classroom) | x1=0.15 | 240 | 8 | — | — | — | PASS |
| sens_nested_n300 | y ~ x1 + (1|school/classroom) | x1=0.15 | 300 | 10 | — | — | — | PASS |
| sens_nested_n450 | y ~ x1 + (1|school/classroom) | x1=0.2 | 450 | 10 | — | — | — | PASS |
| sens_nested_n600 | y ~ x1 + (1|school/classroom) | x1=0.2 | 600 | 10 | — | — | — | PASS |
| sens_nested_n360 | y ~ x1 + (1|school/classroom) | x1=0.12 | 360 | 10 | — | — | — | PASS |
| sens_nested_n500 | y ~ x1 + (1|school/classroom) | x1=0.15 | 500 | 10 | — | — | — | PASS |
Same data, same test — do MCPower and lme4 make the same significance decisions?
| Scenario | Agreement | Kappa | N compared | Result |
|---|---|---|---|---|
| intercept_icc0.1_cl10_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n500_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n500_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n1000_small | 1.000 | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n1000_medium | 1.000 | 1.000 | 100 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_varied | 1.000 | 1.000 | 100 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_small_both | 1.000 | 1.000 | 100 | PASS |
| slopes_cl20_n1000 | 1.000 | 1.000 | 100 | PASS |
| slopes_cl30_n1500 | 1.000 | 1.000 | 100 | PASS |
| slopes_cl20_n2000 | 1.000 | 1.000 | 100 | PASS |
| slopes_cl50_n2000 | 1.000 | 1.000 | 100 | PASS |
| nested_cl10_npp3_n1500 | 1.000 | 1.000 | 100 | PASS |
| nested_cl15_npp4_n2400 | 1.000 | 1.000 | 100 | PASS |
| nested_cl20_npp3_n1800 | 1.000 | 1.000 | 100 | PASS |
MCPower-generated data fitted by both solvers — validates the DGP pipeline.
| Scenario | Agreement | N compared | Result |
|---|---|---|---|
| intercept_icc0.1_cl10_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl10_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl20_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.1_cl50_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl10_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl20_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.2_cl50_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl10_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl20_n1000_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n500_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n500_medium | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n1000_small | 1.000 | 100 | PASS |
| intercept_icc0.5_cl50_n1000_medium | 1.000 | 100 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_varied | 1.000 | 100 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_small_both | 1.000 | 100 | PASS |
| slopes_cl20_n1000 | 1.000 | 100 | PASS |
| slopes_cl30_n1500 | 1.000 | 100 | PASS |
| slopes_cl20_n2000 | 1.000 | 100 | PASS |
| slopes_cl50_n2000 | 1.000 | 100 | PASS |
| nested_cl10_npp3_n1500 | 1.000 | 100 | PASS |
| nested_cl15_npp4_n2400 | 1.000 | 100 | PASS |
| nested_cl20_npp3_n1800 | 1.000 | 100 | PASS |
Independent DGP comparison — MCPower and R generate data separately, compare power estimates.
| Scenario | MCPower Power | R Power | Diff | Result |
|---|---|---|---|---|
| intercept_icc0.1_cl10_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl10_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl10_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl20_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl20_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl20_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl50_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl50_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl50_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.1_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl10_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl10_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl10_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl20_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl20_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl20_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl50_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl50_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl50_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.2_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl10_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl10_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl10_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl20_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl20_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl20_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl50_n500_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl50_n500_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl50_n1000_small | 1.000 | 1.000 | 0.000 | PASS |
| intercept_icc0.5_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_varied | 1.000 | 1.000 | 0.000 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_small_both | 1.000 | 1.000 | 0.000 | PASS |
| slopes_cl20_n1000 | 1.000 | 1.000 | 0.000 | PASS |
| slopes_cl30_n1500 | 1.000 | 1.000 | 0.000 | PASS |
| slopes_cl20_n2000 | 1.000 | 1.000 | 0.000 | PASS |
| slopes_cl50_n2000 | 1.000 | 1.000 | 0.000 | PASS |
| nested_cl10_npp3_n1500 | 1.000 | 1.000 | 0.000 | PASS |
| nested_cl15_npp4_n2400 | 1.000 | 1.000 | 0.000 | PASS |
| nested_cl20_npp3_n1800 | 1.000 | 1.000 | 0.000 | PASS |
Two-proportion z-test with Benjamini-Hochberg FDR correction. Runs power simulations independently when S3 data is unavailable. PASS = cannot reject H0 that powers are equal (adj. p > 0.05).
0 / 95 scenarios rejected at FDR = 5%.
| Scenario | MC Power | R Power | Diff | SE | z | p-value | adj. p | Result |
|---|---|---|---|---|---|---|---|---|
| intercept_icc0.1_cl10_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl10_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl10_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl20_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl20_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl20_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl50_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl50_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl50_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.1_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl10_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl10_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl10_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl20_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl20_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl20_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl50_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl50_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl50_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.2_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl10_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl10_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl10_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl10_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl20_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl20_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl20_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl20_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl50_n500_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl50_n500_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl50_n1000_small | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_icc0.5_cl50_n1000_medium | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_varied | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| intercept_2pred_icc0.2_cl20_n1000_small_both | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| slopes_cl20_n1000 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| slopes_cl30_n1500 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| slopes_cl20_n2000 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| slopes_cl50_n2000 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| nested_cl10_npp3_n1500 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| nested_cl15_npp4_n2400 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| nested_cl20_npp3_n1800 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| sens_n50_d05 | 0.072 | 0.070 | 0.002 | 0.0162 | 0.123 | 0.9020 | 1.0000 | PASS |
| sens_n60_d08_hicc | 0.082 | 0.104 | -0.022 | 0.0184 | -1.198 | 0.2310 | 1.0000 | PASS |
| sens_n80_d08 | 0.108 | 0.098 | 0.010 | 0.0192 | 0.520 | 0.6029 | 1.0000 | PASS |
| sens_n100_d10 | 0.166 | 0.172 | -0.006 | 0.0237 | -0.253 | 0.8002 | 1.0000 | PASS |
| sens_n100_d12_hicc | 0.242 | 0.214 | 0.028 | 0.0265 | 1.055 | 0.2913 | 1.0000 | PASS |
| sens_n120_d10_hicc | 0.176 | 0.172 | 0.004 | 0.0240 | 0.167 | 0.8675 | 1.0000 | PASS |
| sens_n150_d10 | 0.226 | 0.200 | 0.026 | 0.0259 | 1.004 | 0.3153 | 1.0000 | PASS |
| sens_n120_d15_hicc | 0.364 | 0.344 | 0.020 | 0.0302 | 0.661 | 0.5084 | 1.0000 | PASS |
| sens_n100_d15_hicc | 0.322 | 0.306 | 0.016 | 0.0294 | 0.545 | 0.5857 | 1.0000 | PASS |
| sens_n180_d15_hicc | 0.512 | 0.508 | 0.004 | 0.0316 | 0.127 | 0.8993 | 1.0000 | PASS |
| sens_n200_d12 | 0.380 | 0.402 | -0.022 | 0.0309 | -0.713 | 0.4759 | 1.0000 | PASS |
| sens_n200_d15_hicc | 0.554 | 0.562 | -0.008 | 0.0314 | -0.255 | 0.7990 | 1.0000 | PASS |
| sens_n150_d20 | 0.686 | 0.640 | 0.046 | 0.0299 | 1.539 | 0.1239 | 1.0000 | PASS |
| sens_n250_d18 | 0.804 | 0.810 | -0.006 | 0.0250 | -0.240 | 0.8100 | 1.0000 | PASS |
| sens_n300_d15 | 0.724 | 0.706 | 0.018 | 0.0285 | 0.630 | 0.5284 | 1.0000 | PASS |
| sens_n260_d15 | 0.652 | 0.660 | -0.008 | 0.0300 | -0.266 | 0.7900 | 1.0000 | PASS |
| sens_n300_d20 | 0.900 | 0.920 | -0.020 | 0.0181 | -1.105 | 0.2692 | 1.0000 | PASS |
| sens_n300_d20_hicc | 0.906 | 0.910 | -0.004 | 0.0183 | -0.219 | 0.8268 | 1.0000 | PASS |
| sens_n400_d20 | 0.968 | 0.972 | -0.004 | 0.0108 | -0.371 | 0.7108 | 1.0000 | PASS |
| sens_n350_d25_hicc | 0.994 | 0.998 | -0.004 | 0.0040 | -1.002 | 0.3163 | 1.0000 | PASS |
| sens_n200_d18 | 0.696 | 0.694 | 0.002 | 0.0291 | 0.069 | 0.9452 | 1.0000 | PASS |
| sens_n250_d20_hicc | 0.854 | 0.870 | -0.016 | 0.0218 | -0.733 | 0.4633 | 1.0000 | PASS |
| sens_n350_d18 | 0.906 | 0.916 | -0.010 | 0.0180 | -0.555 | 0.5787 | 1.0000 | PASS |
| sens_n500_d30 | 1.000 | 1.000 | 0.000 | 0.0000 | 0.000 | 1.0000 | 1.0000 | PASS |
| sens_2pred_n100 | 0.194 | 0.218 | -0.024 | 0.0256 | -0.938 | 0.3481 | 1.0000 | PASS |
| sens_2pred_n150 | 0.400 | 0.398 | 0.002 | 0.0310 | 0.065 | 0.9485 | 1.0000 | PASS |
| sens_2pred_low | 0.740 | 0.748 | -0.008 | 0.0276 | -0.290 | 0.7719 | 1.0000 | PASS |
| sens_2pred_n300_hicc | 0.892 | 0.872 | 0.020 | 0.0204 | 0.980 | 0.3270 | 1.0000 | PASS |
| sens_2pred_mid | 0.964 | 0.970 | -0.006 | 0.0113 | -0.531 | 0.5954 | 1.0000 | PASS |
| sens_2pred_high | 0.990 | 0.996 | -0.006 | 0.0053 | -1.138 | 0.2552 | 1.0000 | PASS |
| sens_2pred_n250 | 0.738 | 0.756 | -0.018 | 0.0275 | -0.655 | 0.5127 | 1.0000 | PASS |
| sens_2pred_n350 | 0.966 | 0.980 | -0.014 | 0.0103 | -1.366 | 0.1720 | 1.0000 | PASS |
| sens_slopes_n150 | 0.138 | 0.122 | 0.016 | 0.0213 | 0.752 | 0.4519 | 1.0000 | PASS |
| sens_slopes_vlow | 0.180 | 0.164 | 0.016 | 0.0239 | 0.670 | 0.5026 | 1.0000 | PASS |
| sens_slopes_n250 | 0.246 | 0.266 | -0.020 | 0.0276 | -0.725 | 0.4687 | 1.0000 | PASS |
| sens_slopes_low | 0.286 | 0.316 | -0.030 | 0.0290 | -1.034 | 0.3011 | 1.0000 | PASS |
| sens_slopes_n300_k20 | 0.558 | 0.552 | 0.006 | 0.0314 | 0.191 | 0.8486 | 1.0000 | PASS |
| sens_slopes_n350 | 0.392 | 0.390 | 0.002 | 0.0309 | 0.065 | 0.9483 | 1.0000 | PASS |
| sens_slopes_mid | 0.590 | 0.586 | 0.004 | 0.0311 | 0.128 | 0.8978 | 1.0000 | PASS |
| sens_slopes_n450 | 0.714 | 0.780 | -0.066 | 0.0275 | -2.400 | 0.0164 | 1.0000 | PASS |
| sens_slopes_high | 0.814 | 0.800 | 0.014 | 0.0250 | 0.561 | 0.5749 | 1.0000 | PASS |
| sens_slopes_n400_k10 | 0.360 | 0.332 | 0.028 | 0.0301 | 0.931 | 0.3520 | 1.0000 | PASS |
| sens_nested_n100 | 0.172 | 0.158 | 0.014 | 0.0235 | 0.596 | 0.5509 | 1.0000 | PASS |
| sens_nested_n150 | 0.458 | 0.412 | 0.046 | 0.0314 | 1.467 | 0.1423 | 1.0000 | PASS |
| sens_nested_n240 | 0.646 | 0.596 | 0.050 | 0.0307 | 1.630 | 0.1032 | 1.0000 | PASS |
| sens_nested_n300 | 0.746 | 0.710 | 0.036 | 0.0281 | 1.279 | 0.2008 | 1.0000 | PASS |
| sens_nested_n450 | 0.984 | 0.984 | 0.000 | 0.0079 | 0.000 | 1.0000 | 1.0000 | PASS |
| sens_nested_n600 | 0.994 | 0.994 | 0.000 | 0.0049 | 0.000 | 1.0000 | 1.0000 | PASS |
| sens_nested_n360 | 0.650 | 0.598 | 0.052 | 0.0306 | 1.697 | 0.0896 | 1.0000 | PASS |
| sens_nested_n500 | 0.900 | 0.924 | -0.024 | 0.0179 | -1.339 | 0.1804 | 1.0000 | PASS |
Points on the diagonal = perfect agreement