Post

Based on 6 simulations, we provide practical recommendations. For context, the main limiting factor on using BF is practical: they are hard to compute. Researchers mostly have to resort to numerical estimation. BS is flexible but computationally expensive; SDDR less flexible, but cheaper.

The upshot: all methods can give accurate estimates, but the cost-benefit trade-off shifts depending on (1) computational cost of MCMC sampling, (2) normality of the posterior, (3) number of model parameters, and (4) test value tail probability.

1. BS or SDDR? If MCMC sampling is cheap, BS is a good default. But we found an unfavorable cost–benefit trend: with more parameters, BS becomes computationally expensive while BF estimates become less accurate. In models with many parameters, SDDR is often cheaper and at least as accurate.

23/6/26, 2:33 pm0 reposts1 like

We also find that variability of SDDR estimates increases with BF10 (i.e., test value tail probability). For large BF10, SDDR was less reliable than BS, but more reliable for BF01 > 1. Note that for low tail probabilities, reliability may matter less (is BF10 = 1,000 or 1,000,000?).

We also find that the reliability of BS decreases with increasing number of model parameters—these can grow quickly in hierarchical models with random slopes. In our simulation, BS was more reliable than SDDR with few parameters, but SDDR was often more reliable with many (e.g., 780).

Assuming a Gaussian posterior buys great efficiency. In Simulation 5, as few as 5,000 MCMC samples were sufficient to reliably distinguish between, e.g., moderate and strong evidence. When the assumption fails, the Gaussian approximation risks bias.