7.10 Simulating the Effects of Test Choices

Code

theme_set(theme_minimal())

berkeley_blue   <- '#003262'
berkeley_gold   <- '#FDB515'
berkeley_sather <- '#B9D3B6'

7.10.1 Should we use a t-test or a wilcox sign-rank?

There is some open discussion in the applied statistics literature about whether we should ever be using a t-test. In particular, if the underlying distribution that generates the data is not normal, than the assumptions of a t-test are not, technically satisfied and the test does not produce results that have nominal p-value coverage. This is both technically and theoretically true; and yet, researchers, data scientists, your instructors, and the entire world runs t-tests as “test of first recourse.”

What is the alternative to conducting a t-test as the test of first recourse? It might be the Wilcox test. The Wilcox test makes a weaker assumption – of symmetry around the mean or median – which is weaker than the assumption of normality.

Additional points of argument, which you will investigate in this worksheet:

• If the underlying data is normal, then the Wilcox test is nearly as well powered as the t-test.
• If the underlying data is not normal, then the Wilcox test still maintains nominal p-value coverage, whereas the t-test might lose this guarantee.