How to Change Q's Default Statistical Assumptions When Setting Up Projects

Introduction

If you, your organization, or your client has a specific set of statistical assumptions they'd like to see, you can do it once and save it as a template for future projects.

Setting defaults in a project

The default statistical assumptions in Q:

• Are conservative. In particular, the False Discovery Rate Correction is automatically applied on all tables.
• Focus on using statistical testing as a way of highlighting results that are in some sense exceptions, and displays these results using colors and arrows (e.g., )

The defaults can be modified using Edit > Project Options > Customize >Statistical Assumptions.

Common modifications include:

• Changing Show significance to Compare columns.
• Changing the Overall significance level from 0.05 to 0.10 (i.e., from the 95% level of confidence to 90%).
• Set the Minimal sample size for testing to 30.
• Changing both Multiple comparison correction settings to None.
• Either:
  • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), which means that Q will only use UPPERCASE letters when showing significance.
  • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), except those for the rows with a Cutoff p-value of 0.005, 0.01 and 0.05. In conjunction with changing the Overall significance level, this means that Q shows lowercase letters for results significance at the 0.10 level and UPPERCASE for 0.10 and higher.

If wanting to replicate results from other programs it will be necessary to adjust more options (see Results Are Different to those from Another Program). However, use caution when modifying the settings, as Q's default statistical tests are, in general, safer than those in more traditional survey programs. In particular, they deal better with weighting and edge cases, such as if testing against a cell containing 0% or 100%).

Applying the defaults to other projects

Modifications that are made to Statistical Assumptions can be saved as Project Templates, and automatically applied to new projects.

Overriding Q's statistical testing

Rules can be used to modify how Q performs statistical tests. For example, a rule (see How to Apply Independent Samples Column Means and Proportions Tests to a Table) can be used to perform tests that assume samples are independent, even when the underlying data are not independent. Considerable caution should be undertaken when performing using rules to over-ride Q's inbuilt statistical tests:

• The rules are less sophisticated than the in-built tests (i.e., they have fewer checks and balances).
• Rules are essentially modifications of the core functionality of Q and can have unintended consequences (e.g., if inadvertently applied to the wrong table, or, if using them in ways not anticipated when they were created).

Next

How to Modify Significance Tests

How to Show Statistical Significance in Q

How to Modify Significance Tests Using Rules

How to Do Planned Tests of Statistical Significance