How to Change Significance Levels for Column Comparisons

By default, the Column Comparisons in Q use:

Uppercase letters for results that are significant at the 0.001 level (99.9%) and lower-case at the 0.05 level (95%). That is, uppercase results are more significant.
The False Discovery Rate Correction when determining whether or not to show any letters (but not in determining which are lowercase and which are uppercase).

This is different to the norm within the industry, which is to test at the 0.05 and 0.1 levels (i.e., 95% and 90% confidence levels). If Q's settings are left at their default levels, the results reported in Q will tend to be more robust than those that are computed in many other programs, but fewer results will be reported as being significant.

Method

Changing the significance levels

Settings governing how Q tests are set in the Statistical Assumptions menu. These settings can be changed for an entire project or a specific table. To change the settings for a selected table, select the table in the Report tree, then go to Edit > Table Options > Statistical Assumptions. Or right-click the table and select Table Options.

To change the settings for the entire project, go to Edit > Project Options > Customize > Statistical Assumptions.

For example:

Use the Significance levels and appearance grid to control the level of significance associated with different column letters. That is, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lower-case fields (i.e., a,b,c,...,) for 0.1 and 0.05.
Use Overall significance level (top-left) to set the cutoff P value to whatever you want. For example, you could set it to 0.10 (i.e., the 90% level of confidence)
Use the Exception Tests dialog to change the Multiple comparison correction setting to None.
- Note: In Q 5.14 and onward, these two settings are in the Statistical Assumptions > Exception Tests and Statistical Assumptions > Column Comparisons tabs within the Statistical Assumptions menu.