This article describes the different methods you can use to show statistical significance in Q. Where a result is computed as being statistically significant it can be presented in a variety of ways.
Method
Changing the Tests on Tables and Charts
The way that tests are shown on tables and charts is determined by the selection in the Show significance dropdown menu at the top-middle of the Outputs tab: .
These tests will always test the Column % for categorical tables or Average for numeric tables - even if a different statistic is shown. When set to No, no significance is shown. When set to Compare columns, column comparisons, or pairwise tests are used. By default Arrows and Font Colors, or exception tests, are used. This is the same as Arrows and Font Colors separately. This setting can be changed for an entire project in the Edit > Project Options > Customize > Statistical Assumptions menu, and for all future projects using Project Templates.
Arrows
On a standard crosstab, arrows represent the results of Testing the Complement of a Cell. NOTE: Starting in Q v5.14 and onward, this type of significance testing is called Exception tests. For discussions of the interpretations of tests on different types of tables, see Reading Tables and Interpreting Significance Tests.
Arrows point up when a result is significantly higher and down if significantly lower. The length of the arrows relates to the degree of statistical significance, as determined by the Corrected p statistic. The specific relationship between the length of arrows and significance is governed by the settings in the Statistical Assumptions menu table of settings called Significance levels and appearance.
Font Color (and Column and Line Color)
In tables and most charts, significant results are color-coded (by default, blue and red; e.g., ). The testing is conducted in the same way as with Arrows, except that a higher result is by default shown in blue and lower in red.
When using Time Series charts, the lines and bars are color-coded to indicate significance if the Font Color option is selected.
Arrows and Font Colors (default)
Both arrows and colors are used. See the two sections above for details.
Compare columns
Letters or other codes are used to indicate significant differences between results in different columns. Alternatively, these can be manually selected by right-clicking on the table and selecting Statistics - Cells > Column Comparisons and Statistics - Below > Column Names.
Q uses Corrected p when determining whether to assign letter or not, and when determining which symbols to apply (e.g., UPPERCASE or lowercase).
Settings regarding the symbols to be shown are in the Statistical Assumptions sections on Significance levels and appearance and Column comparisons.
Other ways of showing statistical significance
Table names
Where Smart Tables are used, tables are classified as being Significant and Insignificant in the Report Tree, with the p-Values shown in the names of the tables.
Reports from planned tests
Significance tests can be conducted by selecting cells and pressing . See How to Do Planned Tests of Statistical Significance.
Multivariate outputs
Regression and Segments analyses of Experiment and Ranking questions contain statistical tests of the parameters in their outputs.
Cell shading and other tools
Custom ways of showing statistical significance can be created using Rules (see How to Use Rules in Q to get started). One example is Significance Testing in Tables - Color Significant Cells. See How to Modify Significance Tests Using Rules for more information.
Next
How to Change Q's Default Statistical Assumptions When Setting Up Projects
How to Interpret "Inconsistent" Statistical Testing Results