Introduction
This article describes how to go from a standard crosstab or summary grid table...
...to a table with Column Comparisons based on Independent Samples Column Means and Proportions tests:
This rule is also designed to mimic those in introductory statistics courses and in SPSS Custom Tables.
Method
1. Select your table.
2. In the toolbar go to Automate > Browse Online Library > Significance Testing in Tables > Independent Samples Column Means and Proportions Tests.
3. OPTIONAL: By default this rule works on grid-style summary questions, but tick Apply to crosstabs when applicable.
4. OPTIONAL: Tick Apply when Column Comparisons are not shown on the table if you wish to remove the warning which occurs when this statistic is not displayed.
5. OPTIONAL: Tick Set column comparisons to specify which columns of the table should be compared, otherwise leave unticked to compare all columns. When this option is selected, you should enter the names of the columns that you would like to compare in the box provided. Note, span labels are ignored in the table, so you can perform comparisons between columns in different spans, albeit only the lowest-level spans.
In this example, we are comparing columns B through D to column A, so we use the below format:
A/B,A/C,A/D
6. OPTIONAL: Tick Include settings information in footer.
7. Set the Significance Level (alpha) for lowercase letters, e.g. 0.1 (90%).
8. Set Significance Level (alpha) for uppercase letters, e.g. 0.05 (95%).
9. OPTIONAL: Set Multiple comparison correction.
10. OPTIONAL: Set Numeric data variance assumptions (t-test).
11. OPTIONAL: Set Base used in tests of means.
12. OPTIONAL: Set Proportions test.
13. OPTIONAL: Set Base used in test of proportions.
14. Set the Minimum sample size in a column to include in testing.
15. OPTIONAL: Enter the names of Columns to ignore from testing, one per box.
16. Press OK.
17. Select Column Comparisons in the table via right-click > Statistics - Cells.
Please note the following:
- The default formulas used in this rule are not the same as those used by Q to automatically test independent samples. If you wish to use Q's in-built testing for independent samples on data that has a dependent structure, the best way to achieve this is by first Stacking the data file. However, in most situations the default settings in this rule will correspond to the general defaults in Q. Reasons for differences are described in some of the following points.
- This rule works by over-riding/ignoring settings and safe-guards built into Q. Many of the settings that are over-ridden are designed to protect users from problems in their data, so when applying this rule a higher degree of diligence is required in checking results.
- This rule overrides most, but not all, settings in Statistical Assumptions via Edit > Project Options > Customize. In particular:
- Bessel corrections are still used as inputs to computations of Standard Errors and Standard Deviations (i.e., those computed on the tables use this setting, and these then feed into the some of the tests).
- The Weights and significance determines the Effective Base n, which is an input to the Effective sample size used in this rule.
- In various situations, the formulas used for computing significance in this rule may give incorrect/misleading results (e.g., when the assumption of independence is inappropriate and when the various tests selected are wrong).
- Effective sample size is computed as Effective Base n * Column Population / Base Population.
- Weighted sample size is Population.
- When the question selected in the Blue drop-down is a Pick One - Multi, Pick Any - Grid, or a Number - Grid, the sample size used in the test is the Sample Size (or Base Population, or Effective Base n, otherwise the Column n (or the Column Population, or Effective Column n) is used.
- Simplified Independent Complex Samples T-Test - Comparing Two Means is used when Numeric data variance assumptions (t-test) is set to Complex samples.
- Independent Samples T-Test - Unequal Variance is used when Numeric data variance assumptions (t-test) is set to Unequal variance. When this option is selected in conjunction with setting Base used in test of means to Weighted sample size and Multiple comparison correction is set to None, the resulting tests correspond to an an Independent Samples T-Test in SPSS with Equal variances not assumed.
- Independent Samples T-Test - Equal Variance is used when Numeric data variance assumptions (t-test) is set to Equal variance (categories compared). When this option is selected in conjunction with setting Base used in test of means to Weighted sample size and Multiple comparison correction to None, the resulting tests correspond to an Independent Samples T-Test in SPSS with Equal variances assumed and also to Compare column means (t-tests) with Estimate variance only from the categories compared selected in SPSS Custom Tables, except where the columns are from multiple response questions.
- Independent Samples T-Test - Pooled Variance is used when Numeric data variance assumptions (t-test) is set to Equal variance (all non-ignored categories). When this option is selected in conjunction with setting Base used in test of means to Weighted sample size and Multiple comparison correction to None, the resulting tests correspond to Compare column means (t-tests) with Estimate variance only from the categories compared not selected in SPSS Custom Tables, except where the columns are from multiple response questions.
- Simplified Independent Complex Samples T-Test - Comparing Two Proportions is used when Proportions test is set to Complex samples t-test.
- Independent Samples Z-Test - Comparing Two Proportions (Pooled) is used when Proportions test is set to Pooled z-test. When this option is selected in conjunction with setting Base used in test of proportions to Weighted sample size and Multiple comparison correction is set to None, the resulting tests correspond to Compare column proportions (z-tests) in SPSS Custom Tables.
- Independent Samples Z-Test - Comparing Two Proportions (Un-Pooled) is used when Proportions test is set to Un-pooled z-test.
- The footer information relating to statistical testing will relate to Q's default test, and these should be turned off when you use this rule. This is done by editing the Table Options for the relevant tables and un-ticking the options relating to statistical testing in the Footers tab. Information about the settings used in the rule can be added to the table footer by ticking the option in the rule called Include settings information in footer.
Next
How to Apply Significance Testing in Q