How To Replicate SPSS Custom Tables Significance Tests

Introduction

This article describes how to replicate the statistical assumptions settings of SPSS Custom Tables Significance Tests.

Method

1. Select the tables you wish to have the testing applied to in the Report tree, and change Show significance to Compare columns.
2. With the tables still selected, go to Automate > Browse Online Library > Significance Testing >Significance Testing in Tables - Independent Samples Column Means and Proportions Tests.
3. If you get a warning message, press OK. Then go to the Rules tab below the table, select the rule added in step 2, and press Edit rule.
4. If you're using the SPSS Custom Tables setting of Estimate variance only from the categories compared, set Numeric data variance assumptions (t-test) to Equal variance (categories compared). Otherwise, set it to Equal variance (all non-ignored categories).
5. Set the Multiple Comparison Correction to whichever of Bonferroni or None is selected in SPSS.
6. If you wish to specify particular comparisons (e.g., comparing to the NET), check Set column comparisons (eg: A/B, C/D) and modify the comparison as appropriate.
7. If you wish to set up the testing to apply as a default for all items in the table, select Apply > Add to every item in the project and check New items. Note that you should not do this if you are modifying the comparisons (see the previous point), as these will need to be specified differently depending on the number of columns in each table (to do this, apply the rule again from the Online Library as in Point 2 above).
8. Set the other options as shown below and press OK.
9. Press Table (bottom-left corner) to see the resulting table.

Caveats

1. SPSS Custom Tables is not meant to be used with sampling weights. To quote from its documentation (Version 23): "The case weights (or frequency weights) are supposed to be integers representing number of replications of each case. In column proportions tests, we will only check if the column marginal 's are integers. If not, they will be rounded to the nearest integer." Q does not replicate this rounding. This may cause line-ball results to change.
2. SPSS Custom Tables does not perform tests where the percentages in a group are either 0% or 100%. Q does perform tests in this situation.
3. Q does not attempt to replicate SPSS's treatment of multiple response data in statistical tests.
4. SPSS Custom Tables does not provide detailed diagnostics (e.g., p-values, t-statistics, z-statistics), which are necessary in order for us to verify that our computations replicate its computations. Consequently, while we have performed tests, there may be other scenarios where Q's results differ from SPSS's results.

Next

How to Replicate SPSS Significance Tests in Q