This article describes how to determine why the Sample Size (Base n, Column n, Row n) or NET on a table is smaller than expected. The NET can be the overall NET (like in the table below), a custom or span sub-net, or a combined row/column (which is the same as a NET but without also keeping the categories separate). Such as the following table showing a Top 2 Box score across different brands:
Requirements
A table with one of the following that is smaller than expected:
- A combined category (created using Merge on rows/columns of a table)
- A NET (created using Create NET on a table or simply the an overall NET or sub-net on a table)
- A sample size statistic (such as Base n, Column n, Row n)
Knowledge of how NETs and Sample Sizes are computed with missing data. NETs and Sample Sizes are calculated including only common respondents across all the categories being included. It is obvious when there is missing data on a table, when the footnote lists a range, such as "base n = 591-600". To understand the numbers in the table above, adding sample size statistics or Missing n to the table (as seen below) will help you identify what is causing the small number.
In the above table, 9 respondents have missing data for the Pepsi category, thus there are 591 respondents who have data across all categories making the Base n of the table 591. The NET on the table is 96% (567/591) - meaning out of the 591 respondents that answered all categories, 96% of them gave a Top 2 Box score for at least one of the categories. As Column n is the number of people who selected the category in the column (in this instance it is a Top 2 Box score), Column n is the same as the NET of the table at 567.
Increase Sample Size
Method - Uncheck Missing Data
If you believe the missing data is in error and want to include those 9 respondents with missing data for Pepsi in the numbers, right click on the table and select Values to change the Value Attributes. You will need to uncheck Missing Data on all values to include all respondents in the base, similar to below:
However, often this will an unwanted side-effect, whereby the percentages on the table will change to include the larger of the sample sizes. For example, with the data shown above, Pepsi's value of 45.2% is compute as 267 divided by 591, and if its Base n is changed to 600, the percentage drops to 44.5%.
Method - Show the Maximum Sample Size
Q's approach is based on the principle of conservativism - it minimizes the chance that a user of the research will rely upon an unreliable finding. Any approach which puts a higher sample size than what is used by Q runs the risk that users of the research will believe the data to be more robust than is the case. Given this, there is a custom rule you can use, see How to Show Maximum Column Sample Size in a Table to show the maximum Column n at the bottom of the table. For tables where this is not applicable, you can modify the JavaScript code of the rule to use Base n instead, see How to Customize a Rule .
Method - Revise Weights
If your table is being weighted, you should reference Population statistics to see the sample sizes used in the figures and testing on the table. If these numbers appear to be incorrect, review the targets and construction of your weight variable to determine if it needs to be revised. Do note, that if a respondent is given a 0 weight, they will also be removed from the unweighted n statistics as well.
Method - Revise Data
If you have investigated the above options, and you still believe your sample size is too small. You can right click on the variables in question on the Variables and Questions tab and Export Variables to Excel to manually review the raw data in Q. If the sample sizes in Excel match that of Q, you will need to go back to your data provider to investigate.
Make NET = 100%
Method - Rebase to the NET
Rebasing the table to the NET will force the NET to be 100%, see How to Rebase Questions. After running the automation, a new version of the question will be created where respondents not included in the NET will have missing data.
Method - Add a None of These Option
Adding a None of These option will force the NET to be 100%, see How to Add a "None of These" to a Pick Any Question. This will create a new question with a None of These category for respondents who weren't included in the NET originally and thus force the NET to be 100%.
Method - Create a NET Filter
Right click on the NET on the table and select Create Filter. You can then apply the filter to this table to show the NET as 100%.
Also See
How NETs and Sample Sizes are computed with missing data
How to Recode Missing Values in Q