Common Errors in Weights

You may run a survey that does not have any respondents in a particular target weighting group. In this case, it is impossible to create the weight because there is no respondent to apply the target weight to. If a category or combination of categories is empty in the data file, then it is logically impossible for that category to represent more than 0% of the data when weighted. As a result, if you assign a target value to that category that is not zero then the weight cannot be calculated.

In this situation, you will see a message like:

This message tells us that there are two categories in the data that are empty, Males, Less than 18, and Females, Less than 18, but that these categories have targets assigned to them. Approaches to solving this problem include:

1. Enter target values of 0 for the empty categories. This will require the other targets to be adjusted appropriately according to the design of the research.
2. Merge the empty categories with other categories. Merging is done by dragging and dropping categories in the table where weight targets are entered.

Example

For example, let's say you attempt to create a weight with the following variable. Notice that the first category has zero cases. 

Now, suppose you attempt to create the following weight which includes the empty categories:

Notice the error message at the bottom along with a suggestion about how to correct it, which in this case is to set the targets for the empty categories to 0.  

Another solution is to merge the empty categories with other categories that have sample. Merging is done by dragging and dropping categories in the table where weight targets are entered.

Rim Weighting does not Converge

When the weighting involves multiple adjustment variables Q uses an algorithm called Rim Weighting or raking to estimate the weight for each respondent so that each of the different sets of targets is achieved. Because each set of targets is independent of the others, the situation can arise where one set of weight targets contradicts one of the other sets of targets by calculating two different sample sizes for the same group of respondents. When a contradiction like this occurs, it is logically impossible to calculate an appropriate weight.

In cases of this kind you will see the following message:

Diagnosing and Solving the Problem

If you have used more than two adjustment variables then the first step to solving the problem is to identify which pair of questions is in conflict. It is possible that three or more questions can conflict with one another. The following process will allow you to identify which adjustment variables are contributing to the conflict:

1. Use Insert Variables > Weight to create your weight, and include as many weight sets in the weight as possible without getting an error in the Diagnostics Report at the lower right.
2. Save the variable by clicking the New Weight at the lower right.
3. For each remaining adjustment variable:
   
   1. Right-click the weight variable in the Variables and Questions tab and select Edit Weight.
   2. Complete the weight, ensuring that all targets are entered for all adjustment variables.
   3. Remove the current adjustment variable.
   4. Check the Diagnostics Report at the bottom right. If an error is still generated then you know that the adjustment variable that you removed is not the source of the conflict in the weight. If an error is not generated then the adjustment variable that has been removed is contributing to the conflict.
   5. Repeat these steps as needed to identify the source of the conflict.

The final step is to try and identify the reason why the questions that have been identified above conflict with the weight targets. There is no general solution, and the problem can be tricky to identify. Some trial-and-error is required. One approach is to:

1. Create a cross-tab between each pair of questions identified above, and show the n in the Statistics - Cells.
2. Examine the sample sizes in the tables and try to identify those which are particularly small (e.g. n = 1 or n = 2) as these are most likely to present problems.
3. Create your weight again.
4. In the Edit weight window, merge the category with the small sample size with another category, enter a target for the combined category, and then check the Diagnostics Report. If the same error still occurs then a different combination of categories is required, and the process should be repeated. The choice of category to merge will depend both on what makes the most sense from the point of view, and which combination of categories solves the issue.

Example

The following example illustrates a mismatch of weight sets which is particularly extreme. Here, a weight is constructed from three weight sets, one for each of Age, Gender, and Income. The weight generates the error message shown above.

The first step is to try to determine which questions are at fault by creating the weight with each of the weight sets left out. Doing this, we find that leaving Gender out of the weight set still results in the same error, but removing either Age or Income does not. From this, we can conclude that Age and Income conflict with one another.

The next step is to inspect Age and Income to determine where the fault lies. Creating a cross-tab and showing Statistics - Cells > n shows us that there is a single respondent who is in the 50 or more age bracket and $45,001 to $60,000 income bracket. This is the source of the contradiction in the targets.

The targets for age predict that this person should account for 20% of the weighted sample, whereas the targets for income predict that this person should account for 30% of the weighted sample. Since these two conditions cannot be true at the same time, there is no way to calculate a weight for these targets. The solution is to merge one of the trouble categories with another so that this respondent is no longer the sole occupant of two categories with different targets. In this case, it might be appropriate to merge $45,001 to $60,000 with Less than $40,000 or merge 50 or more with 35 to 49. Both of these options will allow the weight to be computed.