This article describes how to create weights from variable(s) in Q. Weighting is a technique which adjusts the results of a survey to bring them into line with some known characteristics of the population. For example, if a sample contains 40% males and the population contains 49% males, weighting can be used to adjust the data to correct for this discrepancy.
- Methods
-
Additional techniques
- Combining categories in an adjustment variable
- Creating two-dimensional weights
- Recomputing weights for each category in a variable (e.g. wave-based weighting)
- Making weights sum to sample size or target values
- Selecting a design weight variable
- Specifying upper and lower bounds for weights & specifying alternative weighting algorithms
- Saving a weight variable
- Applying weights
- Troubleshooting errors in weights
Requirements
- One or more variables in your data set that you want to use to as adjustment variables. These variables can be categorical or numeric.
- The targets for each category in the weight variable. These can be percentages or population counts.
- Pick One or Number questions that can be used as adjustment variables. You cannot use multi-choice questions.
- Familiarity with weighting survey data. See: How to Weight a Survey
Methods
Specifying a single adjustment variable
- Go to the Variables and Questions tab, right-click, and select Insert Variable(s) > Weight
- Use the Adjustment Variable(s) box menu to select a categorical or numeric variable you want to weight by. Then specify whether your targets are Percentages or Population/Count (see Making weights sum to sample size or target values).
If your weight targets are Percentages, the total must sum to 100. When using Population/Count, this indicates the target numbers are populations (also known as counts). If Population/Count are used as targets across multiple adjustment variables, they must all have the same total population/count. - Click the
button once it becomes enabled to add the new weight to your data set.
- If the button is greyed out or disabled, check the Weight diagnostics at the bottom right for any errors or issues.
Using targets on percentages
If your desired targets are 60% Male and 40% Female, you would specify 60 for "Male" and 40 for "Female" as below:
Using targets based on population/count
According to U.S. population estimates, in 2020 there were 162.83 million males and 166.24 million females. Using targets based on a JavaScript formula
(Note: This option is only available in Q v5.17.1 or higher.)
Your JavaScript may be one of either:
- A formula that uses the recompute question values as an input (t), and outputs a number.
- For example, the recompute question "Wave" has 5 waves, with numeric values 1-5. The target for "Male" was 1,000 up to Wave 3, when it changed to 1,500 due to an increased sample. The JavaScript formula for "Male" would be:
if (t < 3) 1000; else 1500;
- A list of target numbers, one for each value of the recompute question. The type of list you enter depends on whether your recompute question is a Date question or a Pick One question.
- Date: enter a list that contains a target for every date period. For example, if the date question represents 5 months, your list will need 5 targets:
[1000, 1000, 5000, 1500, 1500]
-
- Pick One: enter a list that contains targets for each category's numeric value (the same numbers you see in Source Values column of Value Attributes), starting from 0 and finishing at the highest value in the data. This will only work when the values are positive whole numbers (otherwise, you must use a formula, like the above). If the values do not start at 0 or are not contiguous, you must enter a target of '0' for the missed values. For example, if a "Wave" question had the category values 1-5, your list would have 6 items:
[0, 1000, 1000, 1500, 1500, 1500]
Specifying additional adjustment variable(s)
If you want to weight by more than one variable, click the Additional adjustment variable set button and select the variable you want. When you add a second adjustment variable, Q will start using Rim weighting or raking. For example, if you want to not only weight by Gender but also how frequently respondents drink coffee, do the following:
- Click
- Add the desired variable, which in this case is Coffee:
Weight diagnostics
In the lower-right of the window, additional information about the weight is displayed next to the button:
These values are as follows:
- Minimum: Minimum weight value
- Maximum: Maximum weight value
- Maximum / Minimum: the ratio between the maximum and the minimum
- Effective Sample Size: Effective Sample Size (ESS) is an estimate of the sample size required to achieve the same level of precision if that sample was a simple random sample.
Additional techniques
Combining categories in an adjustment variable
You can drag and drop the categories of a variable together to combine them within the weight tool. For example, in the Coffee variable, if you wanted to combine 4 to 5 days a week with Every or nearly every day, simply:
- Click the Every or nearly every day category
- Drag and drop it on the 4 to 5 days a week category.
Note: your percentages will revert to 0's when you drag and drop one category on another so you will have to specify them again if you haven't already.
Creating two-dimensional weights
Additionally, within an adjustment variable in the weighting tool, you can add additional variables to create two-dimensional weights. Use the Select Adjustment Variable(s) drop-down to the right of where you selected your adjustment variable.
For example, if you wanted to weight your data according to Income by Gender, you would specify:
Recomputing weights for each category in a variable (e.g. wave-based weighting)
In some cases, you want to calculate a weight based on group membership or time period. For example, in longitudinal or tracking studies when the data is collected in waves, it may be necessary to recompute the weight for each date period (week, month, etc.). That way each wave uses the same weighting targets, but each wave's weight is calculated separately.
By default, weights in Q are computed for the entire data set. Using the Recompute feature, you can enter the targets that will be used for each wave, rather than the total sample. To do this, use a Pick One question that has each "wave" as a category.
As with all Q's weighting, the weights are recalculated whenever new data is imported, so this option enables the weighting tool to automatically compute new weights for new waves as they appear in the data.
To recompute weights by category:
- From the Recompute weights for menu, select the grouping variable. In this example, the variable is called Interview Date
The categories of the variable selected in that box will be used to divide up the cases and the weight will calculate separately for respondents in each of those categories.
Making weights sum to sample size or target values
When sample size is selected in this drop-down, Q will ensure that the final weight produces Population sizes that sum to the data file's sample size. When target values is selected, the numbers entered as targets are used to produce the final weight, so the total Population on tables will equal the sum of the population targets entered in the weighting tool.
In this example, the unweighted sample size is 327 but the weight has targets of 162,830,000 for men and 166,240,000 for women, so the sum of the targets in the weighting tool is 329,070,000.
Selecting a design weight variable
Design weights are used to compensate for non-proportional stratification. For example, in a population containing 100,000,000 men and 100,000,000 women, if you used quotas or stratification to achieve a sample of 80 men and 20 women, then a design weight is created to take this non-proportional stratification into account. To learn more about Design Weights, click here.
To use a design weight in Q, use the Select Design Weight Variable drop down box to select the design weight variable in your data set.
Specifying upper and lower bounds for weights & specifying alternative weighting algorithms
On the lower left, the Minimum weight and Maximum weight boxes allow you to specify upper and lower bounds for your weights. This technique is called trimming and is designed to ensure that the weight factors are calculated to within certain bounds. For example, if you if you don't want weights larger than 3, then set maximum to 3 and minimum to 0. The bounds you set are used during the optimization process and not just on the final result. Given the complex nature of these algorithms, sometimes seemingly sensible bounds may not converge, and you will get an error message to that effect. In this case, you can try another algorithm.
You can also change the algorithm as well. The choices are Linear, Raking and Logit. The default is Raking. To learn more about how to use trimming and the different algorithms you can read up on How to Improve a Weight and Calibration Weights in our Data Story Guide or click here to watch our weighting webinar.
By default the controls in this section are greyed out and disabled. To enable them, either select two or more adjustment variables, which enables rim weighting/raking, or select a design weight variable.
Saving a weight variable
Once you have an appropriate weight specified, on the bottom-right the button becomes active. Pressing this button takes you back to the main interface and you see a new weight variable in the data set.
Using weights created outside of Q
In addition to building weights in Q, you can use weights calculated outside of Q. See: How to Make Variables Available as Weights for more information.
Applying weights
To apply a variable as a weight:
- Select the output(s) you want to weight on the Outputs tab
- Then choose the appropriate weight variable from the Weight drop-down box at the bottom of the screen.
As with other Q-constructed variables, the weight variable will be automatically updated if you import a new data file. If any of the target categories change in the new data file or new categories are added, the variable's status will become INVALID until the weight is edited and any issues fixed.
Troubleshooting errors in weights
There are Common Errors in Weights which you can resolve in various ways, but if you're having issues with creating your weight or figuring out why results are unexpected please see our How to Troubleshoot Weights article.
Next
How to Construct a Weight for a Self-Weighting Sample
How to Make Variables Available as Weights