Articles in this section

See more

How to Calculate Penalty Analysis in Q

  • Updated
Follow

 

Penalty analysis is a tool used to work out which attributes of a product have the greatest effect on how much people like it. For example, if our product is a chocolate cookie, which of these attributes – crunchiness, flavor, or coating affect – have the biggest impact on how much people like the cookie?

Respondents are asked to rate how much they like the product, often on a 9-point scale. Then, respondents are asked about a set of specific attributes of the product and asked to rate them on the basis of ‘too much’, ‘just about right’, or ‘not enough’. As usual, these scales vary.

Penalty analysis calculations take this data and aim to work out which of the attributes cause the biggest drop-offs in how much people like the product when an attribute is “too much” or “not enough”. This is called the ‘penalty’. In this post I’ll show you how to do some common penalty analysis calculations in Q using R.

 

Requirements

The variables for your just-about-right scale (JAR) must be combined as a Pick One – Multi question, and for this calculation they must be grouped as three categories. The order of the categories must be “Not enough” on the left, followed by “Just about right”, followed by “Too much”. The resulting table should look like the one below.

mceclip0.png

Method

Prepare Your Data

If your variables are not already combined as a Pick One – Multi question:

  1. Select the variables in the Variables and Questions tab.
  2. Right-click and choose Set Question.
  3. Choose the Question Type as Pick One – Multi and click OK.

If your scale has more than three categories, you may need to group them together:

  1. Highlight the column labels to group.
  2. Right-click and select Merge.
  3. Enter the appropriate name and click OK.

The “liking” scale should be set as a Number question, and the table should look like the one below.

liking-scale.png

If you need to change the Question Type, find the question in the Variables and Questions tab and change the Question Type drop-down to Number.

Create your ‘Just-About-Right’ table

All the statistics you need to compute the penalties can be created as follows:

  1. Select your Just-about-right scale in the Blue drop-down menu.
  2. Select your Liking score in the Brown drop-down menu.
  3. Right-click on the table and select Statistics – Cells, and ensure the Average and Row Population statistics are selected.

Your table should look like this:

jar-vs-liking.png

The Averages show the average liking score among people who consider each attribute “Not enough”, “Just about right”, and “Too much”. The Row Population shows the weighted sample size for each of these groups.

Finally, to make the calculations easier:

  1. Right-click the name of the table in the Report.
  2. Select Reference name.
  3. Change the reference name to jar.scores.

This determines how we can refer to this table of results when doing calculations in R.

Calculate the total penalty

Calculate the penalty by working out how much the average liking score drops between “Just about right” and “Not enough”, and between “Just about right” and “Too much”. These drops are weighted by the proportion of respondents in the “Not enough” and “Too much” categories and then added together to give the total penalty for each attribute.

To compute the total penalties we can use a little R code:

  1. Select Create > R Output.
  2. Paste in the code below.
  3. Click Calculate.

The code for the penalty is as follows

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
input.table = jar.scores
scores = input.table[,1:3,1,1] # Get the average scores
pops = input.table[,1:3,1,2] # Get the weighted sample sizes
 
sum.pops = rowSums(pops) # Compute the total sample for each row
 
# Work out the drops in average score between just-about-right and too much
# and just-about-right and not enough for each row. Values less than zero
# are set to zero
diff.low = rep(0, nrow(scores))
diff.high = rep(0, nrow(scores))
for (row in 1:nrow(scores)) {
   diff.low[row] = max(scores[row, 2] - scores[row,1], 0)
   diff.high[row] = max(scores[row, 2] - scores[row, 3], 0)
}
 
# Compute the proportion of the sample in the "not enough"
# and "too much" group  for each row
prop.low = pops[, 1] / sum.pops
prop.high = pops[, 3] / sum.pops
 
# Compute the penalties, weighted by proportions
penalty.low = prop.low*diff.low
penalty.high = prop.high*diff.high
 
# compute the total penalty
total.penalty = penalty.low + penalty.high

This will produce a table like the following, showing which attributes have the biggest penalty.

total-penalties.png

To make a visualization of this:

  1. Select Create > Charts > Visualization > Bar Chart.
  2. Select this table (called total.penalty) under Inputs > Data Source > Output.
  3. Tick Automatic.
  4. Select formatting options in the Chart section of the options on the right.

total-penalty-chart.png

Chart penalty vs % of consumers

It is also important to consider the penalties in comparison to the proportion of the sample who regard the product as being “not right” according to each attribute. This is the percentage of people who rated each attribute as either “too much” or “not enough”.

For this calculation we scale each penalty by the proportion of people who rated that attribute as “not right”, and we plot this weighted penalty against that percentage.

To work out the proportion of respondents who rated each attribute as “not right” and calculate the weighted penalties:

  1. Select Create > R Output.
  2. Enter the code below.
  3. Click Calculate.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
input.table = jar.scores
scores = input.table[,1:3,1,1] # Get the average scores
pops = input.table[,1:3,1,2] # Get the weighted sample sizes
 
sum.pops = rowSums(pops) # Compute the total sample for each row
 
# Work out the drops in average score between just-about-right and too much
# and just-about-right and not enough for each row. Values less than zero
# are set to zero
diff.low = rep(0, nrow(scores))
diff.high = rep(0, nrow(scores))
for (row in 1:nrow(scores)) {
   diff.low[row] = max(scores[row, 2] - scores[row,1], 0)
   diff.high[row] = max(scores[row, 2] - scores[row, 3], 0)
}
 
# Compute the proportion of the sample in the not enough
# and too much group for each row
prop.low = pops[, 1] / sum.pops
prop.high = pops[, 3] / sum.pops
 
# Compute the penalties, weighted by proportions
penalty.low = prop.low*diff.low
penalty.high = prop.high*diff.high
 
# compute the total penalty
total.penalty = penalty.low + penalty.high
 
# work out the percentage of people in either too much or not enough
not.right = prop.high + prop.low
 
# Scale each penalty by the proportion of respondents who rated that category
# too much or not enough
weighted.penalty = total.penalty / not.right
 
# Combine the two together
penalty.not.right = cbind("Not right" = not.right * 100, "Penalty" = weighted.penalty)

The resulting table will look like this:

penalty-not-right-table.png

To visualize the results:

  1. Select Create > Charts > Visualization > Scatterplot.
  2. Select this table (called penalty.not.right) under Data Source > Output.
  3. Select Chart > Show Labels > On Chart.
  4. Tick Automatic.

penalty-not-right-chart.png

Product attributes which are top-most and right-most present the most concern as they both have a large drop-off on the liking scale and have the largest proportion of people who feel the product is “not right” in this area.

 

See Also

How to Learn R

How R Works Differently in Q Compared to Other Programs

How to Use Different Types of Data in R

How to Work with Data in R

How to Work with R Loops

How to Reference Different Items in Your Project in R

How to Work with Conditional R Formulas

How to Add a Custom R Output to your Report

How to Create a Custom R Variable

How to Use R to Add Data Sets

 

 

Related articles