Level Balance and Opt Outs

[dsr]

I wonder if you could comment on a discussion with a colleague over level balance between attributes in a study we are preparing.

Design =3 attributes (T, H, cost)
Each attribute has 5 levels (zero, plus 4 non-zeros)
Sets = 2 treatment options plus a None option

At present zero values for all 3 attributes only appear in the None option

We are considering changing the design so that zero level for H appears among the treatment options as well as in None.
(analytically I would prefer to have zero levels of T and Cost appearing in all options but it makes no sense)

If we make the change:
4 levels of T and cost will appear in the 2 treatment options.
5 levels of H will appear in the 2 treatment options.

Does this change anything regarding level balance?

I don't think it does – there are still 5 levels of all attributes, even if for 2 of them the zeros are confined to the none option.
My colleagues disagree.
They say that in their discipline (health econ) the “attributes and levels” descriptions in papers exclude the opt out zeros, and hence H, T and C would not have the level balance property.

Can you advise - or do we resort to an arm wrestle on the issue?

thanks,
Dan

[johnr]

Hi Dan

There is quite a bit of confusion related to attribute level balance and it is mostly our fault. Given our background in Transportation, we are used to labeled experiments, whereas the vast majority of work in expeirmental design outside of our own research is comming from marketing (and to a lesser extent environmental and health economics), and has exlcusively dealt with unlabelled experiments. The difference is that when dealing with labelled experiments, we consider attribute level balance to occur within an alternative (e.g., the levels for bus fare will be balanced; the levels for train fare will be balanced). In the case of unlabelled alternatives, the literature allows attribute level balance for an attribute to occur across alternatives (e.g., the levels for fare can appear an equal number of times across alternatives A and B, and not necessarily just within the fare column for A, and within the fare column for B).

With the above background, attribute level balance is a function of the number of choice tasks, the number of levels, and the number of alternatives in the design. Consider the following an unlabelled experiment with two alternatives, each with three attributes with A{0,1,2}, B{0,1} and C{0,1,2}, and six choice tasks. The following design is considered to be balanced.

Set A1 B1 C1 A2 B2 C2
1 0 0 0 0 0 0
2 1 1 1 1 1 1
3 2 0 2 2 0 1
4 2 1 2 0 0 1
5 2 1 0 0 1 0
6 1 1 2 1 0 2

This is because if you treat A1 and A2 as the same then 0 appears 4 times, 1 appears 4 times and 2 appears 4 times; For attribute B (across B1 and B2), 0 appears 6 times, and 1 appears 6 times; For C (across C1 and C2) 0 appears 4 times, 1 appears 4 times, and 2 appears 4 times.

Note that if my design had only 5 sets, then I could not obtain attribute level balance over the design.

Note that I concur with your collegues that the no choice typically is ignored in any of these calculations also.

John