Orthogonal design with only contextual variables

[han.li]

Dear Prof. Michiel Bilemer,

I want to estimate the impact of contextual (scenario) attributes on participants preference over several comfort adaptive behaviour. Currently, I have 8 contextual attributes, 1 with 4-level and 7 with 3-level, e.g., Weather: sunny, windy, rainy; Temperature: Less than 10 , 18-26, more than 28.

Basically, I need an orthogonal variation of contextual (scenario) attributes so as to describe the scenario (indoor environmental single- and/or multi-mode discomfort) in text or visualize them later in the survey. The possible solution to restore the indoor comfort includes: window usage, thermostat (heating/cooling or AC) adjustment, blinds usage, and lighting apparatus usage.

I followed the steps in chapter 6 of manual 1.3, but the syntax was tested with 1.2.1 version. (Ngene 1.2.1 was installed on an air-gapped PC in our department with licenses. Ngene 1.3 (evaluation version) was installed on my work PC.)

I set the U(alt2) as 0, without it, Ngene shows error.
$Design1
Design
;alts = OB, alt2
;Rows = 36 (Ngene ver. 1.2.1) or 72 (Ngene ver. 1.3)
;orth = sim
;model:
U(OB) = b3*B[-2, -1, 1, 2] + b4*C[-1, 0, 1]+b5*D[-1, 0, 1]+b6*E[-1, 0, 1] + b7*F[-1, 0, 1] + b8*G[-1, 0, 1] + b9*H[-1, 0, 1] + b10*I[-1, 0, 1]/
U(alt2) = 0
$
Question: If the set-up and syntax is correct, and contextual attributes can be considered as the same for normal attributes.
I encountered the following issues: With Ngene 1.2.1 both 36 rows and 72 rows can be found, however, if using the Ngene 1.3 (evaluation), only one design can be found with 72 rows. Are there any significant change made in Ngene 1.3 result in the differences? Does this means 36 rows design with 1.2.1 version is not a valid design? Or there is something wrong about the syntax, or the experiment set-up (pure contextual attributes) is not applicable in Ngene.
Also, I am not sure if by setting the U(alt2) to 0 might cause any potential problems in this case. Can I consider the alt2 as no-preference? Means, with described scenario, none of the comfort adaptive behaviour is preferred by the participants.
$Design2
Design
;alts = OB, alt2
;Rows = 72
;orth = sim
;model:
U(OB) = b3*B[-2, -1, 1, 2] + b4*C[-1, 0, 1]+b5*D[-1, 0, 1]+b6*E[-1, 0, 1] + b7*F[-1, 0, 1] + b8*G[-1, 0, 1] + b9*H[-1, 0, 1] + b10*I[-1, 0, 1] + b11*B*C + b12*B*D + b13*B*E + b14*B*F + b15*B*G + b16*B*H + b17*B*I + b18*F*G + b19*F*H + b20*F*I + b21*G*H + b22*G*I + b23*H*I/
U(alt2) = 0
$
Question: Does design2 syntax correctly gives me the following two types of interaction effect:
The first one is grouping. B is the contextual attribute representing 4 groups. Since the rest of the attributes are all contextual, can I simply multiply them with B?
The second type of interaction is the interaction within the attributes (C, D, E, F, G, H, I). For instance, multi-mode discomfort situation, if attribute F - level 1 (temperature – above 30 degrees) and attribute G – level 1 (stuff air) appears at the same, they will have a higher impact on comfort adaptive behaviour (in this case window usage is preferred more than thermostat adjustment), compared to single-mode discomfort situation, i.e., either F level 1or G level 1 alone.

What might be your advice and suggestions for above doubts?
Thank you for your time! Have a nice day!
Best regards,
Han

[Michiel Bliemer]

1. Yes there has been a change in orthogonal designs. Specifically, in version 1.2.x we included near-orthogonal designs, whereas in version 1.3.x and further we only consider orthogonal designs. In this case, the smallest orthogonal design is 72. In some studies a near-orthogonal design leads to issues, so to avoid such issues we no longer generate near-orthogonal designs (which have uncorrelated attribute levels but some attribute level combinations are not present). So I suggest you only use truly orthogonal designs.

2. Ngene does not accept U(alt2) = 0. It accepts at minimum a constant, e.g. U(alt2) = b0. If it should be equal to 0, then you can simply omit it when specifying. In other words, just specify U(OB) = .... and then end with "$", without specifying U(alt2). In that case, U(alt2) automatically defaults to zero. Note that U(alt2) or U(alt2) = b0 is consistent with alt2 being an opt-out alternative. It does not mean "no-preference" since that would mean equal utility.

3. You need to specify the utility functions exactly as how you would estimate them in estimation software. Since your variables are categorical, you need to use dummy or effects coding, such as b1.dummy[0|0|0] * A[1,2,3,0], where 0 is the base level. Interactions between categorical variables should not be just A*B unless A and B are numerical variables. For categorical variables, you need to include interactions across each individual level (except the base level), such as:
i1 * A.dummy[1] * B.dummy[1] + i2 * A.dummy[2] * B.dummy[1] + i3 * A.dummy[3] * B.dummy[1] + i4 * A.dummy[1] * B.dummy[2] + i5 * A.dummy[2] * B.dummy[2] + etc. So think about how you are going to estimate the model. Making interactions with categorical variables can lead to a very large number of parameters.

Michiel

[han.li]

Dear Prof. Michiel Bliemer

Thank you for your help and suggestions. It is all clear and make sense now.

Best regards,
Han

[han.li]

Dear Prof. Michiel Bliemer,
Thanks in advance for your time.
I update the utility funtion and write the following syntax:
Design
;alts = OB, alt2
;Rows = 72
;orth = sim
;model:
U(OB) = b1.dummy[0|0]*A[1,2,3] + b2.dummy[0|0]*B[1,2,3] + b3.dummy[0|0]*C[1,2,3] + b4.dummy[0|0]*D[1,2,3] + b5.dummy[0|0]*E[1,2,3] + b6.dummy[0|0]*F[1,2,3] + b7.dummy[0|0]*G[1,2,3] + b8.dummy[0|0|0]*H[1,2,3,4] +
i1*A.dummy[1]*D.dummy[1] +
i2*A.dummy[2]*D.dummy[2] +
i3*B.dummy[1]*D.dummy[2] +
i4*B.dummy[1]*F.dummy[1] +
i5*B.dummy[2]*E.dummy[1] +
i6*B.dummy[3]*F.dummy[1] +
i7*C.dummy[2]*E.dummy[1] +
i8*C.dummy[1]*E.dummy[1] +
i9*C.dummy[2]*D.dummy[1] +
i10*C.dummy[2]*D.dummy[2] +
i11*D.dummy[1]*E.dummy[2] +
i12*D.dummy[2]*E.dummy[2] +
i13*F.dummy[1]*G.dummy[2] +
i14*F.dummy[2]*G.dummy[1] +
i15*F.dummy[1]*G.dummy[1] +
i16*F.dummy[2]*G.dummy[2]
$
Regarding above syntax I have two questions:
1. May I specify an interaction effects for categorical variables like Figure 6.7 (page 73 of NgeneManual120) suggested that "specified two-way interaction effects". In your last comment you suggest to include interactions across all individual level of categorical variables (except base level).
2. Is correct to write the syntax i6*B.dummy[3]*F.dummy[1]? As 3 is B's base level, but it interact with categorical F's non-reference level (i.e., 1). (both B and F are categorical).

Thank you for your time!
Best regards,
Han

[Michiel Bliemer]

Yes you can specify an interaction as x1*x2, but then it assumes that the data is numerical, so in your case it would simply multiply for example B*D, where B is 1,2,3 and D = 1,2,3. Since 1,2,3 are only labels of categories, it is in most cases not appropriate to multiple these labels together as this is meaningless.
So with categorical variables you would multiply individual levels, such as B.dummy[1]*D.dummy[2], where B.dummy[1] equals 1 if B has level 1 and 0 is otherwise, and D.dummy[2] equals 1 if D has level 2 and is 0 otherwise.

As yes, i6*B.dummy[3]*F.dummy[1] is fine.

Michiel

[han.li]

Dear Prof. Michiel Bliemer,
Thank you for your reply. Maybe I am not making myself clear.

Apart from the example i6, all other interaction mentioned in the previous syntax are interaction between categorical variable's specific individual level.

Am I understand correctly that interaction effect for categorical variable can be defined by multiply individual levels, if they follow the example you given in previous comments?

i.e., "With categorical variables you would multiply individual levels, such as B.dummy[1]*D.dummy[2], where B.dummy[1] equals 1 if B has level 1 and 0 is otherwise, and D.dummy[2] equals 1 if D has level 2 and is 0 otherwise."

For now, if I interact all the individual levels of two categorical variable, some of them, in general situation, is unrealistic. For instance, temperature in the summer is highly unlikely to fall below 5 degree Celsius.

Thank you for your time!
Best regards,
Han

[han.li]

Dear Prof. Michiel Bliemer,

Supplement explanation for my previous reply.
Instead of specify each individual level, if I only specify interaction effect for categorical variable with individual level (e.g., only i1*A.dummy[1]*B.dummy[2]), would this make the later estimation cause any bias in the results?

Best regards,
Han

[Michiel Bliemer]

You would only include interactions that you believe will be relevant for your model. This can be all interactions, some interactions, or no interactions.

I am not sure that I understand that interaction effects can be "unrealistic" where you give the example that temperature in the summer is unlikely to fall below 5 degrees. Interactions have to with how preferences of one attribute are affected by the level of another attribute. For example, travel time in a bus may depend on seating availability in the bus, and then you would use b1*traveltime + b2*traveltime*seating.dummy[1]. So you will need to think about what your interaction effects mean.

No matter what interactions you add (or omit), it will not cause biases in model estimation. The only thing that it impacts is the efficiency in data collection. If you add interaction effects during the design phase and you also put them in your model during the estimation phase, then your data collection is most efficient. However, you will in most cases will be able to estimate all interaction effects even if you did not optimise for them in the design phase.

Michiel

[han.li]

Dear Prof. Michiel Bliemer,
Thank you for your explanation.
I tried to interact with categorical variables with all individual levels (except base level, as you suggested) and the number of estimates is quite large (option 1). Then I think what if I only keep the one we expect to have an significant impact only (option 2).

example: categorical A and B b1.dummy[0|0]*A[1,2,3] b2.dummy[0|0]*B[1,2,3]

Option 1 - all individual levels are specified, but only A2+B2 is of our interests: i1*A.dummy[1]*B.dummy[1]+i2*A.dummy.[1]*B.dummy[2]+i3*A.dummy[2]*B.dummy[1]+i4*A.dummy[2]*B.dummy[2]

Option 2 - only 1 is specified and will estimate later: i1*A.dummy[2]*B.dummy[2]

Which one would be the correct input for Ngene?

Thank you for your time!

Best regards,
Han

[Michiel Bliemer]

Option 2 would be best.

There is only a limited amount of information that you can get from a choice experiment, and by specifying interaction effects that you will not use you will take away some information from other effects when you optimise the design for efficiency. So it is best to only include interaction effects that are of interest, as you say just level 2 of A interacted with level 2 of B without including the other levels of A and B in interaction.

Michiel