Efficient design with SQ alt. having dummy variables

[AnzeJap]

Dear Michiel (I pasted all our conversation from previous thread),

***#1

ATR levels codes
atr1 7, 20, 34 1,2,3 (7 is a reference value)
atr2 0%, 5%, 15% 1,2,3 (0% is a reference value)
atr3 none, owner, other 0,1,2 (effects coded, 0 is a reference value)
atr4 .3%, 5%, 20% 1,2,3 (.3 is a reference value)
atr5 none, record, monitoring 0,1,2 (effects coded, 0 is a reference value)
atr6 0, 150, 300, 450, 600, 750, 900 EUR 1,2,3,4,5,6,7 (0 is a reference value)

Reference values are in SQ alternative, however reference values of all except of atr6 can also populate additional two non-SQ alternatives. This is where I see most resemblance between your and our case. (I hope I understood your research design properly).

We are planning to do a pilot study (based on sequential fractional factorial design), from where priors would hopefully be collected. Those are to be fed into a Bayesian effective design. The code we are constructing builds also on your correspondence with Michiel B. on this forum.

In addition to see how your case worked out I am especially concerned on how to implement nominal attributes in SQ alternative, as there will be no prior parameter estimate for the reference values. It is possible to estimate (n-1) parameters only for non-reference attribute levels. If I understand correctly you dealt this with adding the 'require' restriction. Am I right? How can Ngene calculate choice probabilities for SQ alternative as you do not have priors for reference values of dummy coded attributes?

I hope my question make sense ... and thank you very much for your reply.
Anže

***#2 (response)

;alg = mfederov
;require:
sq.atr1 = 7,
sq.atr2 = 0,
sq.atr3 = 0,
sq.atr4 = 0.3,
sq.atr5 = 0,
sq.atr6 = 0
;model:
U(alt1) = b1 * atr1[7,20,34]
+ b2 * atr2[0,5,15]
+ b3.effects[0|0] * atr3[1,2,0] ? 0 = none, 1 = owner, 2 = other
+ b4 * atr4[0.3,5,20]
+ b5.effects[0|0] * atr5[1,2,0] ? 0 = none, 1 = record, 2 = monitoring
+ b6 * atr6[0,150,300,450,600,750,900]
/
U(alt2) = b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6
/
U(sq) = b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6

Note that the SQ alternative uses the exact same parameters as the other alternatives, so the priors for alt1 and alt2 are also used for sq.

If you have other questions for me, please create a separate thread.

***#3
Dear Michiel,

first, thank you very much for responding. I have no messages from JvB. I am not sure I am creating a new thread by this message as I am simply posting a reply to the last post. Please tell me if I am doing this wrong.

I have looked at the code you suggested, and a have a few questions if I may.

1. there is no constant term in the SQ alternative. I may want to add this so that I will be able to estimate the potential aversion/preferences for status-quo?
2. the reference value of atr. 6 (=0) can only occur in the SQ alternative so that the code needs to be slightly changed. I did so, do you think this is ok? (I see resemblance here with the JvB's design in related post)

;alg = mfederov
;require:
sq.atr1 = 7,
sq.atr2 = 0,
sq.atr3 = 0,
sq.atr4 = 0.3,
sq.atr5 = 0,
sq.atr6 = 0
;model:
U(alt1) = b1 * atr1[7,20,34]
+ b2 * atr2[0,5,15]
+ b3.effects[0|0] * atr3[1,2,0] ? 0 = none, 1 = owner, 2 = other
+ b4 * atr4[0.3,5,20]
+ b5.effects[0|0] * atr5[1,2,0] ? 0 = none, 1 = record, 2 = monitoring
+ b6 * atr6[150,300,450,600,750,900]
/
U(alt2) = b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6
/
U(sq) = b0
+ b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6
$

3. the min. number of rows needed for this design. I am a bit confused here as reading your conversation with JvB you were mentioning both no. of rows and no. of choice tasks (sets). The output of Negene are choice tasks (in rows), each having three alternatives, even though the code specifies 'rows'. If I understood correctly you distinguished between both. In some of my previous work I have used these two forms to calculate the min. no.:
- min. n. of choice sets (S): A1+1+(L-1)*A2+1, where A1 is no. of linear effects parameters, A2 no. of non-linear effects parameters (effects coded), and L no. of levels for non-linear effect (Hensher et al. 2005), (in my case (4+1)+(3-1)*2+1=10) and
- min. n. of choice sets (S): K/(J-1) (in my case 8/(2-1)=8).

So this is the number of choice sets? (or am I wrong).

4. I plan to to a pilot survey to estimate the priors. I will use a simple sequential orthogonal fractional factorial design as we have no general idea on priors. (I saw that JvB used an efficient design with estimates from his/her judgement.) I was wondering how is it possible to estimate utilities for SQ alt. and then probabilities as I will have no coefficient estimates for the reference values of effects coded attributes. I best case using a MNL on a pilot data you can estimate N-1 coefficients for those effects, which leaves the reference level with no estimate. Or am I wrong.

Thank you very much again. Your advice comes very precious!
Anže

[Michiel Bliemer]

1. Yes you can add a constant.
2. You should remove sq.atr6=0 because you now defined atr6 without 0, and you need yo use b6 * atr6_sq[0] in the utility function of sq.
3. The number of rows = the number of choice task = design size = S >= K/(J-1) = 9 (including the new constant). I generally multiply this minimum with 2 or 3 to get enough variation in the data.
4. You misunderstand the model you are estimating. Your sq alternatives uses parameters b1...b6 that also appear in alt1 and alt2 of your utility functions. So all parameters are estimated and you can calculate the utilities, and hence choice probabilities, for alt1, alt2, and sq.

Michiel

[AnzeJap]

Dear Michiel,

thank you very much for your help. I really appreciate it. I took into consideration points 1, 2 and 3 and adjusted the code accordingly (I hope correctly). If you can take a look if it seems OK. (just provisionally I added fictitious priors just to run and check the code.)

Thank you also for no.4. And, I was wondering - reading different posts I saw it's quite standard to use efficient design for both, the pilot and main study. Is it still acceptable to go as I planned - an seq. orthogonal fractional factorial design for the pilot and an efficient one for the main study? Attributes and their level being the same.

I saw something interesting and I cannot understand why. If I would change the priors to higher values, like 0.1, 0.2 etc. I am getting a "MNL-D error 'Undefined" warning in my output window. In some of my previous research I got higher values of my coefficients, thus I wanted to check - of course there is no direct link because studies are different, but I was just thinking.

Design
;alts=alt1, alt2, sq
;rows=18
;blocks=2
;eff=(mnl,d)
;alg = mfederov
;require:
sq.atr1 = 7,
sq.atr2 = 0,
sq.atr3 = 0,
sq.atr4 = 0.3,
sq.atr5 = 0
;model:
U(alt1) = b1[0.001] * atr1[7,20,34]
+ b2[0.001] * atr2[0,5,15]
+ b3.effects[0.002|0.001] * atr3[1,2,0] ? 0 = none, 1 = owner, 2 = other
+ b4[0.1] * atr4[0.3,5,20]
+ b5.effects[-0.001|-0.002] * atr5[1,2,0] ? 0 = none, 1 = record, 2 = monitoring
+ b6[0.002] * atr6[150,300,450,600,750,900]
/
U(alt2) = b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6
/
U(sq) = b0[-0.0015]
+ b1 * atr1
+ b2 * atr2
+ b3 * atr3
+ b4 * atr4
+ b5 * atr5
+ b6 * atr6_sq[0]
$

Thank you!
Anze

[Michiel Bliemer]

The script looks fine.

I would typically use either a D-efficient design with (near) zero priors for the pilot study, or an orthogonal design. Both are fine.

Priors should not be set manually as it is easy to make mistakes and generate a very INefficient design. If you choose your priors too large, attributes become dominant in your choice model. For example, setting b6 to 0.2 means multiplying 0.2 with a value between 150 and 900, which is VERY LARGE. Parameters always need to be in proportion to the units of the attributes. Your prior 0.002 is already quite large, it yields 0.002*900 = 1.8. Also b4=0.1 is very large since 0.1*20 = 2. If you do not have information about the priors, please use near zero priors, such as 0 or 0.0000001, 0.0000002 and -0.000001.

Michiel

[AnzeJap]

Dear Michiel,

thank you for your answers!

I would like to ask, which could be a stupid, question about the last point. Should it happen I end up with a problem of large priors after the pilot survey (hypothetically speaking, I do not know if this can happen), is it possible to bypass this with having original attribute values replaces by codes (e.g. 1,2,3,4, etc.)? Or, alternatively, would it be OK to simply take the sign of the coefficient from the pilot survey, ignore the value, and use a close-to-zero value instead?

Can you maybe direct me to a reference on this issue, so I can read some more.

And, thank you again!
Anže

[Michiel Bliemer]

You will not get too large priors after the pilot survey. Parameters automatically scale with the unit, so if you divide all attribute levels by 10 then the parameter will be 10x as large. If you replace the attribute levels with design coding then again the parameters automatically scale. What matters is that the parameter is not too large RELATIVE TO THE ATTRIBUTE LEVELS. In choice models, only utilities matter, so only beta*X (the contribution to utility) matters. If you use pilot data, you need not worry about any scaling.

I am not aware of any specific literature, this insight comes from basic utility theory.

Michiel

[AnzeJap]

Thank you!

Referring to my pilot orthogonal design - does it make any difference if I use design coding or original level values?

[Michiel Bliemer]

If you use zero priors, then there is no difference in using design coding or estimating coding with actual attribute levels. With zero priors, an efficient design becomes more similar to an orthogonal design.

[AnzeJap]

OK, so if understand correctly it does not make a difference if I use either:
atr6[150,300,450,600,750,900] or atr6[1,2,3,4,5,6] in an orthogonal design? If I understood chapter 6.1.3 on orthogonal designs correctly. (yes, I remember ch. 7 of Ngene manual saying this if using zero priors in efficient design)

This is the code for my pilot study:

Design ;alts=alt1, alt2, alt3
;rows=18
;orth=seq
;block=2
;model:
U(alt1)=b01+b1*atr1[0,1,2]+b2*atr2[0,1,2]+b3.effects[0|0]*atr3[0,1,2]+b4*atr4[0,1,2]+b5.effects[0|0]*atr5[0,1,2]+b6*atr6[1,2,3,4,5,6]/
U(alt2)=b01+b1*atr1 +b2*atr2 +b3*atr3 +b4*atr4 +b5*atr5 +b6*atr6;$

I am sorry for bothering you with probably very basic questions.

[Michiel Bliemer]

Orthogonal designs do not use priors, they are ignored. Orthogonal designs also ignore status quo or opt-out alternatives and orthogonal designs also ignore dummy and effects coding.