Second choice with some NEW attributes

[peyman_07]

Dear Michiel,

I have defined some attributes based on the real context and some others based on a hypothetical scenario. The idea is to show the real attributes and ask respondents to select an alternative and then show some new attributes while the previous attribute levels and alternatives are the same as the first choice to investigate how the new setting may influence their second choice. Could you please let me know how I can design such a survey?

Thanks for your support.
Best,
Peyman.

[Michiel Bliemer]

You could first create a design with only existing attributes, call this design X. This can be an efficient or orthogonal design.
Then for the second set of choice tasks you can add additional columns to include new attributes. Optimising this design for efficiency is not trivial and would require programming and solving an optimisation problem, but you can also choose to simply add orthogonal columns to design X to create the second set of choice tasks.

Michiel

[peyman_07]

Thanks, Michiel.

So, let's assume I create design X with only existing attributes using efficient design in Ngene. Then, I create design Y with only hypothetical attributes using orthogonal design in Ngene. In the choice task, I would firstly show design X. After the first choice, I could add design Y (hypothetical) to design X(existing) and ask them to make their second choice. Is that right?

May this method (without extra optimization effort) have any risk in terms of parameter estimation?

Best,
Peyman.

[Michiel Bliemer]

Yes that is right. I do not think that there is much risk involved. In case you want to make sure that you can estimate all parameters, you can simply put the entire design in an Excel spreadsheet and evaluate it in Ngene. If the design has a finite D-error (generally lower than 1) then all parameters can be estimated.

Another way of doing this is to create an efficient design X (using zero or non-zero priors) and then create a separate efficient design Y (with zero priors) only containing the new attributes, and then do the same as I proposed earlier, combine X and Y for the second questions. If both designs are efficient with a finite D-error, then also the combination will have a finite D-error and there is no risk. So instead of adding orthogonal columns you can add columns from an efficient design with zero priors.

Michiel

[peyman_07]

Hi Michiel,

Many thanks for your support.
1- I just did not get one point. How can I evaluate an Excel spreadsheet in Ngene? May I import it in Ngene? Does the Excel file require a particular format?
In the case of importing, would the utility functions include all the attributes?

2- Could you please let me know the advantages and disadvantages of this method of presenting the attributes to respondents? Is there any paper using this approach?

3- Would the final models be estimated using MNL? One model for all attributes and one for the real ones?

4- Please note that the sample size would be 400 at maximum. How can I ensure the parameters can be estimated with this sample size?

Best,
Peyman.

[Michiel Bliemer]

1. I refer to the Ngene manual, see the description of ;eval on page 217, which explains the format in Excel. You simply open the Excel/CSV file in Ngene and then read it in, e.g. ;eval = filename.csv

2. I am not aware of any paper, but if you want to compare choices in the first and second part, then you want the design to be the same as much as possible, therefore keeping the attribute levels the same for the existing attributes seems best. I have no further information.

3. You can estimate a single MNL model on both datasets simultaneously.

4. You conduct a pilot study, obtain parameter estimates, use these as priors, and compute sample size estimates (S-estimates in Ngene), they will give you an indication of what sample size you would need for statistically significant parameter estimates. Of course this is model specific, for some models you would only need 50 respondents, for others you would need 5000 respondents, so I cannot say if 400 is enough in your case, you can never "ensure" it.

Michiel

[peyman_07]

Thanks Michiel for your reply.

Following the manual instructions in the format of the Excel file, I just tried to evaluate the existing design based on the following code:

design

? Morning design
;alts=W,C,S,H
;rows=24
;block=4
;eff=(mnl,d)

;eval=H:\My Documents\Test.xlsx

;model:

U(W)= w*W[0,10,15,20]+dw_w.dummy[0|0]*DW[1,2,3]+ t_w.dummy[0|0]*T[8,12,16]+na_w.dummy[0]*NA[1,0]+ns_w*NS[2,5,8]/
U(H)= dw_h.dummy[0|0]*DW[DW]+ t_h.dummy[0|0]*T[T]+na_h.dummy[0]*NA[NA]+ns_h*NS[NS]/
U(S)= dw_s.dummy[0|0]*DW[DW]+ t_s.dummy[0|0]*T[T]+na_s.dummy[0]*NA[NA]+ns_s*NS[NS]/
U(C)= ASC0

$

I combined two efficient designs as you previously proposed. The D-error value is 0.38. Does this value mean that the design is efficient?

Thanks a lot.

Best,
Peyman.

[Michiel Bliemer]

D-errors are case-specific. A value of 0.38 can be high or low, there is no universal reference point. It does not look like an unreasonable number.

It is the same as a loglikelihood value in model estimation, you cannot compare the LL across studies, you can only compare LL within the same dataset. The same holds for D-errors, you can only compare D-errors within the same model, but not across models.

Michiel

[Michiel Bliemer]

I was thinking that you could do something like the syntax below in Ngene.

Suppose that attributes A, B, and C are your existing attributes, and that D and E are new attributes. Then you can optimise on both designs at the same time as follows:

Code: Select all

design
;alts(first) = alt1, alt2
;alts(second) = alt1, alt2
;rows = 12
;block = 2
;eff = first(mnl,d) + second(mnl,d)
;model(first):
U(alt1) = b1*A[1,2,3] + b2*B[1,2,3] + b3*C[1,2,3] /
U(alt2) = b1*A + b2*B + b3*C
;model(second):
U(alt1) = b1*A[1,2,3] + b2*B[1,2,3] + b3*C[1,2,3] + b4*D[1,2,3] + b5*E[1,2,3] /
U(alt2) = b1*A + b2*B + b3*C + b4*D + b5*E
$


This will generate a single design that is optimised for both, where you simply remove attributes D and E from the design in the first questions.

Michiel

[peyman_07]

Thanks Michiel for this solution. This is called homogenous design, right?

I had thought about it before but the problem is that I do not know how to put a constraint on some attributes. Apparently, it is not possible to add any constraint in such a design, right?

Best,
Peyman.