Importing a .csv design for evaluation

[rebekah_hall]

Hello,

I am trying to evaluate an experimental design used in a pilot study from an imported .csv file but keep getting the following error:
"ERROR: The provided design contains a row number that exceeds the design specification."

I have tried both with/without the headings row and also adjusting the ;rows code to see if it was some kind of glitch but the error remains. Please could I get some advice on the correct format for .csv files or if there is an error in the code I am trying to use.

This is the code I am trying to use for the evaluation:

Code: Select all

design
;alts = alt1*, alt2
;rows = 12
;block= 2
;eval = women pilot experimental design_final_design only.csv
;eff = (mnl, d)
;model :
U(alt1)=  bsens[0.1073] * sens[65,75,85,95] + btime[-0.5449] * time[1,2,3,4] + bconditions.dummy[0.6693] * conditions[1,0] + bcomm.dummy[1.0076|1.2203] * comm[1,2,0] /
U(alt2)=  bsens         * sens              + btime          * time          + bconditions.dummy         * conditions      + bcomm.dummy                * comm     
$

This is the design I am trying to evaluate loaded in to ngene:



Thanks,
Rebekah

[Michiel Bliemer]

You need to add another column to the spreadsheet on the left of choice situation, namely the first column should indicate a respondent number. In your case, it should contain a 1 in each row, so please create a column with a header (I usually use Resp for respondent) and then have 12 ones.

The reason for this column is that Ngene is capable of generating multiple designs for different respondents, also referred to as heterogeneous designs, where the first column indicates designs for the first respondent, the second respondent, etc. In your case, you have a homogeneous design, therefore only need values for a single respondent.

Michiel

[rebekah_hall]

Hi Michiel,

Thanks for your help, this has now worked.

I have another question if you could offer any insights. I am trying to update the design to a Bayesian efficient design using the priors from the pilot but the s-estimates I am producing are very high. Having checked the results from each draw sobol draws seems to produce the best results but the sample size estimate is still ~4000 (mean) with halton draws giving estimates >100,000. Is there a reason why the estimates are so high?

Code: Select all

design
;alts = alt1*, alt2*
;rows = 12
;block= 2
;eff = (mnl,d,median)
;bdraws= sobol(100)
;model :
U(alt1)= bsens[(n,0.1073,0.0545)] * sens[65,75,85,95] + btime[(n,-0.5449,0.436)] * time[1,2,3,4] + bconditions.dummy[(n,0.6693,1.04)] * conditions[1,0] + bcomm.dummy[(n,1.0076,1.459)|(n,1.2203,1.4275)] * comm[1,2,0] /
U(alt2)= bsens                    * sens              + btime                    * time          + bconditions.dummy         * conditions      + bcomm.dummy                                     * comm       
$


Thanks,
Rebekah

[Michiel Bliemer]

If you open the design and click on Design properties, MNL, and then tick the box Efficiency measures by Bayesian draws, you can see the Sobol draws and the resulting S-estimate. What I observe is that some draws lead to extremely high S-estimates, and averaging over them therefore results in a very high mean S-error. A very large S-estimate happens when one of the draws for the dummy coded coefficients is very close to zero. Clearly, when a coefficient is close to zero (i.e., the atttribute is not an important factor in choice), then it is very difficult to estimate this coefficient. Draws close to zero often happen if the standard deviations (obtained from standard errors in estimation) are relatively large.

The solution is that you should not look at mean S-errors as they are sensitive to outliers. You should look at median S-errors. You also correctly specify median in the efficiency measure, and therefore you should only look at the median column in Ngene, not the mean column. The median S-error is below 100, which gives you a much better idea about the minimum sample size needed (for statistically signficant parameter estimates).

Michiel