Importing a .csv design for evaluation

This forum is for posts that specifically focus on Ngene.

Moderators: Andrew Collins, Michiel Bliemer, johnr

Importing a .csv design for evaluation

Postby rebekah_hall » Wed Jan 20, 2021 7:59 am

Hello,

I am trying to evaluate an experimental design used in a pilot study from an imported .csv file but keep getting the following error:
"ERROR: The provided design contains a row number that exceeds the design specification."

I have tried both with/without the headings row and also adjusting the ;rows code to see if it was some kind of glitch but the error remains. Please could I get some advice on the correct format for .csv files or if there is an error in the code I am trying to use.

This is the code I am trying to use for the evaluation:

Code: Select all
design
;alts = alt1*, alt2
;rows = 12
;block= 2
;eval = women pilot experimental design_final_design only.csv
;eff = (mnl, d)
;model :
U(alt1)=  bsens[0.1073] * sens[65,75,85,95] + btime[-0.5449] * time[1,2,3,4] + bconditions.dummy[0.6693] * conditions[1,0] + bcomm.dummy[1.0076|1.2203] * comm[1,2,0] /
U(alt2)=  bsens         * sens              + btime          * time          + bconditions.dummy         * conditions      + bcomm.dummy                * comm     
$


This is the design I am trying to evaluate loaded in to ngene:

Image

Thanks,
Rebekah
rebekah_hall
 
Posts: 4
Joined: Sun Jan 17, 2021 2:50 am

Re: Importing a .csv design for evaluation

Postby Michiel Bliemer » Wed Jan 20, 2021 10:52 am

You need to add another column to the spreadsheet on the left of choice situation, namely the first column should indicate a respondent number. In your case, it should contain a 1 in each row, so please create a column with a header (I usually use Resp for respondent) and then have 12 ones.

The reason for this column is that Ngene is capable of generating multiple designs for different respondents, also referred to as heterogeneous designs, where the first column indicates designs for the first respondent, the second respondent, etc. In your case, you have a homogeneous design, therefore only need values for a single respondent.

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm

Re: Importing a .csv design for evaluation

Postby rebekah_hall » Thu Jan 21, 2021 2:23 am

Hi Michiel,

Thanks for your help, this has now worked.

I have another question if you could offer any insights. I am trying to update the design to a Bayesian efficient design using the priors from the pilot but the s-estimates I am producing are very high. Having checked the results from each draw sobol draws seems to produce the best results but the sample size estimate is still ~4000 (mean) with halton draws giving estimates >100,000. Is there a reason why the estimates are so high?


Code: Select all
design
;alts = alt1*, alt2*
;rows = 12
;block= 2
;eff = (mnl,d,median)
;bdraws= sobol(100)
;model :
U(alt1)= bsens[(n,0.1073,0.0545)] * sens[65,75,85,95] + btime[(n,-0.5449,0.436)] * time[1,2,3,4] + bconditions.dummy[(n,0.6693,1.04)] * conditions[1,0] + bcomm.dummy[(n,1.0076,1.459)|(n,1.2203,1.4275)] * comm[1,2,0] /
U(alt2)= bsens                    * sens              + btime                    * time          + bconditions.dummy         * conditions      + bcomm.dummy                                     * comm       
$


Thanks,
Rebekah
rebekah_hall
 
Posts: 4
Joined: Sun Jan 17, 2021 2:50 am

Re: Importing a .csv design for evaluation

Postby Michiel Bliemer » Thu Jan 21, 2021 9:58 am

If you open the design and click on Design properties, MNL, and then tick the box Efficiency measures by Bayesian draws, you can see the Sobol draws and the resulting S-estimate. What I observe is that some draws lead to extremely high S-estimates, and averaging over them therefore results in a very high mean S-error. A very large S-estimate happens when one of the draws for the dummy coded coefficients is very close to zero. Clearly, when a coefficient is close to zero (i.e., the atttribute is not an important factor in choice), then it is very difficult to estimate this coefficient. Draws close to zero often happen if the standard deviations (obtained from standard errors in estimation) are relatively large.

The solution is that you should not look at mean S-errors as they are sensitive to outliers. You should look at median S-errors. You also correctly specify median in the efficiency measure, and therefore you should only look at the median column in Ngene, not the mean column. The median S-error is below 100, which gives you a much better idea about the minimum sample size needed (for statistically signficant parameter estimates).

Michiel
Michiel Bliemer
 
Posts: 1705
Joined: Tue Mar 31, 2009 4:13 pm


Return to Choice experiments - Ngene

Who is online

Users browsing this forum: No registered users and 8 guests