Understanding S-sample output

[megoded@us.es]

Dear Ngene users+administrators.

I am using a D-efficient Bayesian design with priors obtained following Bliemer et al (2016). Below the code

Code: Select all

Design
;alts=A*,B*, SQ

;rows = 32

;eff = (mnl,d, mean)

;bdraws=sobol(2000)
;bseed = 12345

;block=8,minsum,noimprov(60 secs)

;alg=mfederov

;model:

U(A) =  baesc[(n,0.142,0.064)]*pesc[1,0] + bagest.dummy[(n,0.231,0.023)|(n,0.124,0.018)]*agest[2,1,0] + baprice[(n,-1.775,0.888)]*aprice[0.03,0.09,0.15,0.21,0.27,0.33] +baint[0.037]*agest.dummy[2]*pesc/
U(B) =  baesc*pesc + bagest.dummy*agest + baprice*aprice+baint*agest.dummy[2]*pesc /
U(SQ) = b0[(n,-0.1,0.05)]

$


I have 3 questions regarding the Sample size (S) indicator:

1) THe attribute "esc" has an extremely high "Sb mean estimate". This usually is explained by a low beta prior estimate (and/or high standard error). However it is not the case in our design (compared to the rest of parameters). Why it can be explained?
2) In my output the Sb mean t-ratios are always lower than 1.96 . I expected those ratios to be higher than 1.96 as the S-estimate are calculated to have significant results (therefore higher than 1.96)
3) Can you confirm that ALWAYS the S-estimate will be higher in the bayesian design than when it is considered fixed priors?. I guess it make sense as you had uncertainty.

Thanks for having active this forum that it is extremely helpful!

Maria E.

Thanks agin

[Michiel Bliemer]

Hi Maria,

1. Your prior for esc is (n,0.142,0.064), where the mean prior is relative small AND the standard deviation of the prior is relatively large. When drawing from this distribution, many draws for parameter baesc will therefore be close to 0, which results for these draws in very large sample size estimates. You can see this when you check under "Design properties, MNL" and then click on "Efficiency measures by Bayesian draw", if you scroll down the list you can see some draws for baesc very close to zero, resulting in a very large S-estimate. For example, draw 118 on my screen is -0.003039, which results in an S-estimate of 211,210.8. When taking the average over all draws, this average will therefore be large due to these outliers. Ngene currently does not report the median S-estimate of individual parameter estimates, these would be much smaller in this case and probably worth reporting in a future version of Ngene.

2. The t-ratios indicate the the t-value if ONLY A SINGLE RESPONDENT would be given all (32) choice tasks. Therefore, if the t-ratio is 1.96, then the sample size estimate would be 1. The t-ratios are much smaller than 1.96, indicating that the sample size estimate is much larger than 1.

3. The overall sample size for all parameters will always be larger in a Bayesian efficient design compared to a locally efficient design, that is the 'price' one pays for generating a design that is more robust against prior misspecificaiton. However, sample size estimates of individual parameters MAY be lower in some cases, because of differences in the design, but overall the sample size estimates will be larger.

Michiel

[megoded@us.es]

Thanks Michiel for your reply.

Regarding 1) if we compare the mean/sd of the parameter for pesc (n,0.142,0.064) with the second level of the parameter for agest (n,0.124,0.018) the mean and the ratio mean/sd is lower in agest. This is why it was surprising for me to see such a high sample size for pesc and not for the second level of agest.

Gracias!

Maria E

[Michiel Bliemer]

Actually the ratio of mean/sd is much larger for agest, namely 0.124 / 0.018 = 6.889, versus 0.142/0.064 = 2.219 for pesc. So the prior of pesc is much more uncertain.

[megoded@us.es]

Thanks Michiel. Clear now!

M;