Sign of prior from pilot and assumption not matcing

This forum is for posts that specifically focus on Ngene.

Moderators: Andrew Collins, Michiel Bliemer, johnr

Sign of prior from pilot and assumption not matcing

Postby apk0022 » Thu Oct 31, 2024 3:20 am

Hi,
I conducted the pilot study with 30 participants. I also conducted MNL to obtain the estimates and their standard errors.
However, the signs of 2 of the attributes are not in the direction we assumed. Those attributes are:
1. Prediction of clinical outcomes for a diagnostic test- we got a positive sign for the 'No' level estimate (keeping 'yes' as baseline)
2. Test invasiveness: we got a negative sign for the 'No' level estimate ('yes' is the baseline)

Now, the issue is while designing the Bayesian efficient design, can I change the sign of these priors and use the expected signs in my equation?

What should we normally do in such situations? What might be possible reasons for this to happen?

Here is my code for the Bayesian design, Could you kindly look through it for appropriateness?

Design
;alts = alt1*, alt2* ? the asterisk (*) after an alternative means they are unlabelled. It also ensures that NGENE checks for dominance and balance
;rows = 36
;block = 4
;eff = (mnl, d, mean)
;alg=mfederov
;bdraws = sobol(1000)
;model :
U(alt1) =
b1.dummy[(n,-0.19,0.32)|(n,-0.46,0.31)|(n,-0.13,0.28)]*Fastingtime[2, 3, 4, 1] +
b2.dummy[(n,-0.36, 0.16)]*Invasiveness[2, 1] +
b3.dummy[(n,0.29,0.23)|(n,0.75,0.23)]*Sensitivity[2, 3, 1] + ? The base level must always be at the end of the bracket
b4.dummy[(n,0.62,0.19)|(n,0.96,0.19)]*Specificity[2, 3, 1] +
b5.dummy[(n,0.08,0.16)]*Prediction[2, 1] +
b6.dummy[(n,-0.27,0.24)|(n,-0.48,0.24)|(n,-0.90,0.33)]*Cost[2, 3, 4, 1]
/
U(alt2) = asca[(n,-0.52,0.20)] +
b1.dummy*Fastingtime[2, 3, 4, 1] +
b2.dummy*Invasiveness[2, 1] +
b3.dummy*Sensitivity[2, 3, 1] +
b4.dummy*Specificity[2, 3, 1] +
b5.dummy*Prediction[2, 1] +
b6.dummy*Cost[2, 3, 4, 1]
$

Thank you in advance.
Asmita
apk0022
 
Posts: 4
Joined: Sat Oct 05, 2024 2:51 am

Re: Sign of prior from pilot and assumption not matcing

Postby apk0022 » Thu Oct 31, 2024 5:05 am

Also, additional information for the above situation, the estimate of the attribute 'prediction' is not statistically significant but for 'invasiveness', it is statistically significant. However, the direction of the priors for both of them is not what we expected it to be.
apk0022
 
Posts: 4
Joined: Sat Oct 05, 2024 2:51 am

Re: Sign of prior from pilot and assumption not matcing

Postby Michiel Bliemer » Thu Oct 31, 2024 7:42 pm

You are estimating quite a larger number of parameters (13) with only a sample size of 30, so that means that your parameter estimates will not be very reliable. This means that unexpected signs can occur, especially for attributes that are less important because respondents may have focused on other attributes and ignored the less important attributes. Perhaps "prediction" and "test invasiveness" are less important than other attributes. For your priors, you simply adjust them such that the signs make sense. If you know that prediction needs to have a negative prior for No, then you could use a uniform prior such as (u,-0.08,0).

Michiel
Michiel Bliemer
 
Posts: 1879
Joined: Tue Mar 31, 2009 4:13 pm


Return to Choice experiments - Ngene

Who is online

Users browsing this forum: Yuchenj and 13 guests