Constrained designs

[CMA]

Please refer to page 156 of the Ngene 1.4 USER MANUAL & REFERENCE GUIDE.
> In order to avoid designs with choice situations that are not feasible, Ngene allows constraints to be put on the attribute levels.
Do you have any limitations or concerns when you present a literature using this method?
I think there is a problem with arbitrarily excluding combinations.
If there is another way, please let me know.

[Michiel Bliemer]

There are no concerns with excluding attribute level combinations that are unrealistic or infeasible. Also in revealed preference data you will never be able to observe infeasible attribute level combinations, so you can certainly also exclude them in stated preference data.
Including choice task with unrealistic attribute level combinations can lead to biased results as people do not understand the profiles or do not take the choice experiment seriously.

Note that the manual does not mention "arbitrarily" excluding combinations. Of course any constraints should not create perfectly correlated attributes or make the model unidentifiable.

Michiel

[CMA]

Thank you for your reply.
If you know of any previous studies that exclude unrealistic or unfeasible combinations of attribute levels, please let me know.

[Michiel Bliemer]

I would say that almost half of the Ngene scripts posted here on the forum use constraints. Have a look at the many posts on this forum that impose ;cond or ;reject constraints in order to create realistic choice tasks.

Here is an example of one of my papers, see the Ngene script below where many reject constraints were imposed to create realistic choice tasks.
https://www.emerald.com/insight/content/doi/10.1108/IJPDLM-01-2021-0013/full/html

Code: Select all

design 
;alts = postie*, drone*, locker
;con
;rows = 20
;eff = (mnl,d,mean)
;bdraws = sobol(2000)
;alg = mfederov(candidates = 5000)

;require:
postie.METRO = drone.METRO                              ? scenario variable

;reject:
postie.TIME = 1, postie.TIME = 2,                       ? postie delivery time cannot be within 2 hours or same day
locker.TIME = 1, locker.TIME = 2,                       ? locker delivery time cannot be within 2 hours or same day
postie.WINDOW = 1, postie.WINDOW = 2,                   ? postie cannot deliver in 30-minute or 1-hour window
drone.WINDOW = 4,                                       ? drone cannot deliver in evening
drone.METHOD = 1,                                       ? drone cannot require signature
postie.COST = 2, postie.COST = 4,                       ? postie cannot have a low cost
locker.COST = 2, locker.COST = 4,                       ? locker cannot have a low cost
drone.COST = 16, drone.COST = 18,                       ? drone cannot have a high cost

? postie express service cannot have low cost
postie.TIME = 1 and postie.COST = 6, postie.TIME = 1 and postie.COST = 8, postie.TIME = 1 and postie.COST = 10,
postie.TIME = 2 and postie.COST = 6, postie.TIME = 2 and postie.COST = 8, postie.TIME = 2 and postie.COST = 10,
postie.TIME = 3 and postie.COST = 6, postie.TIME = 3 and postie.COST = 8, postie.TIME = 3 and postie.COST = 10,

? postie regular service cannot have high cost
postie.TIME = 4 and postie.COST = 14, postie.TIME = 4 and postie.COST = 16, postie.TIME = 4 and postie.COST = 18,
postie.TIME = 5 and postie.COST = 14, postie.TIME = 5 and postie.COST = 16, postie.TIME = 5 and postie.COST = 18,
postie.TIME = 0 and postie.COST = 14, postie.TIME = 0 and postie.COST = 16, postie.TIME = 0 and postie.COST = 18,

? drone express service cannot have low cost
drone.TIME = 1 and drone.COST = 2, drone.TIME = 1 and drone.COST = 4, drone.TIME = 1 and drone.COST = 6,
drone.TIME = 2 and drone.COST = 2, drone.TIME = 2 and drone.COST = 4, drone.TIME = 2 and drone.COST = 6,
drone.TIME = 3 and drone.COST = 2, drone.TIME = 3 and drone.COST = 4, drone.TIME = 3 and drone.COST = 6,

? drone regular service cannot have high cost
drone.TIME = 4 and drone.COST = 10, drone.TIME = 4 and drone.COST = 12, drone.TIME = 4 and drone.COST = 14,
drone.TIME = 5 and drone.COST = 10, drone.TIME = 5 and drone.COST = 12, drone.TIME = 5 and drone.COST = 14,
drone.TIME = 0 and drone.COST = 10, drone.TIME = 0 and drone.COST = 12, drone.TIME = 0 and drone.COST = 14,

? locker express service cannot have low cost
locker.TIME = 1 and locker.COST = 6, locker.TIME = 1 and locker.COST = 8, locker.TIME = 1 and locker.COST = 10,
locker.TIME = 2 and locker.COST = 6, locker.TIME = 2 and locker.COST = 8, locker.TIME = 2 and locker.COST = 10,
locker.TIME = 3 and locker.COST = 6, locker.TIME = 3 and locker.COST = 8, locker.TIME = 3 and locker.COST = 10,

? locker regular service cannot have high cost
locker.TIME = 4 and locker.COST = 14, locker.TIME = 4 and locker.COST = 16, locker.TIME = 4 and locker.COST = 18,
locker.TIME = 5 and locker.COST = 14, locker.TIME = 5 and locker.COST = 16, locker.TIME = 5 and locker.COST = 18,
locker.TIME = 0 and locker.COST = 14, locker.TIME = 0 and locker.COST = 16, locker.TIME = 0 and locker.COST = 18,

? express post is only available in metro areas
postie.TIME = 3 and postie.METRO = 0,
drone.TIME = 1 and drone.METRO = 0,
drone.TIME = 2 and drone.METRO = 0,
drone.TIME = 3 and drone.METRO = 0,
locker.TIME = 3 and postie.METRO = 0

;model:
U(postie) = postie[(n,0.9936,0.2087)]
          + metropostie[(n,-0.5852,0.2004)] * METRO[1,0](10,10)  ? each level needs to appear 10 times
                                                             ? 1 = metropolitan area
                                                             ? 0 = rural area
          + time.dummy[(n,0.5791,0.2797)|(n,0.5131,0.2725)|(n,0.1113,0.1468)|(n,-0.2511,0.1177)|(n,-0.2861,0.1217)] * TIME[1,2,3,5,0,4] 
                                                             ? 1 = within 2 hours
                                                             ? 2 = same day
                                                             ? 3 = next day
                                                             ? 4 = 2 business days (base)
                                                             ? 5 = 3 business days
                                                             ? 0 = 5 business days
          + method.dummy[(n,0.2281,0.1497)|(n,0.2995,0.1051)] * METHOD[1,0,2]
                                                             ? 1 = signature required (delivery at door or post-office)
                                                             ? 2 = leave it at the front door (base)
                                                             ? 0 = leave it at a safe place
          + window.dummy[(n,0.3681,0.2403)|(n,0.0055,0.2381)|(n,-0.0789,0.1414)|(n,-0.0558,0.1694)] * WINDOW[1,2,3,4,0]
                                                             ? 1 = choice of 30-minute time window, daytime (9am-5pm)
                                                             ? 2 = 1-hour window, daytime (9am-5pm)
                                                             ? 3 = 2-hour window, daytime (9am-5pm)
                                                             ? 4 = 3-hour window, evening (6pm-9pm)
                                                             ? 0 = no choice, daytime (9am-5pm) (base)
          + cost[(n,-0.1568,0.0126)] * COST[2,4,6,8,10,12,14,16,18]
                                                             ? cost in A$
          /
U(drone)  = drone[(n,-0.2625,0.2258)]
          + metrodrone[(n,-0.3965,0.2432)] * METRO
          + time * TIME
          + method * METHOD
          + window * WINDOW
          + cost * COST
          /
U(locker) = time * TIME
          + cost * COST
$


Michiel

[CMA]

Thank you for your response.
I understood that it's not a problem to exclude realistic choice tasks.
Let me check again, just in case.
This is the code I plan to experiment with n=100.
If this is the case, please let me know how many can be excluded without destroying the experimental design.

Code: Select all

Design
;alts = alt1*, alt2*
;rows = 18
;eff = (mnl, d)
;alg = mfederov(candidates=1088)
;reject:
?alt1
alt1.a = 0 and alt1.b = 2,
alt1.a = 1 and alt1.b = 3,
alt1.a = 1 and alt1.b = 4,
alt1.a = 2 and alt1.b = 2,
alt1.a = 2 and alt1.b = 3,
alt1.a = 2 and alt1.b = 4,
alt1.a = 3 and alt1.b = 2,
alt1.a = 3 and alt1.b = 3,
alt1.a = 3 and alt1.b = 4,
?alt2
alt2.a = 0 and alt2.b = 2,
alt2.a = 1 and alt2.b = 3,
alt2.a = 1 and alt2.b = 4,
alt2.a = 2 and alt2.b = 2,
alt2.a = 2 and alt2.b = 3,
alt2.a = 2 and alt2.b = 4,
alt2.a = 3 and alt2.b = 2,
alt2.a = 3 and alt2.b = 3,
alt2.a = 3 and alt2.b = 4
;model:
U(alt1) = b1.dummy[0|0|0]   * A[1,2,3,0]
        + b2.dummy[0|0|0|0] * B[1,2,3,4,0]
        + b3.dummy[0|0]     * C[1,2,0]
        /
U(alt2) = b1                * A
        + b2                * B
        + b3                * C
$

[Michiel Bliemer]

These constraints are not a problem. As long as the D-error is not very large or Undefined (=infinite) you can estimate the model and the design will work. The D-error will become Undefined if not all coefficients in the model are identifiable. This would happen if the constraints are such that two attributes become perfectly correlated, resulting in multicollinearity.

Michiel

[CMA]

When I create a choice set with constraints, it becomes difficult to balance and the D-error does not get smaller.
In general, I understand that the best D-error is 0.1 or less, but is there any problem with 0.5 or so?
Also, some papers write D-errors, while others don't. Which is more common?

[Michiel Bliemer]

D-errors are not comparable across models (which includes utility specification, priors, and model type). D-errors can only be compared across designs for the same model. This is similar to LL values in model estimation that can only be compared across different models for the same data set.

You can report a D-error, but it is in most cases meaningless, unless you compare multiple designs.

A D-error of 0.5 can be good or bad, it depends on the model. For a model with many dummy variables it is probably quite good, whereas for a model with all numerical attributes with wide attribute level range it is probably not so good. There is not really a reference value that you can use.

Michiel