Memo: Does Q28 retain predictive lift after car access?

Research memorandum — nested Q28 | car Random Forests on a common complete-case frame

Author

Jack J. Burleson

Published

Invalid Date

Research question: Does ride-share days (Q28) retain predictive lift for regular transit after conditioning on car access (Q21 / Q20)?

Parent: q27_q28_predict_transit.qmd · Companions: car_access_predicts_transit.qmd · ca_mobility_joint_predicts_transit.qmd

1 Answer, Response, + Summary of Results

We restrict to respondents with complete Q28 + Q20 + Q21 (n = 149; 58 regular) so nested models share one sample, then compare stratified CV Random Forests ((Breiman, 2001); seed=42).

Short answer: Yes—Q28 retains lift, and car access adds incremental discrimination. On the common frame, Q28 alone AUC ≈ 0.665; Q28+Q21 rises to ≈ 0.730 (+0.066). Q21 alone is ≈ 0.631 (Figure Figure 1).

Figure 1.. Nested Q28|car AUC comparison

1.1 Nested CV performance (common n = 149)

Table Table 1 reports the nested CV performance on the common sample.

Table 1.. Q28 conditioned on car access.
Model ROC-AUC Avg. precision Balanced acc. F1
Q28 + Q21 0.730 0.580 0.746 0.701
Q28 + Q20 + Q21 0.727 — — —
Q28 only 0.665 — — —
Q21 only 0.631 — — —
Chance 0.500 — 0.500 —

Permutation importance in Q28+Q21: Q28 mean \(\Delta\)AUC ≈ 0.15 · Q21 ≈ 0.13.

1.2 Interpretation

  1. Directly answers the Q27/Q28 memo’s open question: Q28 is not merely a proxy for lacking a car—both cues matter on the overlapping sample.
  2. Full-cohort Q28 AUC (≈0.762 on n=241) shrinks to 0.665 when forced onto car-complete rows; missingness still matters for absolute AUC, but the relative Q28>car ordering holds.
  3. Adding license (Q20) on top of Q28+Q21 does not help (0.727 ≈ 0.730).

Sources: ca-personas followup-experiments --experiments nested_q28_car · outputs/followup_experiments/nested_q28_car/ · formal write-up docs/secondary_rq_followup_experiments.md

2 What questions or uncertainties remain?

Would multiple imputation of Q20/Q21 recover the full-cohort Q28 AUC while preserving the incremental car effect?

References

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324